Checkpoint/restore of capabilities

Thu Sep 22 10:16:17 CEST 2016

Hello Denis,

On 09/21/2016 05:42 PM, Denis Huber wrote:
> Hello again,
> 
> I have two small problems where I need some guidance from you :)
> 
> 1. I am trying to understand the mechanism of l4_task_map [1]. Are the 
> following thoughts correct?
> 
> * The destination and source task cap (first 2 args of l4_task_map) can 
> be retrieved through Pd_session::native_pd() and Foc_native_pd::task_cap().
> * Send flexpage (arg #3) describes a memory area which contains the 
> selector number (= address) of the source task's capability.
> * The send base (arg #4) is an integer which contains the address of the 
> capability of the the destination task and also an operation code number 
> for e.g. mapping or granting the capability.
> 
> [1] 
> https://l4re.org/doc/group__l4__task__api.html#ga0a883fb598c3320922f0560263da35e6
> 

That is correct.

> To iterate through all possible capabilities I need to know where the 
> capability space starts (first valid selector number) and where it ends. 
> Where can I find these information? I.e. which source files are relevant?

THe capability space of each component is split between an area
controlled by core, and one controlled by the component itself.
Everything underneath Fiasco::USER_BASE_CAP (in file:
repos/base-foc/include/foc/native_capability.h:63) is used by core, and
has the following layout: the first nine slots are reserved to not
interfere with fixed capabilities of Fiasco.OC/L4Re. The only
capabilities of this fixed area that we use are the task capability
(slot 1) and the parent capability (slot 8).
The rest of the core area is divided into thread-local capabilities.
Every thread has three dedicated capabilities: a capability to its own
IPC gate (so to say its identity), a capability to its pager object, and
a capability to an IRQ object (some kind of kernel semaphore), that is
used for blocking in the case of lock-contention. You can find the
layout information again in the file:
repos/base-foc/include/foc/native_capability.h.

Everything starting from slot 200 is controlled by the component itself.
Each component has a capability allocator, and some kind of registry
containing all currently allocated capabilities that is called "cap map":

  repos/base-foc/src/include/base/internal/cap_*
  repos/base-foc/src/lib/base/cap_*

Currently, the per-component capability allocator is (compile-time)
restricted to a number of up to 4K capabilities. The special component
core can allocate more capabilities, because it always owns every
capability in the system.

The capability space controlled by the component thereby ranges from
200-4296, but it is filled sparsely. When not knowing the "cap map" of a
component, you can however check the validity of a single capability
with `l4_task_cap_valid`, have a look here:

https://l4re.org/doc/group__l4__task__api.html#ga829a1b5cb4d5dba33ffee57534a505af

> 
> 2. I also wanted to look up the mechanism of Noux where it 
> re-initializes the parent cap, the noux session cap, and the caps of a 
> child's environment after a fork. But I cannot find the corresponding files.
> 

AFAIK, in Noux the parent capability in the .data section of the program
gets overwritten:

  repos/ports/src/noux/child.h:458
  repos/ports/src/noux/ram_session_component.h:80

After that parts of the main thread initialization of the target needs
to be re-done, otherwise e.g., the serialized form of the parent
capability in the data section would have no effect. But I'm not well up
with respect to Noux initialization. After some grep, I found this being
the first routine executed by the forked process:

  repos/ports/src/lib/libc_noux/plugin.cc:526

It shows up, how parent capability gets set, and the environment gets
re-loaded.

Best regards
Stefan

> 
> Kind regards,
> Denis
> 
> On 10.09.2016 11:52, Denis Huber wrote:
>> Hello Norman,
>>
>> thank you for your great answer. I will follow your advise and
>> virtualize all necessary services that a target component uses.
>>
>>
>> Kind regards,
>> Denis
>>
>> On 09.09.2016 10:58, Norman Feske wrote:
>>> Hi Denis,
>>>
>>>> The child component shall be migrated from one ECU to another. The
>>>> Genode system on the other ECU may have the Rpc_objects, which the child
>>>> needs (e.g. shared dataspaces), but their object identities are
>>>> different (e.g. other addresses in memory) or the Rpc_objects do not
>>>> exist (e.g. a session object between the child and a service).
>>>
>>> so the problem goes much deeper than merely requesting and populating
>>> the child's capability space. You need the replicate the entire child's
>>> execution environment at the destination ECU. That means for each
>>> capability in possession of the child, your runtime needs to know the
>>> exact meaning. E.g., if the child has a session capability to a session
>>> created with certain session arguments, the same kind of session must be
>>> re-created at the destination ECU. Of course, the same holds for all
>>> dataspaces, threads, and other RPC objects that the child can reference
>>> via the capabilities present in its capability space.
>>>
>>> The logical consequence is that the runtime must virtualize all services
>>> used by the child. E.g. if the child creates a LOG session, the runtime
>>> would create a session to a LOG service in the child's name but hand out
>>> a capability locally implemented LOG-session wrapper - similar to what
>>> you have already done for the RAM service. So when migrating the child,
>>> you now exactly what the various capabilities in the child's capability
>>> space mean and can transfer the underlying state to the destination ECU.
>>>
>>> In principle, this is how Noux solves the fork problem. But in the case
>>> of Noux, I deliberately avoid populating the child's capability space
>>> with Genode capabilities in order to alleviate the need to virtualize
>>> many Genode services. Instead, I let the child use the Noux session as
>>> its only interface to the outside world. At the Noux-session level, the
>>> child does not talk about Genode capabilities but about file
>>> descriptors, for which Noux knows the meaning. Of course there exist a
>>> few capabilities in the child's capability space, in particular the
>>> parent cap, the Noux-session cap, and the caps of the child's
>>> environment. But these few capabilities are manually re-initialized by
>>> the freshly created process after the fork.
>>>
>>> In your case, you want to replicate the child's capability space in a
>>> way that is transparent to the child. Like Noux, you need the have a
>>> complete model of the child's execution environment in your runtime.
>>> Unlike Noux, however, you want to let the child interact with various
>>> Genode services. Consequently, your model needs to capture the those
>>> services.
>>>
>>>> During a restore, I will have to relink the Native_capability to the
>>>> available Rpc_object or simply recreate the Rpc_object. In both cases I
>>>> have to know the types of the Native_capabilities, when I snapshot them
>>>> from the Cap Space of the child. Is there a way to find out the type of
>>>> a Native_capability through an API function?
>>>
>>> As discussed above, the type alone does not suffice. Your runtime needs
>>> to know the actual semantics behind each capability, e.g., not just the
>>> knowledge that a certain capability is a RAM-session capability but also
>>> the information how much quota the RAM session has and which dataspaces
>>> belong to it. Or as another example, you don't just need to know that a
>>> capability is a file-system session but also the session arguments that
>>> were used when the session was created.
>>>
>>>> If there is no ready-to-use function/approach, can I intercept the type
>>>> to which a Native_capability is reinterpreted in Rpc_entrypoint::manage
>>>> as a workaround solution?
>>>
>>> Since your runtime needs to create a representative for each RPC object
>>> the child interacts with in the form of a locally implemented RPC object
>>> (managed by the runtime's entrypoint), you can in principle use the
>>> 'Rpc_entrypoint::apply' method to look up the local RPC object for a
>>> given capability.
>>>
>>> Best regards
>>> Norman
>>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> genode-main mailing list
>> genode-main at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/genode-main
>>
> 
> ------------------------------------------------------------------------------
> _______________________________________________
> genode-main mailing list
> genode-main at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/genode-main
> 

-- 
Stefan Kalkowski
Genode Labs

https://github.com/skalk · http://genode.org/