Restoring child with checkpointed state

David Werner wernerd at ...389...
Wed Jun 21 12:12:57 CEST 2017

Hi everyone,

As I'm stuck with this problem I would appreciate any kind of advice.

Best Regards,

Am 07.06.2017 um 15:13 schrieb David Werner:
> Hi everyone,
> after Denis Huber left the project, I am in charge of making our 
> checkpoint/restore component work.
> Therefore i would like to ask some more questions on the IRQ kernel 
> object.
> 1. When is the IRQ object created? Does every component have an own 
> IRQ object?
> I tried to figure out when the IRQ object is mapped into the object 
> space of a component on its startup. Therefore I took a look at the 
> code in [repos/base-foc/src/core/]. The IRQ 
> object appears in the object space after the "_sem = 
> <Rpc_request_semaphore>();" statement in the constructor.
> As far as I could  follow the implementation the "request_semaphore" 
> RPC call is answered by the "Signal_source_rpc_object" in 
> [base-foc/src/include/signal_source/rpc_object.h] which 
> returns/delegates the native capability "_blocking_semaphore" which is 
> an attribute of the "Signal_source_rpc_object". It seems to me that 
> the IRQ object already exists at this point and is only delegated to 
> the component.
> But when is the IRQ object created and by whom? Is it created when a 
> new PD session is created?
> 2. Does the IRQ object carry any information? Do I need to checkpoint 
> this information in order to be able to recreate the object properly 
> during a restore process? Is the IRQ object created automatically (and 
> i only have to make sure that the object is getting mapped into the 
> object space of the target) or do i have to create it manually?
> In our current implementation of the restore process we restore a 
> component by recreating its sessions to core services (+timer) with 
> the help of information we gathered using a custom runtime 
> environment. After the sessions are restored we place them in the 
> object space at the correct position. Will I also have to somehow 
> store information about the IRQ object? Or is it just some object that 
> needs to exist?
> Kind Regards,
> David
> Am 29.03.2017 um 14:05 schrieb Stefan Kalkowski:
>> Hello Dennis,
>> On 03/27/2017 04:14 PM, Denis Huber wrote:
>>> Dear Genode community,
>>> Preliminary: We implemented a Checkpoint/Restore mechanism on basis of
>>> Genode/Fiasco.OC (Thanks to the great help of you all). We store the
>>> state of the target component by monitoring its RPC function calls 
>>> which
>>> go through the parent component (= our Checkpoint/Restore component).
>>> The capability space is indirectly checkpointed through the 
>>> capability map.
>>> The restoring of the state of the target is done by restoring the RPC
>>> objects used by the target component (e.g. PD session, dataspaces,
>>> region maps, etc.). The capabilities of the restored objects have to be
>>> also restored in the capability space (kernel) and in the capability 
>>> map
>>> (userspace).
>>> For restoring the target component Norman suggested the usage of the
>>> Genode::Child constructor with an invalid ROM dataspace capability 
>>> which
>>> does not trigger the bootstrap mechanism. Thus, we have the full 
>>> control
>>> of inserting the capabilities of the restored RPC objects into the
>>> capability space/map.
>>> Our problem is the following: We restore the RPC objects and insert 
>>> them
>>> into the capability map and then in the capability space. From the
>>> kernel point of view these capabilities are all "IPC Gates".
>>> Unfortunately, there was also an IRQ kernel object created by the
>>> bootstrap mechanism. The following table shows the kernel debugger
>>> output of the capability space of the freshly bootstraped target 
>>> component:
>>> 000204 :0016e* Gate   0015f* Gate   00158* Gate   00152* Gate
>>> 000208 :00154* Gate   0017e* Gate   0017f* Gate   00179* Gate
>>> 00020c :00180* Gate   00188* Gate          --            --
>>> 000210 :       --            --     0018a* Gate   0018c* Gate
>>> 000214 :0018e* Gate   00196* Gate   00145* Gate   00144* IRQ
>>> 000218 :00198* Gate          --            --            --
>>> 00021c :       --     0019c* Gate          --            --
>>> At address 000217 you can see the IRQ kernel object. What does this
>>> object do, how can we store/monitor it, and how can it be restored?
>>> Where can we find the source code which creates this object in Genode's
>>> bootstrap code?
>> The IRQ kernel object you refer to is used by the "signal_handler"
>> thread to block for signals of core's corresponding service. It is a
>> base-foc specific internal core RPC object[1] that is used by the signal
>> handler[2] and the related capability gets returned by the call to
>> 'alloc_signal_source()' provided by the PD session[3].
>> I have to admit, I did not follow your current implementation approach
>> in depth. Thereby, I do not know how to exactly handle this specific
>> signal hander thread and its semaphore-like IRQ object, but maybe the
>> references already help you further.
>> Regards
>> Stefan
>> [1] repos/base-foc/src/core/
>> [2] repos/base-foc/src/lib/base/
>> [3] repos/base/src/core/include/pd_session_component.h
>>> Best regards,
>>> Denis
>>> On 11.12.2016 13:01, Denis Huber wrote:
>>>> Hello Norman,
>>>>> What you observe here is the ELF loading of the child's binary. As 
>>>>> part
>>>>> of the 'Child' object, the so-called '_process' member is 
>>>>> constructed.
>>>>> You can find the corresponding code at
>>>>> 'base/src/lib/base/'. The code parses the ELF 
>>>>> executable
>>>>> and loads the program segments, specifically the read-only text 
>>>>> segment
>>>>> and the read-writable data/bss segment. For the latter, a RAM 
>>>>> dataspace
>>>>> is allocated and filled with the content of the ELF binary's data. In
>>>>> your case, when resuming, this procedure is wrong. After all, you 
>>>>> want
>>>>> to supply the checkpointed data to the new child, not the initial 
>>>>> data
>>>>> provided by the ELF binary.
>>>>> Fortunately, I encountered the same problem when implementing fork 
>>>>> for
>>>>> noux. I solved it by letting the 'Child_process' constructor 
>>>>> accept an
>>>>> invalid dataspace capability as ELF argument. This has two effects:
>>>>> First, the ELF loading is skipped (obviously - there is no ELF to 
>>>>> load).
>>>>> And second the creation of the initial thread is skipped as well.
>>>>> In short, by supplying an invalid dataspace capability as binary 
>>>>> for the
>>>>> new child, you avoid all those unwanted operations. The new child 
>>>>> will
>>>>> not start at 'Component::construct'. You will have to manually create
>>>>> and start the threads of the new child via the PD and CPU session
>>>>> interfaces.
>>>> Thank you for the hint. I will try out your approach
>>>>> The approach looks good. I presume that you encounter 
>>>>> base-foc-specific
>>>>> peculiarities of the thread-creation procedure. I would try to follow
>>>>> the code in 'base-foc/src/core/' to see what the
>>>>> interaction of core with the kernel looks like. The order of 
>>>>> operations
>>>>> might be important.
>>>>> One remaining problem may be that - even though you may by able the
>>>>> restore most part of the thread state - the kernel-internal state 
>>>>> cannot
>>>>> be captured. E.g., think of a thread that was blocking in the 
>>>>> kernel via
>>>>> 'l4_ipc_reply_and_wait' when checkpointed. When resumed, the new 
>>>>> thread
>>>>> can naturally not be in this blocking state because the kernel's 
>>>>> state
>>>>> is not part of the checkpointed state. The new thread would possibly
>>>>> start its execution at the instruction pointer of the syscall and 
>>>>> issue
>>>>> system call again, but I am not sure what really happens in practice.
>>>> Is there a way to avoid this situation? Can I postpone the 
>>>> checkpoint by
>>>> letting the entrypoint thread finish the intercepted RPC function 
>>>> call,
>>>> then increment the ip of child's thread to the next command?
>>>>> I think that you don't need the LOG-session quirk if you follow my
>>>>> suggestion to skip the ELF loading for the restored component
>>>>> altogether. Could you give it a try?
>>>> You are right, the LOG-session quirk seems a bit clumsy. I like your
>>>> idea of skipping the ELF loading and automated creation of CPU threads
>>>> more, because it gives me the control to create and start the threads
>>>> from the stored ip and sp.
>>>> Best regards,
>>>> Denis
>>>> ------------------------------------------------------------------------------ 
>>>> Developer Access Program for Intel Xeon Phi Processors
>>>> Access to Intel Xeon Phi processor-based developer platforms.
>>>> With one year of Intel Parallel Studio XE.
>>>> Training and support from Colfax.
>>>> Order your platform today.
>>>> _______________________________________________
>>>> genode-main mailing list
>>>> genode-main at
>>> ------------------------------------------------------------------------------ 
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites,!
>>> _______________________________________________
>>> genode-main mailing list
>>> genode-main at
> ------------------------------------------------------------------------------ 
> Check out the vibrant tech community on one of the world's most
> engaging tech sites,!
> _______________________________________________
> genode-main mailing list
> genode-main at

More information about the users mailing list