Restoring child with checkpointed state
David Werner
wernerd at ...389...
Wed Jun 28 12:53:41 CEST 2017
Hi Stefan,
thank you for your answer!
Am 22.06.2017 um 16:08 schrieb Stefan Kalkowski:
> Hi David,
>
> On 06/21/2017 12:12 PM, David Werner wrote:
>> Hi everyone,
>>
>> As I'm stuck with this problem I would appreciate any kind of advice.
> sorry for the long delay. See below for my comments.
>
No problem. I'm thankful you provided that information.
>>> Hi everyone,
>>>
>>> after Denis Huber left the project, I am in charge of making our
>>> checkpoint/restore component work.
>>> Therefore i would like to ask some more questions on the IRQ kernel
>>> object.
>>>
>>>
>>> 1. When is the IRQ object created? Does every component have an own
>>> IRQ object?
> It is created when the signal source client (separate thread) is
> created. The signal source client is created once while bootstrapping a
> component. It receives pure signals from the corresponding core service
> and delivers the signals locally, e.g.: unblocks an entrypoint that is
> waiting for signals and IPC.
>
>>> I tried to figure out when the IRQ object is mapped into the object
>>> space of a component on its startup. Therefore I took a look at the
>>> code in [repos/base-foc/src/core/signal_source_component.cc]. The IRQ
>>> object appears in the object space after the "_sem =
>>> <Rpc_request_semaphore>();" statement in the constructor.
>>>
>>> As far as I could follow the implementation the "request_semaphore"
>>> RPC call is answered by the "Signal_source_rpc_object" in
>>> [base-foc/src/include/signal_source/rpc_object.h] which
>>> returns/delegates the native capability "_blocking_semaphore" which is
>>> an attribute of the "Signal_source_rpc_object". It seems to me that
>>> the IRQ object already exists at this point and is only delegated to
>>> the component.
>>>
>>> But when is the IRQ object created and by whom? Is it created when a
>>> new PD session is created?
> It is created by core, when a new SIGNAL session is opened to it. This
> is typically done during the startup of a new component. You are right,
> the request_semaphore() call then just transfers the IRQ object's
> capability from core to the requesting component.
>
>>>
>>>
>>> 2. Does the IRQ object carry any information? Do I need to checkpoint
>>> this information in order to be able to recreate the object properly
>>> during a restore process? Is the IRQ object created automatically (and
>>> i only have to make sure that the object is getting mapped into the
>>> object space of the target) or do i have to create it manually?
> The IRQ object does not carry information, but its state changes when a
> thread attaches or detaches from it. So if you re-create that specific
> IRQ object, the signal handler thread that is using the signal source
> client has to attach again to the replaced IRQ object.
This seems to be very convenient for a restore procedure.
> As being said, this specific IRQ object is part of the SIGNAL session
> and its client state. I'm not sure how your restore mechanism works
> exactly, but if work is done within the component that gets restored,
> you can do the re-attachment there. Otherwise, you would need to change
> the request_semaphore call. So that the information of which thread gets
> attached is part of the server side in core. Instead of attaching to the
> IRQ object itself, the signal handler thread transfers its identity to
> core via request_semaphore. Core attaches the thread and delivers the
> capability. Whenever request_semaphore is called, you detach formerly
> attached threads, as well as when the session is closed.
> Does that make sense for you?
>
> Regards
> Stefan
I think i understand what you are describing here and i will try to
modify the request_semaphore call in that way.
Again, thank you!
Kind Regards,
David
>>>
>>> Kind Regards,
>>> David
>>>
>>>
>>> Am 29.03.2017 um 14:05 schrieb Stefan Kalkowski:
>>>> Hello Dennis,
>>>>
>>>> On 03/27/2017 04:14 PM, Denis Huber wrote:
>>>>> Dear Genode community,
>>>>>
>>>>> Preliminary: We implemented a Checkpoint/Restore mechanism on basis of
>>>>> Genode/Fiasco.OC (Thanks to the great help of you all). We store the
>>>>> state of the target component by monitoring its RPC function calls
>>>>> which
>>>>> go through the parent component (= our Checkpoint/Restore component).
>>>>> The capability space is indirectly checkpointed through the
>>>>> capability map.
>>>>> The restoring of the state of the target is done by restoring the RPC
>>>>> objects used by the target component (e.g. PD session, dataspaces,
>>>>> region maps, etc.). The capabilities of the restored objects have to be
>>>>> also restored in the capability space (kernel) and in the capability
>>>>> map
>>>>> (userspace).
>>>>>
>>>>> For restoring the target component Norman suggested the usage of the
>>>>> Genode::Child constructor with an invalid ROM dataspace capability
>>>>> which
>>>>> does not trigger the bootstrap mechanism. Thus, we have the full
>>>>> control
>>>>> of inserting the capabilities of the restored RPC objects into the
>>>>> capability space/map.
>>>>>
>>>>> Our problem is the following: We restore the RPC objects and insert
>>>>> them
>>>>> into the capability map and then in the capability space. From the
>>>>> kernel point of view these capabilities are all "IPC Gates".
>>>>> Unfortunately, there was also an IRQ kernel object created by the
>>>>> bootstrap mechanism. The following table shows the kernel debugger
>>>>> output of the capability space of the freshly bootstraped target
>>>>> component:
>>>>>
>>>>> 000204 :0016e* Gate 0015f* Gate 00158* Gate 00152* Gate
>>>>> 000208 :00154* Gate 0017e* Gate 0017f* Gate 00179* Gate
>>>>> 00020c :00180* Gate 00188* Gate -- --
>>>>> 000210 : -- -- 0018a* Gate 0018c* Gate
>>>>> 000214 :0018e* Gate 00196* Gate 00145* Gate 00144* IRQ
>>>>> 000218 :00198* Gate -- -- --
>>>>> 00021c : -- 0019c* Gate -- --
>>>>>
>>>>> At address 000217 you can see the IRQ kernel object. What does this
>>>>> object do, how can we store/monitor it, and how can it be restored?
>>>>> Where can we find the source code which creates this object in Genode's
>>>>> bootstrap code?
>>>> The IRQ kernel object you refer to is used by the "signal_handler"
>>>> thread to block for signals of core's corresponding service. It is a
>>>> base-foc specific internal core RPC object[1] that is used by the signal
>>>> handler[2] and the related capability gets returned by the call to
>>>> 'alloc_signal_source()' provided by the PD session[3].
>>>>
>>>> I have to admit, I did not follow your current implementation approach
>>>> in depth. Thereby, I do not know how to exactly handle this specific
>>>> signal hander thread and its semaphore-like IRQ object, but maybe the
>>>> references already help you further.
>>>>
>>>> Regards
>>>> Stefan
>>>>
>>>> [1] repos/base-foc/src/core/signal_source_component.cc
>>>> [2] repos/base-foc/src/lib/base/signal_source_client.cc
>>>> [3] repos/base/src/core/include/pd_session_component.h
>>>>> Best regards,
>>>>> Denis
>>>>>
>>>>> On 11.12.2016 13:01, Denis Huber wrote:
>>>>>> Hello Norman,
>>>>>>
>>>>>>> What you observe here is the ELF loading of the child's binary. As
>>>>>>> part
>>>>>>> of the 'Child' object, the so-called '_process' member is
>>>>>>> constructed.
>>>>>>> You can find the corresponding code at
>>>>>>> 'base/src/lib/base/child_process.cc'. The code parses the ELF
>>>>>>> executable
>>>>>>> and loads the program segments, specifically the read-only text
>>>>>>> segment
>>>>>>> and the read-writable data/bss segment. For the latter, a RAM
>>>>>>> dataspace
>>>>>>> is allocated and filled with the content of the ELF binary's data. In
>>>>>>> your case, when resuming, this procedure is wrong. After all, you
>>>>>>> want
>>>>>>> to supply the checkpointed data to the new child, not the initial
>>>>>>> data
>>>>>>> provided by the ELF binary.
>>>>>>>
>>>>>>> Fortunately, I encountered the same problem when implementing fork
>>>>>>> for
>>>>>>> noux. I solved it by letting the 'Child_process' constructor
>>>>>>> accept an
>>>>>>> invalid dataspace capability as ELF argument. This has two effects:
>>>>>>> First, the ELF loading is skipped (obviously - there is no ELF to
>>>>>>> load).
>>>>>>> And second the creation of the initial thread is skipped as well.
>>>>>>>
>>>>>>> In short, by supplying an invalid dataspace capability as binary
>>>>>>> for the
>>>>>>> new child, you avoid all those unwanted operations. The new child
>>>>>>> will
>>>>>>> not start at 'Component::construct'. You will have to manually create
>>>>>>> and start the threads of the new child via the PD and CPU session
>>>>>>> interfaces.
>>>>>> Thank you for the hint. I will try out your approach
>>>>>>
>>>>>>> The approach looks good. I presume that you encounter
>>>>>>> base-foc-specific
>>>>>>> peculiarities of the thread-creation procedure. I would try to follow
>>>>>>> the code in 'base-foc/src/core/platform_thread.cc' to see what the
>>>>>>> interaction of core with the kernel looks like. The order of
>>>>>>> operations
>>>>>>> might be important.
>>>>>>>
>>>>>>> One remaining problem may be that - even though you may by able the
>>>>>>> restore most part of the thread state - the kernel-internal state
>>>>>>> cannot
>>>>>>> be captured. E.g., think of a thread that was blocking in the
>>>>>>> kernel via
>>>>>>> 'l4_ipc_reply_and_wait' when checkpointed. When resumed, the new
>>>>>>> thread
>>>>>>> can naturally not be in this blocking state because the kernel's
>>>>>>> state
>>>>>>> is not part of the checkpointed state. The new thread would possibly
>>>>>>> start its execution at the instruction pointer of the syscall and
>>>>>>> issue
>>>>>>> system call again, but I am not sure what really happens in practice.
>>>>>> Is there a way to avoid this situation? Can I postpone the
>>>>>> checkpoint by
>>>>>> letting the entrypoint thread finish the intercepted RPC function
>>>>>> call,
>>>>>> then increment the ip of child's thread to the next command?
>>>>>>
>>>>>>> I think that you don't need the LOG-session quirk if you follow my
>>>>>>> suggestion to skip the ELF loading for the restored component
>>>>>>> altogether. Could you give it a try?
>>>>>> You are right, the LOG-session quirk seems a bit clumsy. I like your
>>>>>> idea of skipping the ELF loading and automated creation of CPU threads
>>>>>> more, because it gives me the control to create and start the threads
>>>>>> from the stored ip and sp.
>>>>>>
>>>>>>
>>>>>> Best regards,
>>>>>> Denis
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>>
>>>>>> Developer Access Program for Intel Xeon Phi Processors
>>>>>> Access to Intel Xeon Phi processor-based developer platforms.
>>>>>> With one year of Intel Parallel Studio XE.
>>>>>> Training and support from Colfax.
>>>>>> Order your platform today.http://sdm.link/xeonphi
>>>>>> _______________________________________________
>>>>>> genode-main mailing list
>>>>>> genode-main at lists.sourceforge.net
>>>>>> https://lists.sourceforge.net/lists/listinfo/genode-main
>>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>>
>>>>> Check out the vibrant tech community on one of the world's most
>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>> _______________________________________________
>>>>> genode-main mailing list
>>>>> genode-main at lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/genode-main
>>>>>
>>>
>>> ------------------------------------------------------------------------------
>>>
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>> _______________________________________________
>>> genode-main mailing list
>>> genode-main at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/genode-main
>>>
>>
>> ------------------------------------------------------------------------------
>>
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> genode-main mailing list
>> genode-main at lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/genode-main
More information about the users
mailing list