Process migration and persistence

Mon Apr 4 21:15:05 CEST 2016

Hi
   Daniel, Norman-

   From my perspective..  this discussion about the Potential for Genode to
support possess hibernation, seems like a very productive line of thought
with regard to key features of the next generation Genode API.....

   Daniel mentions Genode's potential to support large numbers of
independent processes (more than might at one time fit in RAM)..
   This brought to mind, a vision of a high performance flash file I/O
layer pre-fetching and storing these efficiently serialized process states
overlapped with the running of others Genode processes as needed...   In
some ways, perhaps we could see this as a very organized form of process
virtualization... but it's apt to be far more efficient with resource
allocation at the system's level...   but then maybe, that always been what
the Genode OS has been about!....    Great vision...lets see this happen..

    -Peter

On Thu, Mar 31, 2016 at 4:05 PM, Daniel Waddington <waddy16925 at ...9...>
wrote:

> The more I think about this (and experiment) the more I think that process
> hibernation is quite feasible in Genode (assuming reconstruction of state
> in dependent services) and much easier than in a traditional UNIX based
> systems where process state is thrown around and difficult to track/get to
> without kernel help.  Being able to serialize processes (and identify all
> state belonging to a process) could be a real differentiator for Genode in
> the long run.  Such a capability would have direct relevance to scalability
> and future storage class memories.
>
> Daniel
>
>
> On Thu, Mar 17, 2016 at 11:41 AM, Daniel Waddington <waddy16925 at ...9...>
> wrote:
>
>> Hi Norman,
>> Thanks for the extensive reply (I expected no less from you ;-)). I need
>> to think on your input. I don't have a concrete use-case yet, I'm simply
>> thinking about fast time-multiplexing of huge numbers (e.g., millions) of
>> application instances that can't all be held in memory at once and can't be
>> combined.  I'm thinking about how Genode/microkernels relates to unikernels
>> and mirage OS/jitsu objectives.
>>
>> Daniel
>>
>>
>> On Thu, Mar 17, 2016 at 10:07 AM, Norman Feske <
>> norman.feske at ...1...> wrote:
>>
>>> Hi Daniel,
>>>
>>> great to see you back in Genode land! :-)
>>>
>>> > I'd like to be able to freeze a process (PD or even a vmm), make it
>>> > persistent, and unfreeze at some later point in time.  Does anyone have
>>> > suggestions/pointers on how to do this?
>>>
>>> This does not work out of the box but I see two principal approaches:
>>>
>>> 1) You can run the to-be-frozen program as a child of a custom runtime
>>>    environment that intercepts the program's PD, CPU, RM, and RAM
>>>    sessions. So the environment knows everything that goes on within
>>>    the program. This is what the Noux runtime is doing. Thanks to
>>>    this approach, we are able to implement fork on top of Genode. This
>>>    is actually very similar to what you want to achieve. Instead of
>>>    replicating the forking program, however, you would serialize/
>>>    unserialize this information. That said, there are some constraints
>>>    worth noting. Most importantly, the new process "forgets" all its
>>>    capabilities. So it must re-obtain them in order to work as
>>>    expected. In Noux processes, this is possible because we hide
>>>    Genode capabilities behind the POSIX API. So only the libc deals
>>>    with capabilities and can actively re-establish them after the
>>>    fork.
>>>
>>>    Another example to look at is the GDB monitor. Similar to Noux,
>>>    it intercepts the said core services. But it uses them to inspect
>>>    and control the to-be-debugged program.
>>>
>>> 2) Our general agenda is to create components as mere state
>>>    machines that are fed with state (e.g., using ROM sessions)
>>>    and propagate their results (e.g., using Report sessions). Since
>>>    such components don't contain unrecoverable internal state,
>>>    we can just kill and restart them. When presented with the
>>>    same imported state (aka ROM sessions) after the restart, they
>>>    end up in the same internal state as before. A practical
>>>    example of this approach is the way, the window decorator
>>>    interacts with the window manager in the Turmvilla scenario.
>>>    Here, we can dynamically replace one decorator component by
>>>    another implementation while retaining all of the window layout.
>>>
>>>    This approach just defines the problem away, which I find very
>>>    nice. :-)
>>>
>>> For complex monolithic applications with comprehensive internal state,
>>> there is the further option to run it with a VM (i.e., VirtualBox).
>>> Although we haven't tried it with our port, VirtualBox comes with the
>>> ability to suspend a VM to a file. I think this feature could be enabled
>>> in our version of VirtualBox without too much trouble.
>>>
>>> Can you share more details about your use case?
>>>
>>> Cheers
>>> Norman
>>>
>>> --
>>> Dr.-Ing. Norman Feske
>>> Genode Labs
>>>
>>> http://www.genode-labs.com · http://genode.org
>>>
>>> Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden
>>> Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Transform Data into Opportunity.
>>> Accelerate data analysis in your applications with
>>> Intel Data Analytics Acceleration Library.
>>> Click to learn more.
>>> http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
>>> _______________________________________________
>>> genode-main mailing list
>>> genode-main at lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/genode-main
>>>
>>
>>
>
>
> ------------------------------------------------------------------------------
> Transform Data into Opportunity.
> Accelerate data analysis in your applications with
> Intel Data Analytics Acceleration Library.
> Click to learn more.
> http://pubads.g.doubleclick.net/gampad/clk?id=278785471&iu=/4140
> _______________________________________________
> genode-main mailing list
> genode-main at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/genode-main
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.genode.org/pipermail/users/attachments/20160404/8cc0ddce/attachment.html>