Genode in distributed scenarios and ROM proxy implementation

[Menno Valkema]

Hi Johannes,

If you have the sources for marshalling conveniently laying around,
please share, I'd like to have a look at them.

Thanks, Menno

On 21-03-16 17:01, Johannes Schlatow wrote:
> Hi Menno,
> 
> On Mon, 21 Mar 2016 12:52:22 +0100
> Menno Valkema <menno.genode at ...9...> wrote:
> 
>> Hi Johannes,
>>
>> This is great. Currently we're trying to accomplish a similar thing for
>> RPC calls and Signals. The approach with statically configured proxies
>> you take, also made most sense to us, which is what we did for RPC
>> calls, and we're now trying to use it for Signals. Combined with your
>> work it seems together we made the first steps towards distributed use
>> of the fundamental IPC mechanisms provided by Genode (shared memory,
>> rpc, and signals).
>>
>> For RPC calls we took a similar approach using a client_proxy and a
>> server_proxy. Say we have a session called abc_session this is how the
>> components collaborate:
>> 1) Abc Rpc client connects to a client_proxy which behaves like a
>> abc_session/session_component
>> 2) The client_proxy marshalls the commands into command structures with
>> opcodes and arguments, and forwards these over a nic connection.
>> 3) server_proxy receives the commands over a nic connection, un-marshals
>> the commands, and forwards the requests to the server while behaving as
>> a abc_session client.
>> 4) the server handles the request, and returns.
>> 5) return value is forwarded to the client in a similar way the commands
>> were transmitted.
>>
>> Where we both have some form of serialized commands (in your case it's
>> handled by the ip backend  with an enum identifying the action is being
>> sent over), we choose, for now, not to specify any sort of
>> transportation mechanism except a bare nic session, and send raw data
>> over nic sessions. In case we need an ip layer, we plan to move this in
>> a separate component which knows how to handle ip, ipsec or some other
>> protocol. However, I like the idea of your implementation to use various
>> back-ends which handle different transportation types.
>>
>> Some challenges we're still looking into are:
>> - Manual marshalling. Right now we marshall the rpc calls manually.
>> Genode already has a great marshalling mechanism in place, however I
>> didn't manage to re-use this, so for now I do it by hand. This seems
>> like a bit of a waste, so in a later stage I hope to look into this again.
> 
> I also started with a generic proxy implementation for RPC some time ago and did make some progress on the marshalling. However, I dismissed the idea as RPC over a network didn't appeal to me as a very good idea, especially when distributed capabilities are required.
> Nevertheless, I might have some code buried in my repositories. Let me know if you're interested in that.
> 
>> - Cross CPU architecture usage of arguments. How to handle integer
>> arguments or structs with integer arguments between big/little endian
>> architectures, or systems with a different notion of word-lengths and
>> struct alignment?
>> - We're still working on Signals.
>> - Routing. Presently we statically configure how the client-proxy and
>> the server-proxy are connected. It would be better if we had more
>> flexibility here, like what's now being provided by init.
>> - Capabilities. It would be great to have some sort of distributed
>> capability system to have stricter control over who talks to who.
>>
>> Regards,
>> Menno
>>
>> PS. I went through you source code to see how you implemented things,
>> and it's quite clear what's going on. However when I tried to run the
>> server by typing 'make run/test-remoterom_backend_nic_ip_server' some
>> methods being called seemed to be missing (calculate_checksum for
>> example), and the compiler exited with an error. It seems some of the
>> work is not in the github repository yet?
> 
> Oh, sorry. I forgot to mention that this requires some modifications of the Ipv4_packet class, see:
> https://github.com/genodelabs/genode/pull/1915
> 
> ------------------------------------------------------------------------------
> Transform Data into Opportunity.
> Accelerate data analysis in your applications with
> Intel Data Analytics Acceleration Library.
> Click to learn more.
> http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
> _______________________________________________
> genode-main mailing list
> genode-main at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/genode-main
>