Genode in distributed scenarios and ROM proxy implementation

[Menno Valkema]

Hi Johannes,

This is great. Currently we're trying to accomplish a similar thing for
RPC calls and Signals. The approach with statically configured proxies
you take, also made most sense to us, which is what we did for RPC
calls, and we're now trying to use it for Signals. Combined with your
work it seems together we made the first steps towards distributed use
of the fundamental IPC mechanisms provided by Genode (shared memory,
rpc, and signals).

For RPC calls we took a similar approach using a client_proxy and a
server_proxy. Say we have a session called abc_session this is how the
components collaborate:
1) Abc Rpc client connects to a client_proxy which behaves like a
abc_session/session_component
2) The client_proxy marshalls the commands into command structures with
opcodes and arguments, and forwards these over a nic connection.
3) server_proxy receives the commands over a nic connection, un-marshals
the commands, and forwards the requests to the server while behaving as
a abc_session client.
4) the server handles the request, and returns.
5) return value is forwarded to the client in a similar way the commands
were transmitted.

Where we both have some form of serialized commands (in your case it's
handled by the ip backend  with an enum identifying the action is being
sent over), we choose, for now, not to specify any sort of
transportation mechanism except a bare nic session, and send raw data
over nic sessions. In case we need an ip layer, we plan to move this in
a separate component which knows how to handle ip, ipsec or some other
protocol. However, I like the idea of your implementation to use various
back-ends which handle different transportation types.

Some challenges we're still looking into are:
- Manual marshalling. Right now we marshall the rpc calls manually.
Genode already has a great marshalling mechanism in place, however I
didn't manage to re-use this, so for now I do it by hand. This seems
like a bit of a waste, so in a later stage I hope to look into this again.
- Cross CPU architecture usage of arguments. How to handle integer
arguments or structs with integer arguments between big/little endian
architectures, or systems with a different notion of word-lengths and
struct alignment?
- We're still working on Signals.
- Routing. Presently we statically configure how the client-proxy and
the server-proxy are connected. It would be better if we had more
flexibility here, like what's now being provided by init.
- Capabilities. It would be great to have some sort of distributed
capability system to have stricter control over who talks to who.

Regards,
Menno

PS. I went through you source code to see how you implemented things,
and it's quite clear what's going on. However when I tried to run the
server by typing 'make run/test-remoterom_backend_nic_ip_server' some
methods being called seemed to be missing (calculate_checksum for
example), and the compiler exited with an error. It seems some of the
work is not in the github repository yet?

-- 
Cyber Security Labs B.V. | Gooimeer 6-31 | 1411 DD Naarden | The
Netherlands | https://nlcsl.com/ | +31 35 631 3253 (office)



On 20-03-16 01:30, Johannes Schlatow wrote:
> Hi,
> 
> as I'm beginning to use Genode in a distributed scenario, I had some thoughts about the use of proxy components in order to transmit particular session interfaces over an arbitrary communication medium. The general concept here is that a client component can connect to a server component running on a different Genode system by using a proxy component on each side.
> 
> For starters, I implemented this for the ROM session. I call this implementation "remote ROM": The remote ROM server is instantiated on the server side and connects to a local ROM service. It relays the content of a particular ROM module the remote ROM client, which is instantiated on another Genode system. The remote ROM client receives the updated ROM content from the remote ROM server and provides a local ROM service.
> 
> In order to generalise the implementation, I separated the network-specific part in a backend library so that the backend can be tailored for the particular communication medium, protocol and policy. I think this renders the implementation highly flexible. I added a backend implementation (nic_ip) as a proof of concept, which uses a Nic service to transmit network packets with a static IPv4 header.
> 
> In my opinion, the remote ROM implementation is already the enabler for quite a lot of distributed scenarios as it allows full publisher-subscriber communication of arbitrary data without adding too much interdependencies between the systems.
> 
> The implementation can be found here:
> https://github.com/ValiValpas/genode-world/tree/remoterom
> 
> *Please feel free to adapt/contribute/improve this concept and its implementation.*
> 
> ------------------------------------------------------------------------------
> Transform Data into Opportunity.
> Accelerate data analysis in your applications with
> Intel Data Analytics Acceleration Library.
> Click to learn more.
> http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
> _______________________________________________
> genode-main mailing list
> genode-main at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/genode-main
>