Hello Genodians
The direction Goa is evolving is really cool! The new and upcoming features address topics that are perfectly aligned with how we currently use, or plan to use Goa.
In addition to what is already addressed in issue #44 [1], in order to run or test some more complex projects we use:
- a source for random (this can be achieved using jitter_sponge) - accessing externally provided ROM modules - multiple Nic/uplink interfaces and a nic_router
For the random source I have a working commit, that I will have to update to cleanly apply on top of staging. This will start jitter_sponge if <requires> <terminal label="random"/> is found. To make this available in general, the jitter_sponge binary would have to be published by genodelabs. Do you think this approach is reasonable?
For accessing externally provided ROM modules I already have a patch that enables this feature. It uses a fs_rom and a lx_fs to access ROM modules listed in <requires> <rom label=""/>. If you think this could be a commonly useful feature, I can contribute it.
Providing multiple Nic/Uplink interfaces and a nic_router will require some more engineering to do right. Our workaround instantiates a fixed configuration which matches what we need. This is obviously not usable in general. I see the following points that need to be considered:
- number and names of interfaces? Probably each listed in <requires> by label. - when to start nic_router? Probably not as easy as start it when multiple interfaces are configured. - how should the nic_router config be provided? Here I don not see yet a satisfying and practical solution.
Best regards, Pirmin
Hi Pirmin,
thanks for your feedback and for sharing your ideas...
On 21/06/2023 11:19, Duss Pirmin wrote:
Hello Genodians
The direction Goa is evolving is really cool! The new and upcoming features address topics that are perfectly aligned with how we currently use, or plan to use Goa.
In addition to what is already addressed in issue #44 [1], in order to run or test some more complex projects we use:
- a source for random (this can be achieved using jitter_sponge) - accessing externally provided ROM modules - multiple Nic/uplink interfaces and a nic_router
For the random source I have a working commit, that I will have to update to cleanly apply on top of staging. This will start jitter_sponge if <requires> <terminal label="random"/> is found. To make this available in general, the jitter_sponge binary would have to be published by genodelabs. Do you think this approach is reasonable?
In principle, such extensions are reasonable. In the case of jitter_sponge, however, this would mean that we needed to make the Goa release dependent on the genode-world repo. I would rather avoid this.
For accessing externally provided ROM modules I already have a patch that enables this feature. It uses a fs_rom and a lx_fs to access ROM modules listed in <requires> <rom label=""/>. If you think this could be a commonly useful feature, I can contribute it.
I could imagine this to become useful for running multiple Goa instances. I particularly have the 'clipboard' ROM (and Report) in mind. Even though running multiple Goa instances is not an established use case (yet), routing unrecognized ROM modules to lx_fs instead of to the black-hole component would not be much of a difference for the naive user. In return, we gain the ability to change ROM content at runtime for development/testing. Combined with a similar setup for the Report service, this would basically provide a report_rom functionality that we can easily introspect.
From my perspective: I'd appreciate your contribution ;-)
Providing multiple Nic/Uplink interfaces and a nic_router will require some more engineering to do right. Our workaround instantiates a fixed configuration which matches what we need. This is obviously not usable in general. I see the following points that need to be considered:
- number and names of interfaces? Probably each listed in <requires> by label.
Currently (i.e. with #44), Goa uses the label as the name of the tap device. Do we agree that, if multiple <nic/> requirements exists, Goa should instantiate several nic_drv components and provide separated networks?
- when to start nic_router? Probably not as easy as start it when multiple interfaces are configured.
For a <nic/> requirement, the nic_router is always instantiated because only the latter provides this service whereas the nic_drv acts as an Uplink client. When multiple nic drivers will be instantiated (i.e. separate networks are required), we could also have multiple instances of the nic_router.
- how should the nic_router config be provided? Here I don not see yet a satisfying and practical solution.
For customising the nic_router config, I would go for integrating the nic_router into the pkg's runtime and, instead of using <nic/> requirements, let the pkg provide a couple of <uplink/> services. This would instruct Goa to instantiate multiple nic drivers (currently restricted to one driver) as Uplink clients.
Cheers Johannes
Hello Johannes,
Thank you for sharing your thoughts.
On 21.06.23 14:47, Johannes Schlatow wrote:
In principle, such extensions are reasonable. In the case of jitter_sponge, however, this would mean that we needed to make the Goa release dependent on the genode-world repo. I would rather avoid this.
I see your point. As we see no reasonable alternative, we will keep and maintain our own patch for this.
From my perspective: I'd appreciate your contribution ;-)
I'm working on bringing the patch to staging. I created issue #55 for this. I'm almost finished with the updated patch.
Providing multiple Nic/Uplink interfaces and a nic_router will require some more engineering to do right. Our workaround instantiates a fixed configuration which matches what we need. This is obviously not usable in general. I see the following points that need to be considered:
- number and names of interfaces? Probably each listed in <requires> by label.
Currently (i.e. with #44), Goa uses the label as the name of the tap device. Do we agree that, if multiple <nic/> requirements exists, Goa should instantiate several nic_drv components and provide separated networks?
In our case, we deploy the resulting pkg archive to real hardware, which provides a nic_router outside of the runtime. Depending on the platform the runtime is executed on, I prefer more general name mapping accordingly. Therefore I suggest on `eth0` to `ethX`, which is mapped to `tap0` to `tapX` for the Linux case.
- when to start nic_router? Probably not as easy as start it when multiple interfaces are configured.
For a <nic/> requirement, the nic_router is always instantiated because only the latter provides this service whereas the nic_drv acts as an Uplink client. When multiple nic drivers will be instantiated (i.e. separate networks are required), we could also have multiple instances of the nic_router.
I like the idea of starting a separate nic_router per <nic label=""/>! I would also start a nic_drv for each to make things easy and straightforward.
- how should the nic_router config be provided? Here I don not see yet a satisfying and practical solution.
For customising the nic_router config, I would go for integrating the nic_router into the pkg's runtime and, instead of using <nic/> requirements, let the pkg provide a couple of <uplink/> services. This would instruct Goa to instantiate multiple nic drivers (currently restricted to one driver) as Uplink clients.
Our use case is that we want to utilize Goa to build Gateway applications that may require multiple network interfaces. We want to test the pkg runtime on Linux via tap devices and then build and deploy the pkg to the Gateway hardware. The device that the pkg archive is deployed to also runs other sub-systems that require networking. By integrating a nic_router in the pkg runtime that provides the Uplink sessions, the nic drivers would be limited to networking for that particular pkg runtime, right?
Regards, Pirmin
Hi Pirmin,
On 23/06/2023 11:00, Duss Pirmin wrote:
Hello Johannes,
Thank you for sharing your thoughts.
On 21.06.23 14:47, Johannes Schlatow wrote:
In principle, such extensions are reasonable. In the case of jitter_sponge, however, this would mean that we needed to make the Goa release dependent on the genode-world repo. I would rather avoid this.
I see your point. As we see no reasonable alternative, we will keep and maintain our own patch for this.
From my perspective: I'd appreciate your contribution ;-)
I'm working on bringing the patch to staging. I created issue #55 for this. I'm almost finished with the updated patch.
Providing multiple Nic/Uplink interfaces and a nic_router will require some more engineering to do right. Our workaround instantiates a fixed configuration which matches what we need. This is obviously not usable in general. I see the following points that need to be considered:
- number and names of interfaces? Probably each listed in <requires> by label.
Currently (i.e. with #44), Goa uses the label as the name of the tap device. Do we agree that, if multiple <nic/> requirements exists, Goa should instantiate several nic_drv components and provide separated networks?
In our case, we deploy the resulting pkg archive to real hardware, which provides a nic_router outside of the runtime. Depending on the platform the runtime is executed on, I prefer more general name mapping accordingly. Therefore I suggest on `eth0` to `ethX`, which is mapped to `tap0` to `tapX` for the Linux case.
Currently, the label of the required nic service determines the name of the tap device in Goa. I understand that, to make archives deployable, one would not want to have a "tapX" label in the runtime. What about we'd be prefixing the specified label by a "tap_" e.g. instead? This way you could have a "eth0" label, which will get mapped to a "tap_eth0" device by Goa.
- when to start nic_router? Probably not as easy as start it when multiple interfaces are configured.
For a <nic/> requirement, the nic_router is always instantiated because only the latter provides this service whereas the nic_drv acts as an Uplink client. When multiple nic drivers will be instantiated (i.e. separate networks are required), we could also have multiple instances of the nic_router.
I like the idea of starting a separate nic_router per <nic label=""/>! I would also start a nic_drv for each to make things easy and straightforward.
This should be pretty straightforward to implement. Would you mind opening an issue for this so we can continue the discussion on github?
- how should the nic_router config be provided? Here I don not see yet a satisfying and practical solution.
For customising the nic_router config, I would go for integrating the nic_router into the pkg's runtime and, instead of using <nic/> requirements, let the pkg provide a couple of <uplink/> services. This would instruct Goa to instantiate multiple nic drivers (currently restricted to one driver) as Uplink clients.
Our use case is that we want to utilize Goa to build Gateway applications that may require multiple network interfaces. We want to test the pkg runtime on Linux via tap devices and then build and deploy the pkg to the Gateway hardware. The device that the pkg archive is deployed to also runs other sub-systems that require networking. By integrating a nic_router in the pkg runtime that provides the Uplink sessions, the nic drivers would be limited to networking for that particular pkg runtime, right?
The major difference between Goa and hardware deployments is that the latter usually have one or multiple managed Nic router(s) while Goa has no way to customize the Nic router config (yet). The current workaround has been to create an pkg archive in Goa that is deployable on hardware and add another Goa project that wraps a Nic router with a custom config around the other pkg. You can find an example of this in my goa-pkgs repo [1] where I have a vnc_server and a test-vnc_server project.
I see that, an some point, we probably need to integrate a means to provide Goa with a custom Nic router config. Maybe this could be done by adding a 'nic_router.config' file into the project directory. If this file is present, Goa would take its content as config for the Nic router instead of the default config. However, what shall happen if there are multiple Nic router instances?
[1] https://github.com/jschlatow/goa-pkgs
Cheers Johannes
Hello Johannes
Thanks for your quick response.
On 6/23/23 13:44, Johannes Schlatow wrote:
Currently, the label of the required nic service determines the name of the tap device in Goa. I understand that, to make archives deployable, one would not want to have a "tapX" label in the runtime. What about we'd be prefixing the specified label by a "tap_" e.g. instead? This way you could have a "eth0" label, which will get mapped to a "tap_eth0" device by Goa.
This is a very neat idea, which maps what I had in mind very well. Shall I create an issue for this?
This should be pretty straightforward to implement. Would you mind opening an issue for this so we can continue the discussion on github?
I did create https://github.com/genodelabs/goa/issues/57.
The major difference between Goa and hardware deployments is that the latter usually have one or multiple managed Nic router(s) while Goa has no way to customize the Nic router config (yet). The current workaround has been to create an pkg archive in Goa that is deployable on hardware and add another Goa project that wraps a Nic router with a custom config around the other pkg. You can find an example of this in my goa-pkgs repo [1] where I have a vnc_server and a test-vnc_server project.
I see that, an some point, we probably need to integrate a means to provide Goa with a custom Nic router config. Maybe this could be done by adding a 'nic_router.config' file into the project directory. If this file is present, Goa would take its content as config for the Nic router instead of the default config. However, what shall happen if there are multiple Nic router instances?
I like that the user can provide a customized 'nit_router.config' if needed. For multiple NIC interfaces, one could use 'nic_router_<label>.config'. I would use 'nic_router.config' only for the case when only one `<nic/>` is required.
I only fear that the project directory could get cluttered with additional files. What do you think abort creating a directory that contains the files to customize the run-stage? With this, one could also take the external ROMs from there.
Regards, Pirmin
Hello Pirmin,
Johannes Schlatow wrote:
What about we'd be prefixing the specified label by a "tap_" e.g. instead?
Since the <requires> subnodes in the pkg/runtime are just XML, there is be no harm in adding custom attributes that can be picked up by Goa as hints. The name of the tap device could be a separate <nic> attribute that is ignored by Sculpt but interpreted by Goa. The use of explicit attribute values avoids the need for introducing conventions.
This reminds me of the similar problem about the distinction between different flavors of <gui> applications, some of which expect a window system (e.g., a multi-window application) where others might expect a raw nitpicker GUI server. Goa could look at a <gui wm="yes"> to decide whether to spawn a themed_wm or not. On Sculpt, those attribute would be ignored. These hints should remain as high-level as possible to grant Goa as much freedom as possible.
If this file is present, Goa would take its content as config for the Nic router instead of the default config. However, what shall happen if there are multiple Nic router instances?
I like that the user can provide a customized 'nit_router.config' if needed. For multiple NIC interfaces, one could use 'nic_router_<label>.config'. I would use 'nic_router.config' only for the case when only one `<nic/>` is required.
The direction bears the risk of making Goa more and more complex over time whereas I think that Goa should better remain largely agnostic wrg to specific components like the NIC router. Support for custom NIC-router configurations would effectively extend the user interface of Goa by the complete interface surface area of the NIC router. This would be a mistake because it sets the precedent to also expose the nitpicker configuration as a Goa feature next. Where do we draw the line?
In the concrete case, it would be most sensible to have dedicated (potentially multiple) Goa runtime projects that contain no code but only a pkg/runtime and a bunch of configurations in raw/ - as done in the Unix tutorial and also suggested by Johannes. Internally, such a pkg/runtime can have as many NIC routers with special rules as one wants. But from the outside, it would still be a subsystem requiring <N> NICs. Instead of adding features to Goa, let us better leverage the recursive nature of Genode.
Cheers Norman
-
Duss Pirmin -
Johannes Schlatow -
Norman Feske -
Pirmin Duss