Dear Genodians,
I start by mentioning that I am Jean-Adrien from Gapfruit. Sorry for the confusion I may create. I mailed you on my behalf today, with my private email address, to avoid interfering with any Gapfruit matter.
I take your email, Norman, as an opportunity to share my enthusiasm about Genode. I may not have a significant contribution to make to the roadmap. But I would like to share some of my interests and motivation. This year I want to spend more time on contributions that can benefit the community.
First, I will fix issues I have with my test machine to run Sculpt fully featured. Currently, networking via Wifi is not possible. Other difficulties may come after improving connectivity support.
Next, I want to put effort into proposing 1) when I have finished with the 22.11 update integration and other work-related priorities.
Then during my spare time, I would like to join efforts on support for Xen or Virt. Layer for Qube, adjusting myself to ongoing plans, if any exist?
Following the "Roadmap 2023" discussion and after having a recent look at "Futur Challenges" [1] on the website, it is with lots of excitement that I found many subjects very close to my interests. Here are the topics:
1. Character device drivers as VFS plugins
I proposed, some time ago [2], an SPI driver. The commits never made it to the staging branch because of the interfacing / "quality". Over time, at Gapfruit, we gained experience using the SPI bus, but only with TPMs. We recently refactored it into a VFS plugin for easier interfacing with libraries such as WolfTPM and TPM TSS. I believe BUS drivers such as I2C and SPI could be unified behind a standard "character device" VFS plugin interface. Can that benefit the community? I can put some effort into improving and contributing those commits.
2. Genode as virtualization layer for Qubes OS
I am a daily user of Qube OS. Sculpt is an attractive alternative! Trying it out as a daily OS is part of my plans. I have little knowledge of the Qube OS internals, only that they use Xen as a hypervisor. I was surprised to find out about the hypervisor abstraction layer they created. It is a game changer for using Genode to build a virtualization layer.
It brings so many questions to my mind. First, does full support for a hypervisor such as Xen need to find its way to Genode? Or is base-nova already a viable candidate? Are all necessary building blocks for AppVM message passing present within Genode? Which approach would you privilege first for investigation? If Xen, support for Genode in a Dom0 scenario first? Or, as an AppVM first, focussing on Qube RPC mechanism support? (I am aware of Xen DomU support, but I do not have all details in mind) Currently, those questions might be off-topic. Overall, it seems a significant challenge.
3. Xen as "Kernel" for Genode
My ultimate goal here would be to have a pure minimal Genode-based OS in Dom0 for only managing guests' VM.
Last summer, by curiosity related to 2), I started investigating necessary changes to the build system for building Xen as a port and generating a boot image using Xen with a Genode base-hw minimal scenario as Dom0. I was able to set up the port, successfully building Xen and booting it, but with a minimal Linux as Dom0 instead of a chosen Genode scenario "run/log," e.g.
I still need to reach my goal. The real work only begins now. My primary technical motivation was working on low-level mechanisms to manage DomU(s) and enable RPC from one DomU to the other. I lost interest over time because I needed to be more confident in its relevance and missed lots of knowledge.
At that time, I was unaware of the work of Julian Stecklina. I would be interested in looking into it! Is it publicly available?
-----
Side note about the AMD Radeon drivers, my x86_64 test machine is an HP Elitebook 849 G2 equipped with an AMD Radeon R7 M260X. If some workforce is needed, I would also be happy to test and provide feedback!
Another topic I was very interested in is initiatives like Spunky [3]. I am not making it a point as this is not to take as seriously as the others. With the gain of popularity from languages like, e.g. Rust. Is there any plan or similar initiative in that regard? I would be up to start one as a pet project. I am interested in trying this out with the Zig programming language [4]. My motivation for such a language comes from a Youtuber I appreciate [5]. He made an interesting video on memory safety within different languages [6]. I do not have any strong opinion on "C++ vs any other lang" topics. If you have other languages in mind, please let me know!
Finally, thank you all for your supportive, kind and professional attitude! Since I started working with the Genode OS Framework, it has always been a pleasure to read from and interact with any of you. I am relatively junior to OS development. Thanks to my co-workers and following daily discussions on this mailing list, I learned a lot!
[1] https://genode.org/about/challenges [2] https://github.com/genodelabs/genode/pull/4187 [3] https://genodians.org/m-stein/2019-05-09-spunky-1 [4] https://github.com/ziglang/zig [5] https://www.youtube.com/@contextfree [6] https://www.youtube.com/watch?v=qeiRGbYCD-0
Cheers, Jean-Adrien
Hello Jean-Adrien,
thank you very much for your enthusiastic posting. I'm happy that you hold our project in such high regard.
You are presenting a firework of ideas! Please see my comments below.
First, I will fix issues I have with my test machine to run Sculpt fully featured. Currently, networking via Wifi is not possible. Other difficulties may come after improving connectivity support.
Don't hesitate to post your hardware configuration (e.g., the model of the Wifi card) here. Maybe, someone else has already taken steps to enable it?
Then during my spare time, I would like to join efforts on support for Xen or Virt. Layer for Qube, adjusting myself to ongoing plans, if any exist?
Even though we brainstormed ideas to bring both projects closer together, I am not aware of any concrete efforts under way.
Following the "Roadmap 2023" discussion and after having a recent look at "Futur Challenges" [1] on the website, it is with lots of excitement that I found many subjects very close to my interests. Here are the topics:
- Character device drivers as VFS plugins
I proposed, some time ago [2], an SPI driver. The commits never made it to the staging branch because of the interfacing / "quality". Over time, at Gapfruit, we gained experience using the SPI bus, but only with TPMs. We recently refactored it into a VFS plugin for easier interfacing with libraries such as WolfTPM and TPM TSS. I believe BUS drivers such as I2C and SPI could be unified behind a standard "character device" VFS plugin interface. Can that benefit the community? I can put some effort into improving and contributing those commits.
I very much appreciate your suggestion. Personally, my experience in these areas (TPMs, SPI) is quite limited. But the option of plugging character-device drivers into a generic VFS character-device plugin sounds attractive.
Maybe this character-device interface could be designed such that the actual driver code can be operated both in a free-standing fashion (e.g., hosted inside a VFS-less component) as well as embedded in a VFS plugin? So the character driver would actually be a "VFS plugin plugin". That would be great!
- Genode as virtualization layer for Qubes OS
...
It brings so many questions to my mind. First, does full support for a hypervisor such as Xen need to find its way to Genode? Or is base-nova already a viable candidate?
From a security perspective, NOVA is preferable over Xen because NOVA's attack surface is more than an order of magnitude smaller, and Genode/NOVA does not require a Dom0 at all. I got the impression that some Qubes developers found prospect of replacing Xen by a microhypervisor (like NOVA) quite intriguing.
On the other hand, we have to keep in mind that Genode/NOVA currently still lacks a few convenience features like suspend/resume. This, however, will be addressed throughout this year.
Speaking of hypervisor candidates, there is current work in progress to enable the use of Genode's base-hw microkernel as hypervisor on x86 as well. So this may become a further option.
Are all necessary building blocks for AppVM message passing present within Genode?
I think this was the point where our past brainstorming got stuck.
If I remember right, Qubes uses Xen's event channels and shared memory as inter-VM-transport mechanism. VirtualBox on Genode offers quite many host-guest integration features, but none seemed to perfectly align with the needs of Qubes. There were multiple options to explore. But the exploration never really happened.
Which approach would you privilege first for investigation?
I think that it would be best to pick up the existing discussion [1] and interview Marek Górecki so that he can share his views and ideas.
[1] https://forum.qubes-os.org/t/qubes-os-based-on-the-genode-os-framework/11735
The biggest question seems to be how to map the Qubes middleware to the mechanisms available under VirtualBox.
If Xen, support for Genode in a Dom0 scenario first?
To me this sounds quite distracting when considering the following topics.
- One will have to convince Genode to run under Xen. - One will need to host the VM management functionality on Genode, which comes down to a huge porting effort. - Once succeeded, there is still no clear migration path away from Xen.
I would probably not pick this way.
Or, as an AppVM first, focussing on Qube RPC mechanism support? (I am aware of Xen DomU support, but I do not have all details in mind) Currently, those questions might be off-topic. Overall, it seems a significant challenge.
That's probably a rather lower-hanging fruit regarding the combination of both projects (compared to the other directions). But on the other hand, it does not add a huge value to either project.
From Qubes' perspective, there are already unikernels like MirageOS that satisfy the desire for low-complexity AppVMs. Genode would not solve a real problem here.
From Genode's perspective, the Qubes middleware interfaces can only be used on top of Qubes/Xen. But replacing Genode's regular use of a microhypervisor like NOVA by Xen is not attractive.
Therefore, this topic may be best pursued by a regular Qubes user as an experiment, just out of curiosity if it is possible to integrate Genode into a familiar environment, without expecting a measurable impact on either project.
- Xen as "Kernel" for Genode
My ultimate goal here would be to have a pure minimal Genode-based OS in Dom0 for only managing guests' VM.
Last summer, by curiosity related to 2), I started investigating necessary changes to the build system for building Xen as a port and generating a boot image using Xen with a Genode base-hw minimal scenario as Dom0. I was able to set up the port, successfully building Xen and booting it, but with a minimal Linux as Dom0 instead of a chosen Genode scenario "run/log," e.g.
I still need to reach my goal. The real work only begins now. My primary technical motivation was working on low-level mechanisms to manage DomU(s) and enable RPC from one DomU to the other. I lost interest over time because I needed to be more confident in its relevance and missed lots of knowledge.
At that time, I was unaware of the work of Julian Stecklina. I would be interested in looking into it! Is it publicly available?
Wow, this was more than 15 years ago. I'm not aware of any remaining public traces of this work. Have you tried contacting Julian about it?
Another topic I was very interested in is initiatives like Spunky [3]. I am not making it a point as this is not to take as seriously as the others. With the gain of popularity from languages like, e.g. Rust. Is there any plan or similar initiative in that regard?
I have to confess that Spunky is currently on ice [1].
[1] https://github.com/genodelabs/genode/issues/3308#issuecomment-1235217730
This year's road map will revive Genode's foray with Rust. But this time, we will explore it by starting with the application level (Rust atop Genode's POSIX support). The goal is to make it very easy to compile Rust software for Genode using the Goa tool. So we can gradually familiarize ourselves with the benefits of the language without introducing any friction.
The other direction you mentioned - replacing C++ of the base framework with something else - is outside the scope of 2023. Given the rationale of [1], I think we should not seek salvation in alternatives right now but focus on tangible evolutionary improvements while using C++, like replacing exceptions with the 'Attempt' pattern [2], and further tightening the base API.
[2] https://genodians.org/nfeske/2021-11-26-attempt-no-exceptions
I would be up to start one as a pet project. I am interested in trying this out with the Zig programming language [4]. My motivation for such a language comes from a Youtuber I appreciate [5]. He made an interesting video on memory safety within different languages [6]. I do not have any strong opinion on "C++ vs any other lang" topics. If you have other languages in mind, please let me know!
Have you considered integrating support for alternative languages like zig into Goa [3]?
[3] https://github.com/nfeske/goa
Finally, thank you all for your supportive, kind and professional attitude! Since I started working with the Genode OS Framework, it has always been a pleasure to read from and interact with any of you. I am relatively junior to OS development. Thanks to my co-workers and following daily discussions on this mailing list, I learned a lot!
Thank you for sharing your many interests and ideas! I take your email as a big compliment. :-)
Cheers Norman
Dear Norman,
Thanks for your feedback!
Don't hesitate to post your hardware configuration (e.g., the model of the Wifi card) here. Maybe, someone else has already taken steps to enable it?
Sure, I would like to look closely at the issue I encountered so I can provide detailed feedback. I'll join my test machine specs with that discussion. Unfortunately, I did not have the time to do so yet.
Maybe this character-device interface could be designed such that the actual driver code can be operated both in a free-standing fashion (e.g., hosted inside a VFS-less component) as well as embedded in a VFS plugin? So the character driver would actually be a "VFS plugin plugin". That would be great!
A nested plugging interface for "character devices" is attractive to reduce the number of sessions. I did not think of that direction. Currently, the SPI's VFS plugin is the full driver. This solution exposes the entire component to the plugin it is attached to. There are better solutions than this. However, we use it only for the TPM, which fits our needs.
One idea I had was to create two drivers, SPI and I2C, as servers implementing the Terminal session. Then create a "client" VFS plugin that can interface with those drivers. Ideally, only one plugin is necessary to interface read/write on a Terminal session. Such a plugin may already exist. It needs further thinking and experimentation!
From a security perspective, NOVA is preferable over Xen because NOVA's attack surface is more than an order of magnitude smaller, and Genode/NOVA does not require a Dom0 at all. I got the impression that some Qubes developers found prospect of replacing Xen by a microhypervisor (like NOVA) quite intriguing.
On the other hand, we have to keep in mind that Genode/NOVA currently still lacks a few convenience features like suspend/resume. This, however, will be addressed throughout this year.
Speaking of hypervisor candidates, there is current work in progress to enable the use of Genode's base-hw microkernel as hypervisor on x86 as well. So this may become a further option.
Thanks for the overall feedback about Xen. It confirms my initial feeling that it might not be relevant for Genode. This is the way.
I think that it would be best to pick up the existing discussion [1] and interview Marek Górecki so that he can share his views and ideas.
[1] https://forum.qubes-os.org/t/qubes-os-based-on-the-genode-os-framework/11735
The biggest question seems to be how to map the Qubes middleware to the mechanisms available under VirtualBox.
I may formulate a post for the Qubes-os forum. However, I would hold it until I have enough knowledge on both ends to argue what's possible.
Wow, this was more than 15 years ago. I'm not aware of any remaining public traces of this work. Have you tried contacting Julian about it?
I did not try to contact him. Eventually, I would if I retake a deep dive into this. But that was more about curiosity.
The other direction you mentioned - replacing C++ of the base framework with something else - is outside the scope of 2023.
Oh, I am sorry I did not mean to replace C++ entirely :) I was thinking of working on a secondary kernel, like base-hw, but fully implemented with Zig instead of C++. The direction I would take is the same as spunky, replacing part of base-hw at first. Plus, Zig claims great interoperability with C, then by extension to C++ via C.
Have you considered integrating support for alternative languages like zig into Goa [3]?
I did not initially think about it. Now that you mention it, it is likely a good starting point!
Cheers, Jean-Adrien
Hello Jean-Adrien,
Oh, I am sorry I did not mean to replace C++ entirely :) I was thinking of working on a secondary kernel, like base-hw, but fully implemented with Zig instead of C++. The direction I would take is the same as spunky, replacing part of base-hw at first. Plus, Zig claims great interoperability with C, then by extension to C++ via C.
thanks for clarifying. Now I got it, and can very much relate to your ambition. With language interoperability solved, the kernel is indeed a compelling playground for testing the waters with a new language that way. It solves interesting problems and its complexity is still manageable by a single person, at least when targeting one specific hardware platform.
I hope that my aspiration to further simplify the base system (in particular reducing the reliance on C++ complexities) over the year will make your life easier.
Cheers Norman
-
Alice -
Norman Feske