Hello,
the end of the year 2015 is approaching. So it is time for planning our activities for the upcoming year. I plan to finalize the roadmap until mid of January. Hereby, I'd like to kick off the discussion. It goes without saying that everyone of you is invited to join in!
Before I come to my suggestions for 2016, let me briefly revisit the outcome of the roadmap for the past year. One year ago, I suggested three main topics to work on, namely the use of Genode as general-purpose OS, the advancement of our custom base-hw kernel platform, and the use of the seL4 kernel. In the discussions following my posting, many further points were raised, most prominently the need for through documentation, package management, and a sustainable quality of the base system in terms of stability and performance. In top of that, we received quite a few wishes of higher-level functionality such as a modern web browser or other typical desktop-computing features. Of course, we were not able to address all those topics but I am overly happy that we reached the point where a hand full of us (including me) are using Genode as their day-to-day OS. When I initially switched to using the Turmvilla scenario [1] at the end of May, the experience was admittedly a bit painful. But now, just a few months later, the beauty of the system becomes apparent as all the pieces come so nicely together. The performance, stability, and device-driver support have reached a level that leaves people impressed every time I have the chance to show off my system. Once people become interested, there is now the book available, which provides a smooth introduction into Genode. The feedback I receive about the book is overwhelmingly positive. So we did something right in 2015. :-)
After having passed the point where a few of us are able to use Genode as day-to-day OS, we should put the emphasis of the upcoming year on ways to make Genode accessible for a wider community. In a recent posting [2], I identified two possible ways to do that. The first way would be publishing a series of step-by-step guides that explain how to put Genode components together in order to create custom system scenarios. Those articles could be accompanied by screencasts or live-system images. Example scenarios could be the Turmvilla scenario, building a home-computer-like system for kids using the Raspberry Pi (like the Genode system my kids are using), building a network appliance like a data diode, tinkering with the USB Armory, etc. Those articles should be inviting to people who enjoy the building of systems. The second way would be to showcase a system with practical value to end users. I am thinking along the lines of a disposable OS like Tails that allows the user to browse the internet via the Tor network. But that is just an idea.
In this spirit, I propose the overall focus of 2016 to be: Let us make Genode accessible to the world outside the inner circle of us enthusiasts.
On a technical level, this motive implicates the following topics:
* The deployment and management of Genode systems, i.e., by bringing forward Emery's work on the Nix package manager. This direction also reinforces the need to achieve binary compatibility between the various base platforms to make the distribution of binary packages, reproduceable builds, and continues test-and-integration scalable.
* Supplementing Genode with user-friendly means to configure the system (e.g., wireless network, audio, display settings). Right now, I use Vim as configuration front end, which is cool, but also intimidating to less technical-minded users.
* Accommodation of common desktop use cases like plugging in a USB stick to work with the files stored on it. Also disk encryption comes into mind.
* Optimization of Genode for the use on a laptop, e.g., addressing fan control, power management, suspend/resume, and similar features.
There are also other possible avenues to support the stated goal:
* Identifying ways of how Genode could play a role in related projects like Qubes OS. For example, we could promote the use of Genode as a tool for building App VMs as Qubes subsystems. Granted, this scenario leaves the architectural benefits of Genode with respect to its small TCB complexity unused as Qubes still relies on Xen, and Linux as Dom0. But Genode would still (possibly) provide value to the Qubes project. Maybe, there would be the prospect to replace Dom0 with Genode in the longer term? However, to drive this direction of work, we would certainly need someone who is actually using Qubes and has the natural incentive to work on such an integration.
* Making Genode-based systems easily deployable on Amazon's EC2. Similar to the previous point, it would be beneficial to have someone working on this topic who is naturally interested in cloud computing.
* Foster the cross-pollination of the seL4 and Genode communities. I got enthusiastic responses about my seL4-related work. There is definitely a strong interest in this kernel and a growing anticipation for formally verified software. Today, seL4 lacks a scalable user-level architecture. This would be the perfect place where Genode could step in. Genode would ultimately allow the seL4 community to move beyond static system scenarios.
Assuming that we succeed in drawing the attention of a broader audience to our project, we should make sure that Genode's API won't undergo major changes soon after this point. Today, I still see a number of deficiencies in the current API. In the past year, we successively moved to a new model of the API (dubbed server API) that promotes the construction of components as state machines rather than procedural programs. All recent components are designed in this style to the great benefit of their robustness. We should finally promote this style towards the base API and get rid of some mistakes we made in the past, in particular the reliance on side effects by using the globally visible Genode::env. I think that we should finalize this API renovation until the mid of 2016. This will also be right the time for updating the Genode book.
These are my thoughts about the upcoming year. Now I am curious about yours! :-)
Cheers Norman
[1] https://github.com/genodelabs/genode/issues/1552 [2] https://github.com/genodelabs/genode/issues/1000#issuecomment-161260312
My first thoughts on this are as follows: 1. I would love to use Genode as my primary OS, but I have been unable to make sufficient progress for any of my computers. The older ones don't have AHCI support, Genode's hardware support for my custom desktop computer is severely limited, and I don't even have any idea what's not working on my laptop because I haven't been able to get AMT to work. If you can add support for an alternative method of getting debugging output (or somehow support my desktop computer's motherboard) then I would happily try again to setup my own Turmvilla-like Genode system. 2. Cross-kernel binary compatibility would be great, especially since I find myself switching kernels for debugging purposes. 3. A Tails-like system would be awesome. According to Snowden, the NSA's main method of spying on Tor users is by compromising the Tor browser. 4. I would love graphical configuration utilities, but I would consider them a much lower priority than improved hardware support. I could make my own utilities, but I would need a lot of help getting Genode to support my hardware. On Dec 22, 2015 5:30 AM, "Norman Feske" <norman.feske@...1...> wrote:
Hello,
the end of the year 2015 is approaching. So it is time for planning our activities for the upcoming year. I plan to finalize the roadmap until mid of January. Hereby, I'd like to kick off the discussion. It goes without saying that everyone of you is invited to join in!
Before I come to my suggestions for 2016, let me briefly revisit the outcome of the roadmap for the past year. One year ago, I suggested three main topics to work on, namely the use of Genode as general-purpose OS, the advancement of our custom base-hw kernel platform, and the use of the seL4 kernel. In the discussions following my posting, many further points were raised, most prominently the need for through documentation, package management, and a sustainable quality of the base system in terms of stability and performance. In top of that, we received quite a few wishes of higher-level functionality such as a modern web browser or other typical desktop-computing features. Of course, we were not able to address all those topics but I am overly happy that we reached the point where a hand full of us (including me) are using Genode as their day-to-day OS. When I initially switched to using the Turmvilla scenario [1] at the end of May, the experience was admittedly a bit painful. But now, just a few months later, the beauty of the system becomes apparent as all the pieces come so nicely together. The performance, stability, and device-driver support have reached a level that leaves people impressed every time I have the chance to show off my system. Once people become interested, there is now the book available, which provides a smooth introduction into Genode. The feedback I receive about the book is overwhelmingly positive. So we did something right in 2015. :-)
After having passed the point where a few of us are able to use Genode as day-to-day OS, we should put the emphasis of the upcoming year on ways to make Genode accessible for a wider community. In a recent posting [2], I identified two possible ways to do that. The first way would be publishing a series of step-by-step guides that explain how to put Genode components together in order to create custom system scenarios. Those articles could be accompanied by screencasts or live-system images. Example scenarios could be the Turmvilla scenario, building a home-computer-like system for kids using the Raspberry Pi (like the Genode system my kids are using), building a network appliance like a data diode, tinkering with the USB Armory, etc. Those articles should be inviting to people who enjoy the building of systems. The second way would be to showcase a system with practical value to end users. I am thinking along the lines of a disposable OS like Tails that allows the user to browse the internet via the Tor network. But that is just an idea.
In this spirit, I propose the overall focus of 2016 to be: Let us make Genode accessible to the world outside the inner circle of us enthusiasts.
On a technical level, this motive implicates the following topics:
The deployment and management of Genode systems, i.e., by bringing forward Emery's work on the Nix package manager. This direction also reinforces the need to achieve binary compatibility between the various base platforms to make the distribution of binary packages, reproduceable builds, and continues test-and-integration scalable.
Supplementing Genode with user-friendly means to configure the system (e.g., wireless network, audio, display settings). Right now, I use Vim as configuration front end, which is cool, but also intimidating to less technical-minded users.
Accommodation of common desktop use cases like plugging in a USB stick to work with the files stored on it. Also disk encryption comes into mind.
Optimization of Genode for the use on a laptop, e.g., addressing fan control, power management, suspend/resume, and similar features.
There are also other possible avenues to support the stated goal:
Identifying ways of how Genode could play a role in related projects like Qubes OS. For example, we could promote the use of Genode as a tool for building App VMs as Qubes subsystems. Granted, this scenario leaves the architectural benefits of Genode with respect to its small TCB complexity unused as Qubes still relies on Xen, and Linux as Dom0. But Genode would still (possibly) provide value to the Qubes project. Maybe, there would be the prospect to replace Dom0 with Genode in the longer term? However, to drive this direction of work, we would certainly need someone who is actually using Qubes and has the natural incentive to work on such an integration.
Making Genode-based systems easily deployable on Amazon's EC2. Similar to the previous point, it would be beneficial to have someone working on this topic who is naturally interested in cloud computing.
Foster the cross-pollination of the seL4 and Genode communities. I got enthusiastic responses about my seL4-related work. There is definitely a strong interest in this kernel and a growing anticipation for formally verified software. Today, seL4 lacks a scalable user-level architecture. This would be the perfect place where Genode could step in. Genode would ultimately allow the seL4 community to move beyond static system scenarios.
Assuming that we succeed in drawing the attention of a broader audience to our project, we should make sure that Genode's API won't undergo major changes soon after this point. Today, I still see a number of deficiencies in the current API. In the past year, we successively moved to a new model of the API (dubbed server API) that promotes the construction of components as state machines rather than procedural programs. All recent components are designed in this style to the great benefit of their robustness. We should finally promote this style towards the base API and get rid of some mistakes we made in the past, in particular the reliance on side effects by using the globally visible Genode::env. I think that we should finalize this API renovation until the mid of 2016. This will also be right the time for updating the Genode book.
These are my thoughts about the upcoming year. Now I am curious about yours! :-)
Cheers Norman
[1] https://github.com/genodelabs/genode/issues/1552 [2] https://github.com/genodelabs/genode/issues/1000#issuecomment-161260312
-- Dr.-Ing. Norman Feske Genode Labs
http://www.genode-labs.com · http://genode.org
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
Hello Ben,
thanks for your comments!
My first thoughts on this are as follows:
- I would love to use Genode as my primary OS, but I have been unable
to make sufficient progress for any of my computers. The older ones don't have AHCI support, Genode's hardware support for my custom desktop computer is severely limited, and I don't even have any idea what's not working on my laptop because I haven't been able to get AMT to work. If you can add support for an alternative method of getting debugging output (or somehow support my desktop computer's motherboard) then I would happily try again to setup my own Turmvilla-like Genode system.
I have followed the SATA story [1] and appreciate your patience. It is unfortunate that it went to a dead end for now. We may consider obtaining the board you mentioned - but I can't give a promise as this line of work is just a side track of us.
[1] https://github.com/genodelabs/genode/issues/1547
Regarding the log output, there are two principle alternatives to AMT, enhancing NOVA or core with the ability to print those messages to the VGA console (similarly to how L4/Fiasco is doing it). Alex has a branch for this feature [2] but I do not know if it works with Genode 15.11. The second way would be to connect a UART ExpressCard, if your laptop is equipped with such a slot.
[2] https://github.com/alex-ab/genode/commits/experimental_vga_console
- I would love graphical configuration utilities, but I would consider
them a much lower priority than improved hardware support. I could make my own utilities, but I would need a lot of help getting Genode to support my hardware.
I am afraid that hardware support will never be enough. With this thinking, we could never even start to move Genode forward on the hardware that it already supports well.
I have seen Turmvilla running decently on a variety of Intel-based laptops equipped with Intel wireless, Intel E1000 networking, Intel integrated graphics, and Intel HDA audio. Why not to take this rough specification as the requirement for running Genode and declare other platforms as unsupported?
Having us few developers constantly extend the range of supported hardware does not scale, and since our testing capabilities are limited, regressions would be likely to occur. So the effort would eventually go wasted. Unless the burden of extending and regularly testing the hardware support is carried by a growing developer community, it remains unfeasible. So I'd prefer to focus on the narrow range of PC platforms mentioned above. For starting out with Genode, a refurbished Lenovo x201 that costs merely 200 USD is just perfect - especially compared to the investment of time needed to support other hardware (I am looking at the back-and-forth of our attempt to enable your SATA device [1]).
Please don't get me wrong. I am not dismissing hardware support as a working topic. We will need to continuously work on that. But when doing so, I'd like to address current-generation or next-generation hardware.
Coming back to the actual discussion about the roadmap, I don't see improved hardware support appearing as a major topic as it is quite intangible. For someone studying the roadmap, "improved hardware support" tells not much.
Cheers Norman
On Wed, Dec 23, 2015 at 09:28:00AM +0100, Norman Feske wrote:
Please don't get me wrong. I am not dismissing hardware support as a working topic. We will need to continuously work on that. But when doing so, I'd like to address current-generation or next-generation hardware.
Lurker here,
I think the best goal we can have right now is supporting hardware that work with fully free software since relying on vendors to implement Genode-specific quirks is going to be a losing battle. Perhaps it'd be better to focus instead of generations, but by how hackable and long-term supported a project is.
Old Thinkpads can generally be maintained through replacing parts and using free BIOS. It also means if you say, install Libreboot (or Autoboot, or even Coreboot if you're that hardware) you can get debug support for your processor that might not support AMT.
ARM is where I'm hanging around at the moment given there's boards out there with schematics based on long-term chipsets used by the automotive industry. For instance, I'm typing this from an i.MX6 chipset on a Novena board. A lot of the microkernels already have support for this chipset, I'd just need to get the Novena tools running on it. Thanks to etnaviv it has free graphics drivers.
On the opposite end we have low-end boards that people use for toying with. Beaglebones, Banana Pis, the USB Armory. These would be incredibly useful to point people to if they want to toy with Genode without setting up an expensive machine. These all run U-Boot which could be used to provide serial support.
Cheers Norman
Cheers, Jookia.
Hi Jookia,
thanks for joining the discussion!
ARM is where I'm hanging around at the moment given there's boards out there with schematics based on long-term chipsets used by the automotive industry. For instance, I'm typing this from an i.MX6 chipset on a Novena board. A lot of the microkernels already have support for this chipset, I'd just need to get the Novena tools running on it. Thanks to etnaviv it has free graphics drivers.
Do you have any concrete plans for Genode on the Novena board?
Cheers Norman
On Thu, Dec 24, 2015 at 12:47:13PM +0100, Norman Feske wrote:
Hi Jookia,
thanks for joining the discussion! Do you have any concrete plans for Genode on the Novena board?
Concrete? Hmm. I'm still setting up NixOS on it and documenting my setup in a system-agnostic way which could be translated to Genode. I'd like to use Genode on it as a replacement for my existing system eventually. I haven't done too much more research as I'm mainly waiting for a package manager. Ideally I'd like to leverage the isolation I get on Genode to compose a complex system.
Configuring firewalls and routes when you have multiple interfaces, including containers on GNU/Linux is a nightmare. I tried to avoid containers as much as I could, but eventually I had to give in as there's no way to have per-interface DNS on GNU unless you're running a container, due to system state.
I'd like to have better sandboxing for my applications. I could use containers in NixOS, and it's tempting as I don't have the pay much of the cost of duplicating a rootfs, but still complex to set up when you want to share data such as through a GUI or file system. One good reason for this is security, but development is probably a bigger factor for me.
I'd like to allocate resources and CPU time to processes easier. Right now I've installed BFQ and BFS in my Linux kernel which will magically speed up my system, though I still don't have the knowledge or tools to set up cgroups to limit resources. I'm not even trying to do anything too amazing, it's just difficult to say to my computer 'Hey, run my photo rendering but do it in the background when I'm not using my disk or CPU' without doing all this and installing schedtool (which I haven't managed to set up yet.)
It's not that I don't have time to learn these things, I'm fortunate enough that I do- it's that it's tedious even for someone who does somewhat low level hacking. The more I try and get my computer to do multiple things the more it feels like I don't have the tools for it to do them. Not because I don't have the hardware or software, but because I don't have enough isolation for things to not clobber each other.
That's my motivation. So off the top of my head for things I'd need in Genode: Free software, and lots of it. A GTK port, a Tor port, an OpenVPN port, something like JACK for audio (routing applications mainly), video and photo editing through Blender and RawTherapee, web browsing through Tor Browser, some kind of routing system that can handle NAT along with a network stack suited for a router (DHCP, IPv6, etc), and a 'proper' firewall system.
Having GNU/Linux running in TrustZone and using Xpra to view them in Genode would help- though I'd still want to port my free video drivers (etnaviv) to Genode. Unfortunately I'm certain that's out of my skill level and the half a dozen people working on etnaviv are much too busy for this. Unless I can do porting without understanding more than the modules and APIs. From what I know I can't run the i.MX6 GPU in the TrustZone side, so this might actually be the 'one thing' that stops me from using Genode as I tend to need acceleration for video playback. I'm still not sure about this. I'd be fine doing the work if it doesn't require in-depth graphics knowledge but more grunt work.
Now, I mentioned 'proper' firewall system above in quotes. I'm not going to do another rant, but rather focus on what I'd like to see on a firewall system: Integration with system routing. I'd like to focus on per-application rules rather than per-interface or per-port, and multiplexers to combine applications or interfaces. I'd also like to see applications that do routing too. My current rules are complex and involve giving applications (defined by port or user) access to interfaces based on interface and subnet.
I've been toying with the idea of running a VPN in a container on GNU/Linux and exporting a HTTP proxy so it handles DNS automatically while having no DNS or direct Internet access on the host. In Genode this could be replaced by routing applications to a router application which is composed with OpenVPN perhaps.
Either way, that's my brain dump for now. I'm eager to get Genode going on the Novena and probably more obscure hardware in the future like lowRISC if I ever see a way to get a kernel on it. Having a fully free software stack allows me to do all this. Happy holidays in UTC+12!
Cheers Norman
Cheers, Jookia.
On Thu, Dec 24, 2015 at 11:46:42PM +1100, Jookia wrote:
Now, I mentioned 'proper' firewall system above in quotes. I'm not going to do another rant, but rather focus on what I'd like to see on a firewall system: Integration with system routing. I'd like to focus on per-application rules rather than per-interface or per-port, and multiplexers to combine applications or interfaces. I'd also like to see applications that do routing too. My current rules are complex and involve giving applications (defined by port or user) access to interfaces based on interface and subnet.
A nice diagram that covers networking emotions: https://ipfs.pics/QmdQZ3NrrwWSqhzuUQeT6Qp8iW6Zy7RoPSnKqyiCDaYyj6
I think we have some similar networking goals, I've been taking a break on my main projects to try and port Cjdns, which is a meshnetworking engine. Wireless links are the obvious transport medium, but almost all links are actually UDP tunnels over the internet as a practical matter, so what you get in effect is a giant VPN. Because the address allocation is distributed (a requirement for a mesh network) it could be a good mate to Genode because each application can get its own IP and you can maybe throw away port numbers and firewalls. The problem is the code base is more or less maintained by single person, and the addresses is not routable back and forth to the arpanet, but you can create tunnels.
https://github.com/cjdelisle/cjdns I am over here http://fc00.org/#fcc4:c658:e905:393d:529f:5473:055f:4298
It may or may not work, but either way I'm porting libuv, which is a nice library for writing networked servers, so atleast then we get a nice base to start writing simple things like DNS proxies. Its actually been quite satisfying to port because the low-level lwIP callback interface is so much simplier than POSIX sockets, especially when all you want to do is trigger a callback.
Cheers, Emery
On Thu, Dec 24, 2015 at 03:22:44PM +0100, Emery Hemingway wrote:
A nice diagram that covers networking emotions: https://ipfs.pics/QmdQZ3NrrwWSqhzuUQeT6Qp8iW6Zy7RoPSnKqyiCDaYyj6
Yikes, though defense in depth is a valid strategy. ;) At home I'm running libreCMC on my router with a bridged modem.
I think we have some similar networking goals, I've been taking a break on my main projects to try and port Cjdns, which is a meshnetworking engine. Wireless links are the obvious transport medium, but almost all links are actually UDP tunnels over the internet as a practical matter, so what you get in effect is a giant VPN. Because the address allocation is distributed (a requirement for a mesh network) it could be a good mate to Genode because each application can get its own IP and you can maybe throw away port numbers and firewalls. The problem is the code base is more or less maintained by single person, and the addresses is not routable back and forth to the arpanet, but you can create tunnels.
Ironically I haven't gotten ipfs to work over Tor yet since it doesn't have proxy support. I've always wanted to set up a mesh network but I haven't found enough people (more than one) in my local area.
I've been working on a project (almost out of design stage) in regard to networking, but I haven't thought of the idea of having per-application addresses. Looking at IPv6, this seems obvious in retrospect. I'm not too concerned personally about compatibility with the Internet, I'd much rather see us building overlay networks and transitioning the overlays to hardware.
https://github.com/cjdelisle/cjdns I am over here http://fc00.org/#fcc4:c658:e905:393d:529f:5473:055f:4298
I've seen cjdns, though I've looked more at BATMAN. They probably serve two different purposes though. I spend most my time toying with Tor.
It may or may not work, but either way I'm porting libuv, which is a nice library for writing networked servers, so atleast then we get a nice base to start writing simple things like DNS proxies. Its actually been quite satisfying to port because the low-level lwIP callback interface is so much simplier than POSIX sockets, especially when all you want to do is trigger a callback.
Quite interesting! One peculiar problem I've hit which caused me to use a container is that DNS resolution is done through the C library, which in GNU tends to mean 'read /etc/resolv.conf and use that'. There's no way to say 'get me a DNS server through this interface', there's no concept of it. A DNS proxy would be a great help, though 'proxy' in Genode terms can mean a lot of things. Perhaps 'service' would work better?
Shamelessly hijacking the topic since I forgot to mention it in my post earlier and it kind of relates: A distributed operating system would be a great thing to have these days, and Genode's IPC means we could have an easier way of proxying our problems over the network. We're already going to do some work to have a GUI interface to multiple VMs, so it wouldn't be much work to have those VMs be physical machines or on other machines. In the end it'd be nice to have a set of computers configured centrally and accessible by however thin a client needed, and perhaps extended to allow migration across compatible devices.
It'd be really nice to see a system where I can work on my desktop, boot up my laptop, and alongside being able to remote in, transfer over actual running applications and then turn off my desktop. Looking more at the application level, it'd be an interesting thought experiment to look at this as less of an OS or application that needed to be migrated, and more of process trees and their resources. Having routing done by the process tree you migrate and IPs for each application sounds like it'd allow roaming between machines.
Cheers, Emery
Cheers, Jookia.
Well, it seems that Genode is now working on my laptop. I used the Turmvilla run script (with a few modifications), and it seems to work for the most part (This isn't the place to discuss the remaining issues.)
As for suggestions, is there any difficulty or drawback in switching from 16-bit color to 24/32-bit color? This isn't particularly important, but it seems like it shouldn't be very hard.
On Wed, Dec 23, 2015 at 1:28 AM, Norman Feske <norman.feske@...1...> wrote:
Hello Ben,
thanks for your comments!
My first thoughts on this are as follows:
- I would love to use Genode as my primary OS, but I have been unable
to make sufficient progress for any of my computers. The older ones don't have AHCI support, Genode's hardware support for my custom desktop computer is severely limited, and I don't even have any idea what's not working on my laptop because I haven't been able to get AMT to work. If you can add support for an alternative method of getting debugging output (or somehow support my desktop computer's motherboard) then I would happily try again to setup my own Turmvilla-like Genode system.
I have followed the SATA story [1] and appreciate your patience. It is unfortunate that it went to a dead end for now. We may consider obtaining the board you mentioned - but I can't give a promise as this line of work is just a side track of us.
[1] https://github.com/genodelabs/genode/issues/1547
Regarding the log output, there are two principle alternatives to AMT, enhancing NOVA or core with the ability to print those messages to the VGA console (similarly to how L4/Fiasco is doing it). Alex has a branch for this feature [2] but I do not know if it works with Genode 15.11. The second way would be to connect a UART ExpressCard, if your laptop is equipped with such a slot.
[2] https://github.com/alex-ab/genode/commits/experimental_vga_console
- I would love graphical configuration utilities, but I would consider
them a much lower priority than improved hardware support. I could make my own utilities, but I would need a lot of help getting Genode to support my hardware.
I am afraid that hardware support will never be enough. With this thinking, we could never even start to move Genode forward on the hardware that it already supports well.
I have seen Turmvilla running decently on a variety of Intel-based laptops equipped with Intel wireless, Intel E1000 networking, Intel integrated graphics, and Intel HDA audio. Why not to take this rough specification as the requirement for running Genode and declare other platforms as unsupported?
Having us few developers constantly extend the range of supported hardware does not scale, and since our testing capabilities are limited, regressions would be likely to occur. So the effort would eventually go wasted. Unless the burden of extending and regularly testing the hardware support is carried by a growing developer community, it remains unfeasible. So I'd prefer to focus on the narrow range of PC platforms mentioned above. For starting out with Genode, a refurbished Lenovo x201 that costs merely 200 USD is just perfect - especially compared to the investment of time needed to support other hardware (I am looking at the back-and-forth of our attempt to enable your SATA device [1]).
Please don't get me wrong. I am not dismissing hardware support as a working topic. We will need to continuously work on that. But when doing so, I'd like to address current-generation or next-generation hardware.
Coming back to the actual discussion about the roadmap, I don't see improved hardware support appearing as a major topic as it is quite intangible. For someone studying the roadmap, "improved hardware support" tells not much.
Cheers Norman
-- Dr.-Ing. Norman Feske Genode Labs
http://www.genode-labs.com · http://genode.org
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
Hi Norman, and other fellow Genodians-
I like the direction Genode is going... here are a few reflection's following Norman's call for discussion:
* Norman wrote: " *Supplementing Genode with user-friendly means to configure...**Right now, **I use Vim as configuration front end, which is cool...*" My thought: One can certainly get a lot of configuration usability out of a basic, potentially colored, 2D ASCII text document. Not to mention how much lighter and more portable the (potentially remote) terminal display can be. So while GUI tool kits are great for some things, I'd like it if little headless nodes running Genode (Snickerdoodle http://krtkl.com/) would be just as easy to re/configure. Thus I vote that Genode's next generation of [ASCII] configuration files are both very easy to read, in addition to edit with a (potentially augmented) terminal emulation program... i.e. lets not prematurely jump on the GUI bandwagon, before Genode achieves a similar ease of configuration using a significantly lighter level of display / keyboard interaction... given the goal stated above, why not at the same time provide a very nice set of terminal interaction API's within the Genode framework, that one can count on as a lower level system resource, such that other light weight apps that run on Genode can also count that this terminal interaction API might be made available as shared resource.
* Norman writes: "*bringing **forward Emery's work on the Nix package manager.*" My thought: the Nix pure function based package management system looks interesting.. but I do have some concern that the Genode project might in some way mange to align it self with the emerging standards for application container management... Now I'm not saying that Nix/Genode would not have something say as it joins the container standardization party.. This may be all the more reason to make the effort toward fully functional integration with the emerging OS (potentially bootable) and Application container standards.
* Yes auto mounting USB thumb drives (using a fully protected USB driver stack) would be great...
* Yes, a state machine based perspective makes a great foundation for the design of Genode API's... Yes..Yes bring it on!!
* Norman bring up the seL4 formally verified micro kernel.. In similar directions, I would like to see Genode development explore the potentials of working relationships with LowRisc http://www.lowrisc.org/, as well as BERI and BERI2 (formaly verifyed) CPU projects http://www.cl.cam.ac.uk/research/security/ctsrd/beri/... these are all security oriented, open source CPU hardware design research projects... where I gather the perspective of those at the top of there game designing the Genode OS would be more than welcomed.... I'll be happy if I can keep up with the innovation that is likely to result from such a collaboration...
Hope this helps all the best Peter (SaxMan) Lindener
Hello Peter,
thanks for your sharing your thoughts. Let me please comment on your remarks about configuration mechanisms for Genode...
On Tue, Dec 22, 2015 at 02:08:17PM -0800, Peter Lindener wrote:
- Norman wrote: " *Supplementing Genode with user-friendly means to
configure...**Right now, **I use Vim as configuration front end, which is cool...*" My thought: One can certainly get a lot of configuration usability out of a basic, potentially colored, 2D ASCII text document. Not to mention how much lighter and more portable the (potentially remote) terminal display can be. So while GUI tool kits are great for some things, I'd like it if little headless nodes running Genode (Snickerdoodle http://krtkl.com/) would be just as easy to re/configure. Thus I vote that Genode's next generation of [ASCII] configuration files are both very easy to read, in addition to edit with a (potentially augmented) terminal emulation program... i.e. lets not prematurely jump on the GUI bandwagon, before Genode achieves a similar ease of configuration using a significantly lighter level of display / keyboard interaction...
I suggest you have a look at our release documentation for Genode 15.11 [1]. There we explain how Genode components are configured individually and assembled to meet functional and security requirements of specific use-case scenario. Furthermore, the basis for our configuration is XML syntax since the beginning (see [2] and [3]). For thorough documentation of the concept of component configuration you may have a look into the Genode Foundations book Chapter 4.6.
From my point of view, this sheds some light on Norman's comment about
using Vim to configure Genode, which should be taken literally: Any aspect of the system can be configured by editing XML config files and additionally our reporting mechanism is using the XML syntax too. I admit that I'm not striving for mere ASCII support but envision to use UTF-8 in the future for XML attribute values and node content, but that's a small detail. The mentioned appealing GUI "agents" are nothing but alternative front ends, but on the other hand important to get less technical users on board.
[1] http://genode.org/documentation/release-notes/15.11#Uniform_run-time_configu... [2] http://genode.org/documentation/release-notes/12.05#System_reconfiguration_a... [3] http://genode.org/documentation/developer-resources/init
given the goal stated above, why not at the same time provide a verynice set of terminal interaction API's within the Genode framework, that one can count on as a lower level system resource, such that other light weight apps that run on Genode can also count that this terminal interaction API might be made available as shared resource.
If I'm not mistaking this comment entirely, I'd like to refer you to our neat APIs for XML handling, which are the Xml_node and the Xml_generator. We apply these tools throughout the Genode source if configurations and reports are processed. The most prominent component using them is Init, but also the windows manager components, our platform driver, device drivers, VirtualBox, and many more.
So, what's currently missing for your use case may be a terminal-based "agent" for dynamic configuration if Vim is unsuitable. I personally see no need to add ncurses-like tools tailored for configuration, but your opinion may differ.
Regards
Hi Christian-
Yes I get it that Genode already has a fairly nice configuration process utilizing XML.... and you might be on the right track when you take my comment about the use of a potentially augmented text / XML structure editor for easier display / editing of these XML config files.. ... but I have to confess I'm not all so much of a fan of XML in terms of it's direct readability.... maybe it's a good middle ground... but in all honesty, I would prefer a Json like syntax over XML.... or more exactly... When people suggest that every thing should be encoded as an XML file... I tend to concur with those in t crowed who just role there eyes... That is: there does seem potential for the use of domain specific languages... i.e. XML does not always seem to be the best answer... I'm just suggesting that the complexity jump to GUI based config may not be the best trade-off in the context of smaller embedded Genode systems..
Hope this helps. -Peter
On Wed, Dec 23, 2015 at 3:21 AM, Christian Helmuth < christian.helmuth@...1...> wrote:
Hello Peter,
thanks for your sharing your thoughts. Let me please comment on your remarks about configuration mechanisms for Genode...
On Tue, Dec 22, 2015 at 02:08:17PM -0800, Peter Lindener wrote:
- Norman wrote: " *Supplementing Genode with user-friendly means to
configure...**Right now, **I use Vim as configuration front end, which
is
cool...*" My thought: One can certainly get a lot of configuration usability
out
of a basic, potentially colored, 2D ASCII text document. Not to mention
how
much lighter and more portable the (potentially remote) terminal display can be. So while GUI tool kits are great for some things, I'd like it if
little
headless nodes running Genode (Snickerdoodle http://krtkl.com/) would
be
just as easy to re/configure. Thus I vote that Genode's next generation of [ASCII] configuration files are both very easy to read, in addition to edit with a (potentially augmented) terminal emulation program... i.e. lets not prematurely jump on the GUI bandwagon, before Genode achieves a similar ease of configuration using a significantly lighter level of display / keyboard interaction...
I suggest you have a look at our release documentation for Genode 15.11 [1]. There we explain how Genode components are configured individually and assembled to meet functional and security requirements of specific use-case scenario. Furthermore, the basis for our configuration is XML syntax since the beginning (see [2] and [3]). For thorough documentation of the concept of component configuration you may have a look into the Genode Foundations book Chapter 4.6.
From my point of view, this sheds some light on Norman's comment about using Vim to configure Genode, which should be taken literally: Any aspect of the system can be configured by editing XML config files and additionally our reporting mechanism is using the XML syntax too. I admit that I'm not striving for mere ASCII support but envision to use UTF-8 in the future for XML attribute values and node content, but that's a small detail. The mentioned appealing GUI "agents" are nothing but alternative front ends, but on the other hand important to get less technical users on board.
[1] http://genode.org/documentation/release-notes/15.11#Uniform_run-time_configu... [2] http://genode.org/documentation/release-notes/12.05#System_reconfiguration_a... [3] http://genode.org/documentation/developer-resources/init
given the goal stated above, why not at the same time provide a verynice set of terminal interaction API's within the Genode framework, that one can count on as a lower level system resource, such that other light weight apps that run on Genode can also count that this terminal interaction API might be made available as shared resource.
If I'm not mistaking this comment entirely, I'd like to refer you to our neat APIs for XML handling, which are the Xml_node and the Xml_generator. We apply these tools throughout the Genode source if configurations and reports are processed. The most prominent component using them is Init, but also the windows manager components, our platform driver, device drivers, VirtualBox, and many more.
So, what's currently missing for your use case may be a terminal-based "agent" for dynamic configuration if Vim is unsuitable. I personally see no need to add ncurses-like tools tailored for configuration, but your opinion may differ.
Regards
Christian Helmuth Genode Labs
http://www.genode-labs.com/ · http://genode.org/ https://twitter.com/GenodeLabs · /ˈdʒiː.nəʊd/
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
Hi Peter,
of it's direct readability.... maybe it's a good middle ground... but in all honesty, I would prefer a Json like syntax over XML.... or more exactly... When people suggest that every thing should be encoded as an XML file... I tend to concur with those in t crowed who just role there eyes...
please don't mistake Genode's reliance on XML as our only and all-encompassing solution to everything. XML is not necessarily the user-facing configuration front end. To draw an analogy, XML is to Genode's configuration concept what an intermediate representation (IR) is to a compiler. While it is possible to write programs in LLVM's IR, programmers usually use a front-end language like C.
We picked XML as our "intermediate representation" as it is easy to parse, easy to generate, expressive, has no ambiguities, comes with no surprises, and is human-manageable. Whether XML or json or S-expressions or something else would be the "best" syntax is debatable. It ultimately comes down to personal taste so that there is no conclusive answer. However, what is not debatable is the need for a _coherent_ concept throughout the system. So we had to make a choice and apply is consequently. We picked XML and it delivered exactly what we expected. In fact, I have never heard a compelling argument against using it for our purpose - other than personal taste (which is actually not a compelling argument).
Personally, I am not a friend of editing XML by hand. But I am also no friend of editing LaTeX by hand. Still I am a huge fan of LaTeX. To make using LaTeX more convenient for me, I'm using a preprocessor (GOSH). The same idea can be applied for the configuration of a Genode system. It is entirely possible to come up with domain-specific languages for different configuration needs. The "compiler" of such a language would output XML as the intermediate format understood by Genode's components.
Conceptually, such a transformation tool that transform a convenient domain-specific language to a XML-formatted configuration of a Genode component is simply a configuration front end. Its role within the system would be not any different from a GUI-based configuration front end. When deciding where to put our efforts in near future, I would opt for the GUI direction for two reasons: Even though I don't find the editing of XML files super convenient, I am perfectly okay with it for the time being. I don't feel that it stands in my way. Second, a GUI-based means to configure Genode makes Genode approachable for a wider user base (another textual front end won't achieve that). E.g., coming back to the example of a throw-away Tor browser OS, it is crucially important that the user can join to a wireless network by a few easily discoverable mouse clicks.
Cheers Norman
Hello list,
For this next year I would personally like to see Genode host more network services, so more library support, more configuration idiom, and better support for virtual hosting environments. This is because I would like to migrate a few services that I manage myself, and to make Genode binary distribution self-hosted. Competing with Linux and BSD on performance is quite literally an uphill battle, but I feel there will be some great benefits hidden in Genode's IP stack isolation.
On the subject of configuration, perhaps components could export an XML description of their configuration options over a report session, and then a generic Qt GUI could dynamicly generate a configuration dialog? I am imagining that there could be a nice library to handle both configuration declaration, documentation, and retrieval.
I of course want to polish up Nix and make it a pratical tool for other people to start using. My first priority is to modify the Make build system to build arbitrary library targets and to link components against external objects (actually just those library targets, but at arbitrary locations). The second priority will be to fetch binaries over the network. I am sort of working on the second right now, and the second can supplant the first, but robust self hosting is something that I want to be done by early next year.
My only Linux experience has been with Gento and NixOS, and I really don't want to accidentally create a distribution with unwilling maintainers (like me), so as we go forward I want to do my best to optimize for some sort of modular and anarchistic distribution model.
On Tue, Dec 22, 2015 at 02:08:17PM -0800, Peter Lindener wrote:
My thought: the Nix pure function based package management system looks interesting.. but I do have some concern that the Genode project might in some way mange to align it self with the emerging standards for application container management... Now I'm not saying that Nix/Genode would not have something say as it joins the container standardization party.. This may be all the more reason to make the effort toward fully functional integration with the emerging OS (potentially bootable) and Application container standards.
If a Nix generated Genode system is going to be embeddable in another OS, I would want to make the interface sharply defined and not integrated, that is there would be minimal crosstalk between the two. I don't know much about what is going on with containerisation but I think want to try and make something that is easy to wrap, not externally integrated, if that makes any sense.
On that point, when I was at NixCon this year I there was some talk of fixing .NET DLL hell and better Windows deployment support, so if all that Windows goop can be tolerated then it will be easy to wiggle Genode in.
Anyway, good hacking to you all next year, Emery
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi Emery,
thanks a lot for sharing your plans!
On 24.12.2015 13:08, Emery Hemingway wrote:
For this next year I would personally like to see Genode host more network services, so more library support, more configuration idiom, and better support for virtual hosting environments. This is because I would like to migrate a few services that I manage myself, and to make Genode binary distribution self-hosted. Competing with Linux and BSD on performance is quite literally an uphill battle, but I feel there will be some great benefits hidden in Genode's IP stack isolation.
On the subject of configuration, perhaps components could export an XML description of their configuration options over a report session, and then a generic Qt GUI could dynamicly generate a configuration dialog? I am imagining that there could be a nice library to handle both configuration declaration, documentation, and retrieval.
I am actually thinking in a similar direction but looking at the problem from the "desktop" perspective. Technically, I don't see a component as an ELF executable. I think that we will need to define some kind of container format with a standardized structure. The content of such archive could be inspected by a management component (e.g., "desktop") to configure and dynamically integrate the component into the Genode system. The archive should contain meta data such as a formal description of the component configuration like you have in mind, the type of the component, the dependencies from other components as well as the component's binary (or even more than one binary such as shared libraries) and other data.
I of course want to polish up Nix and make it a pratical tool for other people to start using. My first priority is to modify the Make build system to build arbitrary library targets and to link components against external objects (actually just those library targets, but at arbitrary locations). The second priority will be to fetch binaries over the network. I am sort of working on the second right now, and the second can supplant the first, but robust self hosting is something that I want to be done by early next year.
Very cool!
My only Linux experience has been with Gento and NixOS, and I really don't want to accidentally create a distribution with unwilling maintainers (like me), so as we go forward I want to do my best to optimize for some sort of modular and anarchistic distribution model.
I hope that throughout the year, we will learn what a "distribution" actually means in the context of Genode. Right now, I still fail to get a grip on it.
If a Nix generated Genode system is going to be embeddable in another OS, I would want to make the interface sharply defined and not integrated, that is there would be minimal crosstalk between the two. I don't know much about what is going on with containerisation but I think want to try and make something that is easy to wrap, not externally integrated, if that makes any sense.
Perfect sense. :-)
Cheers Norman
- -- Dr.-Ing. Norman Feske Genode Labs
http://www.genode-labs.com · http://genode.org
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
On 12/22/15 13:28, Norman Feske wrote:
First, congrats on getting the Turmvilla scenario to work so well. Please make that easy to repeat for others.
I'll also like to encourage the team to continue on the SeL4 kernel. Although each of the other kernels, NOVA, Fiasco, etc are probably more secure than any Linux or BSD ever can be, the fact that these are little known doesn't offer the confidence that these are actually better choice. Building on top of SeL4, that has that reputation, could boost the perception of Genode being a viable alternative to Linux/BSD in the minds of a larger public.
This year, I'll go forward in my quest to get a web server running on Genode in production.
For that my wish list is: - better ways to debug the system; - removal of the catchall's that hide information; - insight into process' activities: * where's my CPU being eaten; * who's talking to whom, or who stopped talking; - IPv6; what's a website without it these days;
When that works, I'll focus on getting my Go-lang programs running. These programs manage private keys and certificates and maintain state in a sqlite3 database.
With that, my year will be full :-)
In other thoughts, having a throwaway distro like Tails on top of Genode is a great idea. It would certainly get Genode on the radar of security-minded people.
However, as I see it, the kernel is the least of the worries. It's the monolitic nature of Firefox that needs to be tamed. The real power of Genode comes, imho, from splitting monolitic programs into separate sandboxes.
The low hanging fruit: split off image parsing into a separate process. This process receives a stream of data and returns a memory space with a bit-blittable image that can be copied to the frame buffer.
More challenging would be to break Firefox into more and more smaller parts with the ulimate goal: Every parser into its own sandbox.
I'd love to see Genode provide the mechanisms and support to make breaking up monolitic programs easily. That's where it's power shines.
Or am I preaching to the choir?
Cheers, Guido.
Should we be porting Firefox or Chromium? Chromium is already designed to use sandboxing. Also, given the critical security role of web browsers, it seems most logical to heavily modify whichever browser we port to take full advantage of Genode's security features. On Jan 1, 2016 6:44 AM, "Guido Witmond" <guido@...231...> wrote:
On 12/22/15 13:28, Norman Feske wrote:
First, congrats on getting the Turmvilla scenario to work so well. Please make that easy to repeat for others.
I'll also like to encourage the team to continue on the SeL4 kernel. Although each of the other kernels, NOVA, Fiasco, etc are probably more secure than any Linux or BSD ever can be, the fact that these are little known doesn't offer the confidence that these are actually better choice. Building on top of SeL4, that has that reputation, could boost the perception of Genode being a viable alternative to Linux/BSD in the minds of a larger public.
This year, I'll go forward in my quest to get a web server running on Genode in production.
For that my wish list is:
- better ways to debug the system;
- removal of the catchall's that hide information;
- insight into process' activities: * where's my CPU being
eaten; * who's talking to whom, or who stopped talking;
- IPv6; what's a website without it these days;
When that works, I'll focus on getting my Go-lang programs running. These programs manage private keys and certificates and maintain state in a sqlite3 database.
With that, my year will be full :-)
In other thoughts, having a throwaway distro like Tails on top of Genode is a great idea. It would certainly get Genode on the radar of security-minded people.
However, as I see it, the kernel is the least of the worries. It's the monolitic nature of Firefox that needs to be tamed. The real power of Genode comes, imho, from splitting monolitic programs into separate sandboxes.
The low hanging fruit: split off image parsing into a separate process. This process receives a stream of data and returns a memory space with a bit-blittable image that can be copied to the frame buffer.
More challenging would be to break Firefox into more and more smaller parts with the ulimate goal: Every parser into its own sandbox.
I'd love to see Genode provide the mechanisms and support to make breaking up monolitic programs easily. That's where it's power shines.
Or am I preaching to the choir?
Cheers, Guido.
genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
On Fri, Jan 01, 2016 at 04:44:30PM -0700, Nobody III wrote:
Should we be porting Firefox or Chromium? Chromium is already designed to use sandboxing. Also, given the critical security role of web browsers, it seems most logical to heavily modify whichever browser we port to take full advantage of Genode's security features.
Porting Tor Browser (based on Firefox) is probably one of the best moves that could be done given how privacy-orientated it is above any other browser. It's a requisite for having proper anonymous web browisng on Genode.
Although it's not a security matter, Chromium doesn't exactly have a good history when it comes to keeping the code base free. On a development level, it just seems to be code meant to be turned in to Chrome, not an intentional browser by itself like Firefox.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi Guido,
thank you for your remarks regarding the road map.
On 01.01.2016 14:43, Guido Witmond wrote:
For that my wish list is: - better ways to debug the system; - removal of the catchall's that hide information;
This should be examined case by case. The best way to go about it would be to open individual issues for instances in the code that you find questionable. There we can discuss the code and possibly improve it. In my opinion, however, this point is too minor to be mentioned as part of the road map.
- insight into process' activities: * where's my CPU being eaten;
- who's talking to whom, or who stopped talking;
There are already some tools in place that may of help, like the CPU load display, or the TRACE service of core. I agree that improvements of these tools, or the creation of proper profiling tools would be great. But it also takes a lot of effort. Without funding or significant participation of the community outside of Genode Labs, I feel that we won't be able to make big advances in this respect in 2016.
- IPv6; what's a website without it these days;
Are you going to investigate this topic? If yes, can you share which directions you find worth exploring?
Cheers Norman
- -- Dr.-Ing. Norman Feske Genode Labs
http://www.genode-labs.com · http://genode.org
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
Hello,
On 22.12.2015 13:28, Norman Feske wrote:
Today, I still see a number of deficiencies in the current API. In the past year, we successively moved to a new model of the API (dubbed server API) that promotes the construction of components as state machines rather than procedural programs. All recent components are designed in this style to the great benefit of their robustness. We should finally promote this style towards the base API and get rid of some mistakes we made in the past, in particular the reliance on side effects by using the globally visible Genode::env.
in the meantime, I substantiated this topic a bit more:
https://github.com/genodelabs/genode/issues/1832
Cheers Norman
Hi,
This is my first post on the list and I will try to substantiate some of my ideas/wishes regarding the future of Genode as requested by Norman.
First, I must say that I still have some trouble understanding the use case for Genode as a desktop OS. Not that I don't like the idea but I think this is such an enormous undertaking that a believe might be like aiming too far or maybe like missing lower hanging fruit. It might be better to shorten the hops to the future in easier and faster achievable milestones. Genode then gets truly usable faster, albeit within a smaller community, but one that truly cares. It also gets tried in production sooner, which is very important to gain stability and real user feedback.
One such step might be to make Genode a viable and usable platform for certain domains within embedded computing. This is a much easier undertaking as embedded systems not have the same requirements regarding ease of use and completeness regarding features. The GUI subsystem goes away and hardware support need not be as broad as for desktop OSes. I also think that Genode has several additional properties that are highly sought after in various embedded domains like responsiveness, stability and power efficiency. These characteristics should be exploited together with the main benefit of Genode as a secure and safe OS to gain maximum impact.
I am glad to see that you have found the state machine being a usable pattern. I happen to believe that it would be very beneficial for Genode to mostly switch to a fully asynchronous communication paradigm based on independent cooperative state machines. Such a paradigm would maybe need some enhancements to the asynchronous communication protocol to be truly efficient. For instance I believe it might be beneficial if parameters could be added to signals in a similar fashion to that of RPCs. I have seen the motivation for not adding parameters to signals, but I fail to see why the various components shouldn't be able to donate dataspace for such storage like they may donate dataspace for RPC call parameter storage.
I checked the loopback example you provided regarding a server component written according to the state machine pattern. I find it somewhat rudimentary and full of if then statements. Now the task at hand is simple so a simple pattern makes sense. However for more complex components the illustrated approach doesn't scale. I believe that Genode would gain in efficiency and clarity by integrating a solid state machine framework supporting UML statecharts to some degree including hierarchical state machines, It turns out that it not is very complicated and quite efficient to integrate such a framework providing decent UML statecharts support. I wont go into details how it is done, you can check [1] for an elaborate discussion of an existing implementation of the state machine idiom. Basically a state machine framework consists of three parts:
1) Abstractions for describing state machines 2) Event buffers for storing queued events 3) Runner executing state machines and forwarding events to the individual state machine event queues
Neither of the parts is particularly difficult to design. Several descriptions exists for 1) [1, 2, 3, 7]. Points 2) and 3) are well described in [1] and should not be too difficult to implement. According to my own analysis the complete framework QP from [1] probably is quite easy to port to Genode. For third party developers like me it makes sense as both Genode and QP are provided under the same GPL license but possibly not for Genode Labs as they would have to license QP. QP provides for most stuff required to execute UML statecharts. In addition one gets a) QSPY, a very elaborate debugging platform for inspecting system behavior (would definitely fit well with Guido Witmonds debugging request) and b) a graphical UML state machine designer which appeals to me as it makes third party certification much easier due to the higher abstraction level in a statechart compared to that of raw c++ code.
My recommendation now is that Norman and the Genodians take a deeper look at the state of the art regarding state machine design and consider if there is some merit in making use of some existing frameworks or standards, most notably the UML statecharts standard. I can of course implement the required functionality myself but I believe that there is real benefit in having full integration into Genode to ease integration with built in components. Having experts taking a look at the various trade offs regarding state machine framework integration into Genode within the areas of signaling, execution, implementation patterns and so on also is of real benefit.
Some changes have been initiated to make Genode more secure and less error prone. I think of the abolition of unnecessary pointers and the planned removal of global state. These are standard recommendations in a number of prominent style guides [4, 5]. I would like to see a more systematic approach to the improvement of Genode. As Bruce Schneier once stated; "Security is a process, not a product" [6]. With this I mean that objectives first should be decided. Is security and safety important? Is it important to gain the trust of people with little knowledge of Genode internals? If yes to both questions Misra C++ compliance makes sense as it is the standard in the area. A subset of Misra C++ should then be selected for implementation over time. Rules not implemented should be motivated very well as to why they not pose a security threat. Then the rules should be ordered in a list and gradually implemented and thereafter checked for adherence by tools like Flexelint. This is the process that some day might lead to Misra C++ compliance. The same reasoning apply to the upcoming C++ core guidelines or anything else worth adhering to.
Regarding the upcoming update of the book Genode Foundations. What I miss in the book is a connection the real code and examples. I believe that new readers much easier understand and remember how to use Genode if the connection to code and the inner workings are made clearer. Here I would take the opportunity to again recommend [1] as a book that makes this connection very clear. OK, it is easier to describe state machines but I still believe that Norman could find some inspiration in that book for the upcoming release of Genode Foundations.
See you at FOSDEM,
Pontus
[1] Miro Samek, Practical UML Statecharts in C/C++, second ed., 2008 (google the title and you´ll find a pdf on the first search page) [2] http://www.drdobbs.com/hierarchical-state-machine-design-in-c/184402040 [3] http://accu.org/index.php/journals/252 [4] https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md [5] http://www.misra-cpp.com [6] https://www.schneier.com/essays/archives/2000/04/the_process_of_secur.html [7] http://www.boost.org/doc/libs/1_60_0/libs/statechart/doc/index.html
On Tue, Dec 22, 2015 at 1:28 PM, Norman Feske <norman.feske@...1...> wrote:
Hello,
the end of the year 2015 is approaching. So it is time for planning our activities for the upcoming year. I plan to finalize the roadmap until mid of January. Hereby, I'd like to kick off the discussion. It goes without saying that everyone of you is invited to join in!
Before I come to my suggestions for 2016, let me briefly revisit the outcome of the roadmap for the past year. One year ago, I suggested three main topics to work on, namely the use of Genode as general-purpose OS, the advancement of our custom base-hw kernel platform, and the use of the seL4 kernel. In the discussions following my posting, many further points were raised, most prominently the need for through documentation, package management, and a sustainable quality of the base system in terms of stability and performance. In top of that, we received quite a few wishes of higher-level functionality such as a modern web browser or other typical desktop-computing features. Of course, we were not able to address all those topics but I am overly happy that we reached the point where a hand full of us (including me) are using Genode as their day-to-day OS. When I initially switched to using the Turmvilla scenario [1] at the end of May, the experience was admittedly a bit painful. But now, just a few months later, the beauty of the system becomes apparent as all the pieces come so nicely together. The performance, stability, and device-driver support have reached a level that leaves people impressed every time I have the chance to show off my system. Once people become interested, there is now the book available, which provides a smooth introduction into Genode. The feedback I receive about the book is overwhelmingly positive. So we did something right in 2015. :-)
After having passed the point where a few of us are able to use Genode as day-to-day OS, we should put the emphasis of the upcoming year on ways to make Genode accessible for a wider community. In a recent posting [2], I identified two possible ways to do that. The first way would be publishing a series of step-by-step guides that explain how to put Genode components together in order to create custom system scenarios. Those articles could be accompanied by screencasts or live-system images. Example scenarios could be the Turmvilla scenario, building a home-computer-like system for kids using the Raspberry Pi (like the Genode system my kids are using), building a network appliance like a data diode, tinkering with the USB Armory, etc. Those articles should be inviting to people who enjoy the building of systems. The second way would be to showcase a system with practical value to end users. I am thinking along the lines of a disposable OS like Tails that allows the user to browse the internet via the Tor network. But that is just an idea.
In this spirit, I propose the overall focus of 2016 to be: Let us make Genode accessible to the world outside the inner circle of us enthusiasts.
On a technical level, this motive implicates the following topics:
The deployment and management of Genode systems, i.e., by bringing forward Emery's work on the Nix package manager. This direction also reinforces the need to achieve binary compatibility between the various base platforms to make the distribution of binary packages, reproduceable builds, and continues test-and-integration scalable.
Supplementing Genode with user-friendly means to configure the system (e.g., wireless network, audio, display settings). Right now, I use Vim as configuration front end, which is cool, but also intimidating to less technical-minded users.
Accommodation of common desktop use cases like plugging in a USB stick to work with the files stored on it. Also disk encryption comes into mind.
Optimization of Genode for the use on a laptop, e.g., addressing fan control, power management, suspend/resume, and similar features.
There are also other possible avenues to support the stated goal:
Identifying ways of how Genode could play a role in related projects like Qubes OS. For example, we could promote the use of Genode as a tool for building App VMs as Qubes subsystems. Granted, this scenario leaves the architectural benefits of Genode with respect to its small TCB complexity unused as Qubes still relies on Xen, and Linux as Dom0. But Genode would still (possibly) provide value to the Qubes project. Maybe, there would be the prospect to replace Dom0 with Genode in the longer term? However, to drive this direction of work, we would certainly need someone who is actually using Qubes and has the natural incentive to work on such an integration.
Making Genode-based systems easily deployable on Amazon's EC2. Similar to the previous point, it would be beneficial to have someone working on this topic who is naturally interested in cloud computing.
Foster the cross-pollination of the seL4 and Genode communities. I got enthusiastic responses about my seL4-related work. There is definitely a strong interest in this kernel and a growing anticipation for formally verified software. Today, seL4 lacks a scalable user-level architecture. This would be the perfect place where Genode could step in. Genode would ultimately allow the seL4 community to move beyond static system scenarios.
Assuming that we succeed in drawing the attention of a broader audience to our project, we should make sure that Genode's API won't undergo major changes soon after this point. Today, I still see a number of deficiencies in the current API. In the past year, we successively moved to a new model of the API (dubbed server API) that promotes the construction of components as state machines rather than procedural programs. All recent components are designed in this style to the great benefit of their robustness. We should finally promote this style towards the base API and get rid of some mistakes we made in the past, in particular the reliance on side effects by using the globally visible Genode::env. I think that we should finalize this API renovation until the mid of 2016. This will also be right the time for updating the Genode book.
These are my thoughts about the upcoming year. Now I am curious about yours! :-)
Cheers Norman
[1] https://github.com/genodelabs/genode/issues/1552 [2] https://github.com/genodelabs/genode/issues/1000#issuecomment-161260312
-- Dr.-Ing. Norman Feske Genode Labs
http://www.genode-labs.com · http://genode.org
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
I am (as always) excited to see Genode progress in 2016. There have been made good points made already on this thread - hopefully I can add some more.
## Growth of the Genode community
One of the key things I would like to see this year is the growth of the community behind Genode. Genode is a great technical foundation upon which to build things, but not enough people know about it and what it can do. As well as the book, we need tutorials, screencasts and talks about it, not just necessarily targeted at the communities that are already interested in security + microkernels.
This touches a little bit I guess on Genode Labs providing a framework and not a solution, but I think by making Genode more accessible lots of interesting projects will arise.
Obviously the above is not an invitation to schedule more things into the Genode Labs diary about writing more tutorials, but rather to start a discussion about what people think needs to be done in this area and perhaps some volunteers can step up (I am certainly happy to write about my experiences with the USB Armory for example when I get the USB driver to finally work).
## Modelling with state machines
I like the direction being taken in https://github.com/genodelabs/genode/issues/1832 wrt. making the APIs simpler and more state-machine based.
Some DSL for modelling larger state machines is definitely going to prove useful, though whether one should be adopted as part of Genode is debatable (will servers really be that huge?). I have seen state machines with 10s of states and many hundreds of transitions and it does become difficult to reason about them purely because you cannot keep them all in your head. I personally would prefer a custom textual DSL which is easy to see and edit in vim, but there are obviously many approaches out there.
http://mbeddr.com looks interesting as another idea of how to approach this problem (have not used it)
## A shell
It would be really neat to have a small shell where you can change the system on the fly. I guess it would be similar to the graphical launcher in some respects, though I imagine some configuration could also be changed directly instead of having to fire up a text editor. Any scripting language would do, perhaps Python or Lua (so it would need a small binding and then creation of an API). Then it would be a good show-case of, say, configuring an IP address (and a new bit of hardware) in Genode vs. Linux. (Maybe, that's just the first idea that came to mind).
Maybe even generally it would be nice to have at least a scripting language that can be used to write servers in (for faster prototyping for example).
## Desktop + software porting
I think that having Genode on the Desktop is a good goal to have, and even having basic networking + a browser can get you very far (just look at ChromeOS). Personally I am not interested in x86 desktop cases (for all the reasons and more that were summarised in Joanna Rutkowska's talk recently: https://media.ccc.de/v/32c3-7352-towards_reasonably_trustworthy_x86_laptops). I don't have a Novena but other i.MX6 hardware - I haven't looked at running Genode + a GUI on it yet though. So maybe that's a good h/w target outside of the x86 world.
In terms of software I would like to see ported:
* tor * rust language * pf (*sounds difficult) * zeromq * node
(I would like to take on Rust, as projects like https://github.com/redox-os/redox or the newly announced https://robigalia.org are surely going to get some following this year).
## Other points
* I agree with Pontus' point about providing real code examples in the book - playing with code is usually how I learn best and a full-fledged example is not always the best place to start, not least because Genode has a lot of unique terminology * Disk encryption would be very useful in actually having a production-like desktop example! (Is the rump based cgd still working fine?)
Hopefully see you all at FOSDEM
Kind regards, Vladimir
-- ***Email confidentiality notice*** This message is private and confidential. Storage and automated scanning of this message for selectors and metadata is strictly prohibited.
Solid Ninja Ltd. is a limited company registered in England and Wales. Registration number 9622534. Registered office Unit 11, Hove Business Centre, East Sussex, BN3 6HA
6. Jan 2016 23:33 by ponast@...9...:
Hi,
This is my first post on the list and I will try to substantiate some of my ideas/wishes regarding the future of Genode as requested by Norman.
First, I must say that I still have some trouble understanding the use case for Genode as a desktop OS. Not that I don't like the idea but I think this is such an enormous undertaking that a believe might be like aiming too far or maybe like missing lower hanging fruit. It might be better to shorten the hops to the future in easier and faster achievable milestones. Genode then gets truly usable faster, albeit within a smaller community, but one that truly cares. It also gets tried in production sooner, which is very important to gain stability and real user feedback.
One such step might be to make Genode a viable and usable platform for certain domains within embedded computing. This is a much easier undertaking as embedded systems not have the same requirements regarding ease of use and completeness regarding features. The GUI subsystem goes away and hardware support need not be as broad as for desktop OSes. I also think that Genode has several additional properties that are highly sought after in various embedded domains like responsiveness, stability and power efficiency. These characteristics should be exploited together with the main benefit of Genode as a secure and safe OS to gain maximum impact.
I am glad to see that you have found the state machine being a usable pattern. I happen to believe that it would be very beneficial for Genode to mostly switch to a fully asynchronous communication paradigm based on independent cooperative state machines. Such a paradigm would maybe need some enhancements to the asynchronous communication protocol to be truly efficient. For instance I believe it might be beneficial if parameters could be added to signals in a similar fashion to that of RPCs. I have seen the motivation for not adding parameters to signals, but I fail to see why the various components shouldn't be able to donate dataspace for such storage like they may donate dataspace for RPC call parameter storage.
I checked the loopback example you provided regarding a server component written according to the state machine pattern. I find it somewhat rudimentary and full of if then statements. Now the task at hand is simple so a simple pattern makes sense. However for more complex components the illustrated approach doesn't scale. I believe that Genode would gain in efficiency and clarity by integrating a solid state machine framework supporting UML statecharts to some degree including hierarchical state machines, It turns out that it not is very complicated and quite efficient to integrate such a framework providing decent UML statecharts support. I wont go into details how it is done, you can check [1] for an elaborate discussion of an existing implementation of the state machine idiom. Basically a state machine framework consists of three parts:
- Abstractions for describing state machines
- Event buffers for storing queued events
- Runner executing state machines and forwarding events to the individual state machine event queues
Neither of the parts is particularly difficult to design. Several descriptions exists for 1) [1, 2, 3, 7]. Points 2) and 3) are well described in [1] and should not be too difficult to implement. According to my own analysis the complete framework QP from [1] probably is quite easy to port to Genode. For third party developers like me it makes sense as both Genode and QP are provided under the same GPL license but possibly not for Genode Labs as they would have to license QP. QP provides for most stuff required to execute UML statecharts. In addition one gets a) QSPY, a very elaborate debugging platform for inspecting system behavior (would definitely fit well with Guido Witmonds debugging request) and b) a graphical UML state machine designer which appeals to me as it makes third party certification much easier due to the higher abstraction level in a statechart compared to that of raw c++ code.
My recommendation now is that Norman and the Genodians take a deeper look at the state of the art regarding state machine design and consider if there is some merit in making use of some existing frameworks or standards, most notably the UML statecharts standard. I can of course implement the required functionality myself but I believe that there is real benefit in having full integration into Genode to ease integration with built in components. Having experts taking a look at the various trade offs regarding state machine framework integration into Genode within the areas of signaling, execution, implementation patterns and so on also is of real benefit.
Some changes have been initiated to make Genode more secure and less error prone. I think of the abolition of unnecessary pointers and the planned removal of global state. These are standard recommendations in a number of prominent style guides [4, 5]. I would like to see a more systematic approach to the improvement of Genode. As Bruce Schneier once stated; "Security is a process, not a product" [6]. With this I mean that objectives first should be decided. Is security and safety important? Is it important to gain the trust of people with little knowledge of Genode internals? If yes to both questions Misra C++ compliance makes sense as it is the standard in the area. A subset of Misra C++ should then be selected for implementation over time. Rules not implemented should be motivated very well as to why they not pose a security threat. Then the rules should be ordered in a list and gradually implemented and thereafter checked for adherence by tools like Flexelint. This is the process that some day might lead to Misra C++ compliance. The same reasoning apply to the upcoming C++ core guidelines or anything else worth adhering to.
Regarding the upcoming update of the book Genode Foundations. What I miss in the book is a connection the real code and examples. I believe that new readers much easier understand and remember how to use Genode if the connection to code and the inner workings are made clearer. Here I would take the opportunity to again recommend [1] as a book that makes this connection very clear. OK, it is easier to describe state machines but I still believe that Norman could find some inspiration in that book for the upcoming release of Genode Foundations.
See you at FOSDEM,
Pontus
[1] Miro Samek, Practical UML Statecharts in C/C++, second ed., 2008 (google the title and you´ll find a pdf on the first search page) [2] > http://www.drdobbs.com/hierarchical-state-machine-design-in-c/184402040 [3] > http://accu.org/index.php/journals/252 [4] > https://github.com/isocpp/CppCoreGuidelines/blob/master/CppCoreGuidelines.md [5] > http://www.misra-cpp.com [6] > https://www.schneier.com/essays/archives/2000/04/the_process_of_secur.html [7] > http://www.boost.org/doc/libs/1_60_0/libs/statechart/doc/index.html
On Tue, Dec 22, 2015 at 1:28 PM, Norman Feske <> norman.feske@...1...> > wrote:
Hello,
the end of the year 2015 is approaching. So it is time for planning our activities for the upcoming year. I plan to finalize the roadmap until mid of January. Hereby, I'd like to kick off the discussion. It goes without saying that everyone of you is invited to join in!
Before I come to my suggestions for 2016, let me briefly revisit the outcome of the roadmap for the past year. One year ago, I suggested three main topics to work on, namely the use of Genode as general-purpose OS, the advancement of our custom base-hw kernel platform, and the use of the seL4 kernel. In the discussions following my posting, many further points were raised, most prominently the need for through documentation, package management, and a sustainable quality of the base system in terms of stability and performance. In top of that, we received quite a few wishes of higher-level functionality such as a modern web browser or other typical desktop-computing features. Of course, we were not able to address all those topics but I am overly happy that we reached the point where a hand full of us (including me) are using Genode as their day-to-day OS. When I initially switched to using the Turmvilla scenario [1] at the end of May, the experience was admittedly a bit painful. But now, just a few months later, the beauty of the system becomes apparent as all the pieces come so nicely together. The performance, stability, and device-driver support have reached a level that leaves people impressed every time I have the chance to show off my system. Once people become interested, there is now the book available, which provides a smooth introduction into Genode. The feedback I receive about the book is overwhelmingly positive. So we did something right in 2015. :-)
After having passed the point where a few of us are able to use Genode as day-to-day OS, we should put the emphasis of the upcoming year on ways to make Genode accessible for a wider community. In a recent posting [2], I identified two possible ways to do that. The first way would be publishing a series of step-by-step guides that explain how to put Genode components together in order to create custom system scenarios. Those articles could be accompanied by screencasts or live-system images. Example scenarios could be the Turmvilla scenario, building a home-computer-like system for kids using the Raspberry Pi (like the Genode system my kids are using), building a network appliance like a data diode, tinkering with the USB Armory, etc. Those articles should be inviting to people who enjoy the building of systems. The second way would be to showcase a system with practical value to end users. I am thinking along the lines of a disposable OS like Tails that allows the user to browse the internet via the Tor network. But that is just an idea.
In this spirit, I propose the overall focus of 2016 to be: Let us make Genode accessible to the world outside the inner circle of us enthusiasts.
On a technical level, this motive implicates the following topics:
The deployment and management of Genode systems, i.e., by bringing forward Emery's work on the Nix package manager. This direction also reinforces the need to achieve binary compatibility between the various base platforms to make the distribution of binary packages, reproduceable builds, and continues test-and-integration scalable.
Supplementing Genode with user-friendly means to configure the system (e.g., wireless network, audio, display settings). Right now, I use Vim as configuration front end, which is cool, but also intimidating to less technical-minded users.
Accommodation of common desktop use cases like plugging in a USB stick to work with the files stored on it. Also disk encryption comes into mind.
Optimization of Genode for the use on a laptop, e.g., addressing fan control, power management, suspend/resume, and similar features.
There are also other possible avenues to support the stated goal:
Identifying ways of how Genode could play a role in related projects like Qubes OS. For example, we could promote the use of Genode as a tool for building App VMs as Qubes subsystems. Granted, this scenario leaves the architectural benefits of Genode with respect to its small TCB complexity unused as Qubes still relies on Xen, and Linux as Dom0. But Genode would still (possibly) provide value to the Qubes project. Maybe, there would be the prospect to replace Dom0 with Genode in the longer term? However, to drive this direction of work, we would certainly need someone who is actually using Qubes and has the natural incentive to work on such an integration.
Making Genode-based systems easily deployable on Amazon's EC2. Similar to the previous point, it would be beneficial to have someone working on this topic who is naturally interested in cloud computing.
Foster the cross-pollination of the seL4 and Genode communities. I got enthusiastic responses about my seL4-related work. There is definitely a strong interest in this kernel and a growing anticipation for formally verified software. Today, seL4 lacks a scalable user-level architecture. This would be the perfect place where Genode could step in. Genode would ultimately allow the seL4 community to move beyond static system scenarios.
Assuming that we succeed in drawing the attention of a broader audience to our project, we should make sure that Genode's API won't undergo major changes soon after this point. Today, I still see a number of deficiencies in the current API. In the past year, we successively moved to a new model of the API (dubbed server API) that promotes the construction of components as state machines rather than procedural programs. All recent components are designed in this style to the great benefit of their robustness. We should finally promote this style towards the base API and get rid of some mistakes we made in the past, in particular the reliance on side effects by using the globally visible Genode::env. I think that we should finalize this API renovation until the mid of 2016. This will also be right the time for updating the Genode book.
These are my thoughts about the upcoming year. Now I am curious about yours! :-)
Cheers Norman
[1] >> https://github.com/genodelabs/genode/issues/1552 [2] >> https://github.com/genodelabs/genode/issues/1000#issuecomment-161260312
-- Dr.-Ing. Norman Feske Genode Labs
http://www.genode-labs.com%3E%3E · >> http://genode.org
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
-- Hälsningar,
Pontus
genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
On Thu, Jan 07, 2016 at 10:47:52PM +0000, Vladimir Lushnikov wrote:
I don't have a Novena but other i.MX6 hardware - I haven't looked at running Genode + a GUI on it yet though. So maybe that's a good h/w target outside of the x86 world.
I have a Novena as my main machine now, so if you need testing for it I'be happy to do so and provide feedback.
If we're talking about cool things microkernels can do, I'd say the fact you can get low level hardware access with minimum overhead for drivers in user space, and through paravirtualization have a system almost as fast as bare metal.
It'd be interesting to see high-level languages used to write drivers (this is mentioned a bit in the 2015 roadmap with Haskell) rather than something like C.
Thanks for the offer - I could get a Novena - but time (and experience with tricky hardware problems) is more difficult to acquire ;)
Just looking at the sources now, I see that the only framebuffer driver written for iMX has been for the iMX53 - doing a quick search leads me to think that the IPU on the iMX6 is a very similar interface (http://lxr.free-electrons.com/source/drivers/gpu/ipu-v3/ipu-common.c)
Have you tried playing with the framebuffer driver? I imagine quite a few other things don't work (USB for sure, but probably ethernet as well?)
Vladimir -- ***Email confidentiality notice*** This message is private and confidential. Storage and automated scanning of this message for selectors and metadata is strictly prohibited.
Solid Ninja Ltd. is a limited company registered in England and Wales. Registration number 9622534. Registered office Unit 11, Hove Business Centre, East Sussex, BN3 6HA
7. Jan 2016 23:29 by 166291@...9...:
On Thu, Jan 07, 2016 at 10:47:52PM +0000, Vladimir Lushnikov wrote:
I don't have a Novena but other i.MX6 hardware - I haven't looked at running Genode + a GUI on it yet though. So maybe that's a good h/w target outside of the x86 world.
I have a Novena as my main machine now, so if you need testing for it I'be happy to do so and provide feedback.
If we're talking about cool things microkernels can do, I'd say the fact you can get low level hardware access with minimum overhead for drivers in user space, and through paravirtualization have a system almost as fast as bare metal.
It'd be interesting to see high-level languages used to write drivers (this is mentioned a bit in the 2015 roadmap with Haskell) rather than something like C.
genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
On Thu, Jan 07, 2016 at 11:44:45PM +0000, Vladimir Lushnikov wrote:
Just looking at the sources now, I see that the only framebuffer driver written for iMX has been for the iMX53 - doing a quick search leads me to think that the IPU on the iMX6 is a very similar interface (http://lxr.free-electrons.com/source/drivers/gpu/ipu-v3/ipu-common.c)
Interesting- I wouldn't be surprised if there's not much difference, when looking at the security sheet I saw it has the same problem of the GPU acceleration being tied to the rest of the graphics when it comes to TrustZone.
Have you tried playing with the framebuffer driver? I imagine quite a few other things don't work (USB for sure, but probably ethernet as well?)
I haven't tested Genode on it, I'm planning on toying with it on a T400 thinkpad with Libreboot installed before devoting time to porting things, or trying to as I have no hardware experience. I've been spending my time getting to know the existing toolchain and setting up NixOS on The Novena - I've chosen NixOS given the work on using the Nix package manager in Genode.
Vladimir
Jookia. :)
Hi Vladimir,
thanks for your participation in the discussion!
Obviously the above is not an invitation to schedule more things into the Genode Labs diary about writing more tutorials, but rather to start a discussion about what people think needs to be done in this area and perhaps some volunteers can step up (I am certainly happy to write about my experiences with the USB Armory for example when I get the USB driver to finally work).
I very much appreciate your stance. We at Genode Labs will try to do our share to provide such tutorial-like material. But to reach a wider audience, we ultimately need a community effort by multiple independent people publishing opinions, tutorials, and testimonials. For example, right now, Genode is almost unknown to the HN crowd. If I posted submissions about Genode (like release announcements), this would count as shameless self-promotion. It would be different if such information were spread by people outside the group of core developers.
Some DSL for modelling larger state machines is definitely going to prove useful, though whether one should be adopted as part of Genode is debatable (will servers really be that huge?). I have seen state machines with 10s of states and many hundreds of transitions and it does become difficult to reason about them purely because you cannot keep them all in your head. I personally would prefer a custom textual DSL which is easy to see and edit in vim, but there are obviously many approaches out there.
http://mbeddr.com http://mbeddr.com/ looks interesting as another idea of how to approach this problem (have not used it)
I 100% share your sentiment. Thanks for the pointer to mbeddr.
## A shell
It would be really neat to have a small shell where you can change the system on the fly. I guess it would be similar to the graphical launcher in some respects, though I imagine some configuration could also be changed directly instead of having to fire up a text editor.
May the existing CLI monitor (os/src/app/cli_monitor) serve as a starting point? This is what I am currently using in the Turmvilla scenario.
Any scripting language would do, perhaps Python or Lua (so it would need a small binding and then creation of an API). Then it would be a good show-case of, say, configuring an IP address (and a new bit of hardware) in Genode vs. Linux. (Maybe, that's just the first idea that came to mind).
Since I switched to Turmvilla, my perspective on those kind of management problems has changed a bit. I routinely carry out most configuration tasks by merely editing (or copying) files, which is super simple compared to the zoo of configuration interfaces (and the corresponding tools) known from Linux. I don't think that it can get much more convenient and flexible than that. If we meet at FOSDEM, I'll show you what I mean. ;-)
In principle, thanks to Noux, it is possible to use any existing text-processing solution for automating configuration tasks as long as it can produce XML. In my opinion, a standardization or a new API is not needed at that end.
We actually ported Lua and Python to Genode some years ago and intended to use either of both for scripting / automated testing purposes. But so far, these ports remained largely unused.
In terms of software I would like to see ported:
- tor
- rust language
- pf (*sounds difficult)
- zeromq
- node
(I would like to take on Rust, as projects like https://github.com/redox-os/redox or the newly announced https://robigalia.org https://robigalia.org/ are surely going to get some following this year).
Great that you plan to look into Rust on Genode! Would you like to see this point mentioned in the road map?
- Disk encryption would be very useful in actually having a
production-like desktop example! (Is the rump based cgd still working fine?)
The rump-based CGD is still supposed to work but we don't use it in practice. Personally, I think that the rump-based solution is overkill for the simple task of encrypting blocks. Hence, I'd like to include a low-complexity block-encryption component in the road map.
Cheers Norman
Hi Pontus,
welcome to the list and thank you for the thought-provoking comments!
First, I must say that I still have some trouble understanding the use case for Genode as a desktop OS. Not that I don't like the idea but I think this is such an enormous undertaking that a believe might be like aiming too far or maybe like missing lower hanging fruit.
I will try to clarify the goal and motivation.
The goal is to enable the productive use of Genode by a narrow group of users with reasonably with well-defined requirements. I gave two examples of such groups, namely us developers and the users of a disposable TOR-browser appliance. The immediate goal is *not* to create a desktop OS for the masses. This would be unrealistic. But with the goal narrowed like this, the aspiration is feasible.
The motivation behind this goal is manifold:
* The use of Genode on the desktop stresses the point that Genode scales well.
* The dynamic workloads of desktop computing present much stronger requirements with respect to the quality of Genode (in particular regarding stability, flexibility, and performance), compared to the domain of low-complexity static embedded systems. We want to satisfy those requirements, not avoid them. If we successfully overcome the challenges of dynamic desktop workloads, I am positive that we implicitly cover the weaker requirements of the embedded space.
* By having us developers use Genode every day, we create the natural motivation to work on important problems that are otherwise easily dismissed as boring or as "minor details". Since I switched to Genode as my day-to-day OS, I discovered and fixed many of such issues that remained unnoticed in the years before.
* By constantly stressing the system by a growing number of people, we get a certain kind of quality assurance for free. I am particularly thinking of device drivers (like graphics) for which no automated tests exist. In the past, regressions in those parts of the system sometimes remained unnoticed for a shockingly long time. By constantly using those parts, they stay healthy.
* I am able to spontaneously demonstrate the Genode concepts live on the system that I use. This carries a much stronger message than bunch of slides.
* It makes my geek heart very happy.
It might be better to shorten the hops to the future in easier and faster achievable milestones. Genode then gets truly usable faster, albeit within a smaller community, but one that truly cares. It also gets tried in production sooner, which is very important to gain stability and real user feedback.
As a matter of fact, one particular "community that truly cares" are the ones who currently invest in the development on Genode by commissioning paid projects to Genode Labs. The majority of such project work is actually geared towards the desktop use case.
I can follow your argument. From the business perspective of a solution provider, it makes perfect sense to pick low-hanging fruits first. But that is not Genode's role. Genode is not picking fruits. Genode's role is the one of a ladder that allows you to pick high-hanging fruits also.
One such step might be to make Genode a viable and usable platform for certain domains within embedded computing. This is a much easier undertaking as embedded systems not have the same requirements regarding ease of use and completeness regarding features. The GUI subsystem goes away and hardware support need not be as broad as for desktop OSes. I also think that Genode has several additional properties that are highly sought after in various embedded domains like responsiveness, stability and power efficiency. These characteristics should be exploited together with the main benefit of Genode as a secure and safe OS to gain maximum impact.
The interesting question is: What is missing?
I am glad to see that you have found the state machine being a usable pattern. I happen to believe that it would be very beneficial for Genode to mostly switch to a fully asynchronous communication paradigm based on independent cooperative state machines. Such a paradigm would maybe need some enhancements to the asynchronous communication protocol to be truly efficient. For instance I believe it might be beneficial if parameters could be added to signals in a similar fashion to that of RPCs. I have seen the motivation for not adding parameters to signals, but I fail to see why the various components shouldn't be able to donate dataspace for such storage like they may donate dataspace for RPC call parameter storage.
Regardless of how buffers are dimensioned and who pays for them, they can overrun. The crux is how to deal with this situation?
If the signal sender is responsible to respond to such an overrun, signals would lose their fire-and-forget semantics because the sender may need to deal with the unexpected consequences of submitting a signal. How would that be any better than the use of synchronous RPC in the first place? If the receiver is responsible, it becomes prone to denial-of-service problems. In either case, we would create an inter-dependency between both parties that is not desired.
The feature you describe (signals with arguments) is actually accommodated by the packet stream, which uses a combination of signals with a shared dataspace. But it escalates the buffer-overrun problem to the user of the API.
I checked the loopback example you provided regarding a server component written according to the state machine pattern. I find it somewhat rudimentary and full of if then statements. Now the task at hand is simple so a simple pattern makes sense. ... I believe that Genode would gain in efficiency and clarity by integrating a solid state machine framework supporting UML statecharts to some degree including hierarchical state machines, ...
I followed the pointers you provided but remain unconvinced about the value that UML state charts bring to the table. I guess, this is caused by my lack of experience with such tools. It goes without saying that I am all for reducing complexity and increasing clarity. Right now, I simply fail to grasp how the FSM frameworks you cited would improve the situation. It looks like I will need to study the book you referred to.
Alternatively, have you considered to prove your assertion by modeling the NIC-loopback example with the tools you suggest? Maybe this example would help me to understand the benefit.
My recommendation now is that Norman and the Genodians take a deeper look at the state of the art regarding state machine design and consider if there is some merit in making use of some existing frameworks or standards, most notably the UML statecharts standard.I can of course implement the required functionality myself but I believe that there is real benefit in having full integration into Genode to ease integration with built in components.
Thank you for the recommendation. To come to a conclusion how this information may benefit Genode, someone would first need to investigate, understand, and experiment. Admittedly, I am unable to connect the dots right now. But I don't want to sit on the critical path here. You seem to see a clearer picture. It certainly would be the best for you to go forward and share your findings.
Having experts taking a look at the various trade offs regarding state machine framework integration into Genode within the areas of signaling, execution, implementation patterns and so on also is of real benefit.
I'd be happy to review such work from Genode's perspective.
Some changes have been initiated to make Genode more secure and less error prone. I think of the abolition of unnecessary pointers and the planned removal of global state. These are standard recommendations in a number of prominent style guides [4, 5].
I skimmed over the Misra document [5], which is indeed very useful. It turns out that we happen to intuitively follow most of the rules. In the few cases where we don't comply, we have a rationale. Thanks for the good pointer!
I would like to see a more systematic approach to the improvement of Genode. As Bruce Schneier once stated; "Security is a process, not a product" [6]. With this I mean that objectives first should be decided. Is security and safety important? Is it important to gain the trust of people with little knowledge of Genode internals? If yes to both questions Misra C++ compliance makes sense as it is the standard in the area. A subset of Misra C++ should then be selected for implementation over time. Rules not implemented should be motivated very well as to why they not pose a security threat. Then the rules should be ordered in a list and gradually implemented and thereafter checked for adherence by tools like Flexelint. This is the process that some day might lead to Misra C++ compliance. The same reasoning apply to the upcoming C++ core guidelines or anything else worth adhering to.
I sense that this discussion is rooted in your desire to give Genode a stamp (aka certification) that expresses a certain code quality standard - a stamp that non-technical people can understand.
Technically, while acknowledging that our code can always be improved, I don't think that code quality holds Genode back. In contrary, the feedback we get about the code is often quite enthusiastic. The real questions seem to be: What are the steps needed to obtain the desired stamp, and who invests the effort to take those steps?
I am not at all opposed to adopting the Misra C++ rules as I agree with them (a few exceptions notwithstanding). But personally, I miss a strong-enough incentive to pursue this direction on our own account.
Regarding the upcoming update of the book Genode Foundations. What I miss in the book is a connection the real code and examples. I believe that new readers much easier understand and remember how to use Genode if the connection to code and the inner workings are made clearer. Here I would take the opportunity to again recommend [1] as a book that makes this connection very clear. OK, it is easier to describe state machines but I still believe that Norman could find some inspiration in that book for the upcoming release of Genode Foundations.
I don't plan to change the existing book structurally. The update will merely adjust the content to the changes of the code (API, source-code layout, internals).
Do you think that your needs regarding code examples could be satisfied with a series of tutorial-like articles like I suggested in my original email?
See you at FOSDEM,
Very cool! :-)
I'd like to come back to the original topic - the road map. Do you have concrete plans that you are willing to share with us?
Cheers Norman
I have added some comments inline on Normans reply. Maybe we can continue the discussion at FOSDEM? After all, the 2015 roadmap broke down after summer so it does not seem to be set in stone even if published next week just ahead of FOSDEM :)
See you,
Pontus
On Fri, Jan 8, 2016 at 6:11 PM, Norman Feske <norman.feske@...1...> wrote:
Hi Pontus,
welcome to the list and thank you for the thought-provoking comments!
First, I must say that I still have some trouble understanding the use case for Genode as a desktop OS. Not that I don't like the idea but I think this is such an enormous undertaking that a believe might be like aiming too far or maybe like missing lower hanging fruit.
I will try to clarify the goal and motivation.
The goal is to enable the productive use of Genode by a narrow group of users with reasonably with well-defined requirements. I gave two examples of such groups, namely us developers and the users of a disposable TOR-browser appliance. The immediate goal is *not* to create a desktop OS for the masses. This would be unrealistic. But with the goal narrowed like this, the aspiration is feasible.
The motivation behind this goal is manifold:
The use of Genode on the desktop stresses the point that Genode scales well.
The dynamic workloads of desktop computing present much stronger requirements with respect to the quality of Genode (in particular regarding stability, flexibility, and performance), compared to the domain of low-complexity static embedded systems. We want to satisfy those requirements, not avoid them. If we successfully overcome the challenges of dynamic desktop workloads, I am positive that we implicitly cover the weaker requirements of the embedded space.
By having us developers use Genode every day, we create the natural motivation to work on important problems that are otherwise easily dismissed as boring or as "minor details". Since I switched to Genode as my day-to-day OS, I discovered and fixed many of such issues that remained unnoticed in the years before.
By constantly stressing the system by a growing number of people, we get a certain kind of quality assurance for free. I am particularly thinking of device drivers (like graphics) for which no automated tests exist. In the past, regressions in those parts of the system sometimes remained unnoticed for a shockingly long time. By constantly using those parts, they stay healthy.
I am able to spontaneously demonstrate the Genode concepts live on the system that I use. This carries a much stronger message than bunch of slides.
It makes my geek heart very happy.
This answer makes perfect sense. In particular when I hear that the ones that pay for your services have this goal. Keep going.
It might be better to shorten the hops to the future in easier and faster achievable milestones. Genode then gets truly usable faster, albeit within a smaller community, but one that truly cares. It also gets tried in production sooner, which is very important to gain stability and real user feedback.
As a matter of fact, one particular "community that truly cares" are the ones who currently invest in the development on Genode by commissioning paid projects to Genode Labs. The majority of such project work is actually geared towards the desktop use case.
I can follow your argument. From the business perspective of a solution provider, it makes perfect sense to pick low-hanging fruits first. But that is not Genode's role. Genode is not picking fruits. Genode's role is the one of a ladder that allows you to pick high-hanging fruits also.
I like the metaphor "Genode is the ladder that allows the rest of us to pick high hanging fruit". But still I think it makes sense with some economics thinking like calculating the current value of investments. If you for instance can enable some new application domain with low effort it makes sense to do so. Here I think Vladimirs proposal regarding disk encryption makes sense. How much would it take for Genode to enable a stateless design according to Joannas ideas in http://blog.invisiblethings.org/papers/2015/x86_harmful.pdf ? Maybee not so much and it would open an new domain with lots of possibilities. Just an example.
One such step might be to make Genode a viable and usable platform for certain domains within embedded computing. This is a much easier undertaking as embedded systems not have the same requirements regarding ease of use and completeness regarding features. The GUI subsystem goes away and hardware support need not be as broad as for desktop OSes. I also think that Genode has several additional properties that are highly sought after in various embedded domains like responsiveness, stability and power efficiency. These characteristics should be exploited together with the main benefit of Genode as a secure and safe OS to gain maximum impact.
The interesting question is: What is missing?
Application examples and white papers outlining unknown areas where Genode shines and then maybe some polishing to make them shine better. Well designed examples kick-starts developers imagination in the various domains and that matters. For instance, if it can be shown that embedded control can be performed both securely and with very low latency, say submillisecond latency, a new domain opens up in sensitive control systems, say power plants. Today the solution is to cut of internal networks from the Internet due to internal control communication being totally insecure and unencrypted (and you don't trust the endpoints). These plants would gain very much if they could connect internal networks to the internet to upload and download all kinds of stuff. Today the solution is to put truly awkward filters in the path or maybe a data diode that only allows for incoming traffic. A big disadvantage when considering that what you really want is to mine data out of the plant.
I am glad to see that you have found the state machine being a usable pattern. I happen to believe that it would be very beneficial for Genode to mostly switch to a fully asynchronous communication paradigm based on independent cooperative state machines. Such a paradigm would maybe need some enhancements to the asynchronous communication protocol to be truly efficient. For instance I believe it might be beneficial if parameters could be added to signals in a similar fashion to that of RPCs. I have seen the motivation for not adding parameters to signals, but I fail to see why the various components shouldn't be able to donate dataspace for such storage like they may donate dataspace for RPC call parameter storage.
Regardless of how buffers are dimensioned and who pays for them, they can overrun. The crux is how to deal with this situation?
Look at the Internet, it is not failsafe and neither need your framework be. Not all applications depend on all signals going forward. Sometimes only the latest data matters like for control applications. Older control values simply doesn't matter. In games the result will for instance be some little glitch in an update sequence. In practice not even that happens as the missing signal is detected and the game runs some smoothing filter over the missing signal.
If you know the channel isn't failsafe that knowledge will have to be built into the application like it is for TCP/IP. My idea (taken from [1]) is to let asynchronous signals with data go via ring buffers. If these overflow you lose the oldest data and so it be. If a consumer is concerned with loosing data then just donate some additional ring buffer capacity to reduce the risk. Given that Genode can be finely tuned with respect to process priorities, CPU access slots and memory allocation this problem should not be very big.
If the signal sender is responsible to respond to such an overrun, signals would lose their fire-and-forget semantics because the sender may need to deal with the unexpected consequences of submitting a signal. How would that be any better than the use of synchronous RPC in the first place? If the receiver is responsible, it becomes prone to denial-of-service problems. In either case, we would create an inter-dependency between both parties that is not desired.
Denial of service is a risk yes. But so is sidechannel attacks, the physical hardware and many other things which Genode doesn't solve. Pick your poison sort of choice. I think some design guides for asynchronous signals with data outlining the dangers and pitfalls might be an option. Remember that the risk profile differs among users. Sometimes data leakage is what matters, with service availability a distant second. If you cannot support them all (because of complexity maybe) you should at least know why you chose one over the other.
The feature you describe (signals with arguments) is actually accommodated by the packet stream, which uses a combination of signals with a shared dataspace. But it escalates the buffer-overrun problem to the user of the API.
The current packet stream does not allow for several consumers, only one from what I understand. But yes, it looks like a usable option to start with if we care less for efficiency. But likely only little need to be added to allow different consumers to have their own read pointers in a circular buffers. Additional logic is also required to handle the case of overflows and the accounting for donated dataspace from different consumers. Hopefully not too complicated stuff.
I checked the loopback example you provided regarding a server component written according to the state machine pattern. I find it somewhat rudimentary and full of if then statements. Now the task at hand is simple so a simple pattern makes sense. ... I believe that Genode would gain in efficiency and clarity by integrating a solid state machine framework supporting UML statecharts to some degree including hierarchical state machines, ...
I followed the pointers you provided but remain unconvinced about the value that UML state charts bring to the table. I guess, this is caused by my lack of experience with such tools. It goes without saying that I am all for reducing complexity and increasing clarity. Right now, I simply fail to grasp how the FSM frameworks you cited would improve the situation. It looks like I will need to study the book you referred to.
Alternatively, have you considered to prove your assertion by modeling the NIC-loopback example with the tools you suggest? Maybe this example would help me to understand the benefit.
I am planning an example but not for the NIC loopback as it is too trivial to show on any advantages. The current line of work is to build the FSM from scratch to make better use of modern C++, avoid licensing issues and better integrate the solution with the existing signal mechanism. The idea is to allow for several cooperating FSMs to coexist in a protection domain that also can talk to FSMs in other components via the existing Genode communication infrastructure. An early prototype might be the loopback component, the real test is likely a new ip or udp stack layer.
My recommendation now is that Norman and the Genodians take a deeper look at the state of the art regarding state machine design and consider if there is some merit in making use of some existing frameworks or standards, most notably the UML statecharts standard.I can of course implement the required functionality myself but I believe that there is real benefit in having full integration into Genode to ease integration with built in components.
Thank you for the recommendation. To come to a conclusion how this information may benefit Genode, someone would first need to investigate, understand, and experiment. Admittedly, I am unable to connect the dots right now. But I don't want to sit on the critical path here. You seem to see a clearer picture. It certainly would be the best for you to go forward and share your findings.
Having experts taking a look at the various trade offs regarding state machine framework integration into Genode within the areas of signaling, execution, implementation patterns and so on also is of real benefit.
I'd be happy to review such work from Genode's perspective.
Some changes have been initiated to make Genode more secure and less error prone. I think of the abolition of unnecessary pointers and the planned removal of global state. These are standard recommendations in a number of prominent style guides [4, 5].
I skimmed over the Misra document [5], which is indeed very useful. It turns out that we happen to intuitively follow most of the rules. In the few cases where we don't comply, we have a rationale. Thanks for the good pointer!
I would like to see a more systematic approach to the improvement of Genode. As Bruce Schneier once stated; "Security is a process, not a product" [6]. With this I mean that objectives first should be decided. Is security and safety important? Is it important to gain the trust of people with little knowledge of Genode internals? If yes to both questions Misra C++ compliance makes sense as it is the standard in the area. A subset of Misra C++ should then be selected for implementation over time. Rules not implemented should be motivated very well as to why they not pose a security threat. Then the rules should be ordered in a list and gradually implemented and thereafter checked for adherence by tools like Flexelint. This is the process that some day might lead to Misra C++ compliance. The same reasoning apply to the upcoming C++ core guidelines or anything else worth adhering to.
I sense that this discussion is rooted in your desire to give Genode a stamp (aka certification) that expresses a certain code quality standard
- a stamp that non-technical people can understand.
Technically, while acknowledging that our code can always be improved, I don't think that code quality holds Genode back. In contrary, the feedback we get about the code is often quite enthusiastic. The real questions seem to be: What are the steps needed to obtain the desired stamp, and who invests the effort to take those steps?
I am not at all opposed to adopting the Misra C++ rules as I agree with them (a few exceptions notwithstanding). But personally, I miss a strong-enough incentive to pursue this direction on our own account.
Personally I have only been involved in projects over the last couple years that do require Misra compliance. That said, I believe you first should ask your current customers if Misra compliance adds anything of value, if not Misra can wait. I also believe that you might miss out on opportunities where requirements state Misra compliance but you cannot provide it, a pity when it looks like you actually can do it quite easily.
The benefits of Misra compliance it is that you get an independent judgement regarding certain qualities of your code that matter to many customers. If it is worth the effort aiming for compliance depends on your opinion of what your future customers will want.
Regarding the upcoming update of the book Genode Foundations. What I miss in the book is a connection the real code and examples. I believe that new readers much easier understand and remember how to use Genode if the connection to code and the inner workings are made clearer. Here I would take the opportunity to again recommend [1] as a book that makes this connection very clear. OK, it is easier to describe state machines but I still believe that Norman could find some inspiration in that book for the upcoming release of Genode Foundations.
I don't plan to change the existing book structurally. The update will merely adjust the content to the changes of the code (API, source-code layout, internals).
Do you think that your needs regarding code examples could be satisfied with a series of tutorial-like articles like I suggested in my original email?
I don't believe that you efficiently first read theory for 300 pages and then start reading another 300 pages of examples. Better is to mix theory with practice. All efficient teaching makes use of that formula. Just check some classes on Khan Academy or EdX and you will see that they switch frequently between theory and practice. Exactly this is done by Samek in [1] in quite an elegant way for the same topic, i.e. the presentation of an open source framework.
See you at FOSDEM,
Very cool! :-)
I'd like to come back to the original topic - the road map. Do you have concrete plans that you are willing to share with us?
I have a vision but it is not a plan until I master Genode. First thing first. 2016 is in my roadmap a travel from novice to guru... and beyond :)
Cheers Norman
-- Dr.-Ing. Norman Feske Genode Labs
http://www.genode-labs.com · http://genode.org
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
Hi Pontus,
Not all applications depend on all signals going forward. Sometimes only the latest data matters like for control applications. Older control values simply doesn't matter.
I agree that there are situations where intermediate states can be dropped. But there are other situations (think of a block-write request) where this must not happen. Therefore, the low-level mechanisms provided by Genode must support both.
I am wondering, do you feel the need to dispute Genode's low-level mechanisms at this point? They have been designed with great care for orthogonality, accountability, dependability, and performance over the course of the past 10 years while considering the wealth of use cases we encountered on our way towards general-purpose computing.
Please don't get me wrong. I don't want to dismiss your ideas. But I think that they can be realized on top of the existing low-level mechanisms (signals, RPC, shared memory) quite well. We don't need to change Genode to accommodate them. Should Genode's mechanisms become the bottleneck, I am of course open for changes. But this should be evidenced by real application scenarios first.
If you know the channel isn't failsafe that knowledge will have to be built into the application like it is for TCP/IP. My idea (taken from [1]) is to let asynchronous signals with data go via ring buffers. If these overflow you lose the oldest data and so it be. If a consumer is concerned with loosing data then just donate some additional ring buffer capacity to reduce the risk. Given that Genode can be finely tuned with respect to process priorities, CPU access slots and memory allocation this problem should not be very big.
For cases like this, which I would subsume as state propagation, I recommend you to consider the report and ROM session interfaces. The report session allows a component to export state in a non-blocking way whereas the ROM session allows a component to respond to and import external state.
The current packet stream does not allow for several consumers, only one from what I understand.
For the propagation of state from one producer to multiple consumers, please consider the report-ROM component, which represents a publisher-subscriber mechanism. It implicitly drops intermediate states if reports are produced at a higher rate than consumed (as ROMs). In principle, similar components could be created for other session interfaces such as the NIC session (for broadcasting NIC packets to many clients).
But likely only little need to be added to allow different consumers to have their own read pointers in a circular buffers.
Keeping this consumer-specific state is what Genode's sessions are for. E.g., for a NIC server with multiple clients, each session holds the read and write pointers for the respective client (the request queue and ack queue of the packet stream). Furthermore, the communication buffer is tied to the respective session, which answers the question of how to dimension and accounting it.
I am planning an example but not for the NIC loopback as it is too trivial to show on any advantages. The current line of work is to build the FSM from scratch to make better use of modern C++, avoid licensing issues and better integrate the solution with the existing signal mechanism. The idea is to allow for several cooperating FSMs to coexist in a protection domain that also can talk to FSMs in other components via the existing Genode communication infrastructure. An early prototype might be the loopback component, the real test is likely a new ip or udp stack layer.
This sounds interesting! If you have anything to show, I'd be delighted to have a look. :-)
Cheers Norman
A short an quick answer to Normans last reply:
* I think top priority for the 2016 Roadmap should be given to paying customers of Genode Labs without whom Genode would be a much less useful product to the rest of us. * Of course no major change in the Genode API should be planned from a simple suggestion in this mail thread without deeper motivation in terms of use cases and working examples. I plan to create some examples with the existing mechanisms suggested by Norman to see what performance is achieved in various areas like code quality, stability and speed. If I get that far some basis for change to the 2017 Roadmap may be at hand. The reason I highlighted the issue with state machines is that I believe they may be one additional killer feature of Genode if implemented correctly throughout and therefore I gave this heads up now. BTW, hierarchical state machines (HSMs) can grow to 10s of states without having an explosion in the number of transitions. With cooperating independent HSMs the number of states scale even higher. This is why HSMs and possibly cooperating HSMs (within a component) merit support by the Genode OS rather than just plain flat FSMs. * I give thumbs up for encrypted external block storage and I second Guido's request for better support for the seL4 kernel.
Regards,
Pontus
On Mon, Jan 11, 2016 at 11:51 AM, Norman Feske <norman.feske@...1...> wrote:
Hi Pontus,
Not all applications depend on all signals going forward. Sometimes only the latest data matters like for control applications. Older control values simply doesn't matter.
I agree that there are situations where intermediate states can be dropped. But there are other situations (think of a block-write request) where this must not happen. Therefore, the low-level mechanisms provided by Genode must support both.
I am wondering, do you feel the need to dispute Genode's low-level mechanisms at this point? They have been designed with great care for orthogonality, accountability, dependability, and performance over the course of the past 10 years while considering the wealth of use cases we encountered on our way towards general-purpose computing.
Please don't get me wrong. I don't want to dismiss your ideas. But I think that they can be realized on top of the existing low-level mechanisms (signals, RPC, shared memory) quite well. We don't need to change Genode to accommodate them. Should Genode's mechanisms become the bottleneck, I am of course open for changes. But this should be evidenced by real application scenarios first.
If you know the channel isn't failsafe that knowledge will have to be built into the application like it is for TCP/IP. My idea (taken from [1]) is to let asynchronous signals with data go via ring buffers. If these overflow you lose the oldest data and so it be. If a consumer is concerned with loosing data then just donate some additional ring buffer capacity to reduce the risk. Given that Genode can be finely tuned with respect to process priorities, CPU access slots and memory allocation this problem should not be very big.
For cases like this, which I would subsume as state propagation, I recommend you to consider the report and ROM session interfaces. The report session allows a component to export state in a non-blocking way whereas the ROM session allows a component to respond to and import external state.
The current packet stream does not allow for several consumers, only one from what I understand.
For the propagation of state from one producer to multiple consumers, please consider the report-ROM component, which represents a publisher-subscriber mechanism. It implicitly drops intermediate states if reports are produced at a higher rate than consumed (as ROMs). In principle, similar components could be created for other session interfaces such as the NIC session (for broadcasting NIC packets to many clients).
But likely only little need to be added to allow different consumers to have their own read pointers in a circular buffers.
Keeping this consumer-specific state is what Genode's sessions are for. E.g., for a NIC server with multiple clients, each session holds the read and write pointers for the respective client (the request queue and ack queue of the packet stream). Furthermore, the communication buffer is tied to the respective session, which answers the question of how to dimension and accounting it.
I am planning an example but not for the NIC loopback as it is too trivial to show on any advantages. The current line of work is to build the FSM from scratch to make better use of modern C++, avoid licensing issues and better integrate the solution with the existing signal mechanism. The idea is to allow for several cooperating FSMs to coexist in a protection domain that also can talk to FSMs in other components via the existing Genode communication infrastructure. An early prototype might be the loopback component, the real test is likely a new ip or udp stack layer.
This sounds interesting! If you have anything to show, I'd be delighted to have a look. :-)
Cheers Norman
-- Dr.-Ing. Norman Feske Genode Labs
http://www.genode-labs.com · http://genode.org
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
On Fri, 8 Jan 2016, Norman Feske wrote:
Hi Pontus,
welcome to the list and thank you for the thought-provoking comments!
First, I must say that I still have some trouble understanding the use case for Genode as a desktop OS. Not that I don't like the idea but I think this is such an enormous undertaking that a believe might be like aiming too far or maybe like missing lower hanging fruit.
I will try to clarify the goal and motivation.
The goal is to enable the productive use of Genode by a narrow group of users with reasonably with well-defined requirements. I gave two examples of such groups, namely us developers and the users of a disposable TOR-browser appliance. The immediate goal is *not* to create a desktop OS for the masses. This would be unrealistic. But with the goal narrowed like this, the aspiration is feasible.
As a person who aspires to use Genode, and therefore also aspires to see the community of developers who contribute to Genode grow, what you say makes sense and I see how the desktop is more important than it might appear.
I'm late to the discussion, but for very similar reasons, may I offer this, perhaps obvious (?) use case as a driver for development: IoT applications on platforms that are appealing and accessible to "amateur" IoT developers, who likely are also professional programmers, playing with Genode in their spare time.
There is a clear 'itch to scratch' for developers not satisfied with the IoT marketplace -- security, freedom, hackability. Of course it is a growth area, so there is less entrenched competition. To make another obvious point, the risk, cost, and inconvenience of getting started with a Raspberry Pi that is not one's main desktop/laptop are all very low, especially since many people already own one that is sitting in a cupboard unused!
I don't know what really attracts the developers who contribute the most -- I think it is probably quite individual -- but two things that it seems can only help are:
1. Increasing the audience of potential Genode developers by supporting popular platforms that can be used for IoT applications, like Raspberry Pi and Raspberry Pi 2.
2. People like helping others, so an audience of people not developing Genode, but using it to develop IoT apps, might encourage some new Genode developers. To this end, I'd like to suggest targetting ease of getting "up and running" with these platforms: for example, it would be amazing if there were a downloadable RPi2 image that one could just write to a micro SD card, ssh in and run bash and python to process video from the standard raspberrypi.org camera module. Of course I'm sure this is a lot of work, but I think even steps towards it are likely to increase the interested audience who will push it further in that direction.
IoT makes a connection between tinkering and work. A similar combination is arguably what got Linux where it is now...
On a separate subject: I'm sure any progress on seL4 will attract new people.
John
I'll also add that supporting more kernels (perhaps aside from seL4) is unlikely to help anywhere near as much as stabilizing and documenting the Genode API. Right now, it takes a lot of searching to get even a basic familiarity with any given part of the API (e.g. filesystems).
On Sat, Jan 16, 2016 at 2:47 PM, John Lee <jjl@...348...> wrote:
On Fri, 8 Jan 2016, Norman Feske wrote:
Hi Pontus,
welcome to the list and thank you for the thought-provoking comments!
First, I must say that I still have some trouble understanding the use case for Genode as a desktop OS. Not that I don't like the idea but I think this is such an enormous undertaking that a believe might be like aiming too far or maybe like missing lower hanging fruit.
I will try to clarify the goal and motivation.
The goal is to enable the productive use of Genode by a narrow group of users with reasonably with well-defined requirements. I gave two examples of such groups, namely us developers and the users of a disposable TOR-browser appliance. The immediate goal is *not* to create a desktop OS for the masses. This would be unrealistic. But with the goal narrowed like this, the aspiration is feasible.
As a person who aspires to use Genode, and therefore also aspires to see the community of developers who contribute to Genode grow, what you say makes sense and I see how the desktop is more important than it might appear.
I'm late to the discussion, but for very similar reasons, may I offer this, perhaps obvious (?) use case as a driver for development: IoT applications on platforms that are appealing and accessible to "amateur" IoT developers, who likely are also professional programmers, playing with Genode in their spare time.
There is a clear 'itch to scratch' for developers not satisfied with the IoT marketplace -- security, freedom, hackability. Of course it is a growth area, so there is less entrenched competition. To make another obvious point, the risk, cost, and inconvenience of getting started with a Raspberry Pi that is not one's main desktop/laptop are all very low, especially since many people already own one that is sitting in a cupboard unused!
I don't know what really attracts the developers who contribute the most -- I think it is probably quite individual -- but two things that it seems can only help are:
- Increasing the audience of potential Genode developers by supporting
popular platforms that can be used for IoT applications, like Raspberry Pi and Raspberry Pi 2.
- People like helping others, so an audience of people not developing
Genode, but using it to develop IoT apps, might encourage some new Genode developers. To this end, I'd like to suggest targetting ease of getting "up and running" with these platforms: for example, it would be amazing if there were a downloadable RPi2 image that one could just write to a micro SD card, ssh in and run bash and python to process video from the standard raspberrypi.org camera module. Of course I'm sure this is a lot of work, but I think even steps towards it are likely to increase the interested audience who will push it further in that direction.
IoT makes a connection between tinkering and work. A similar combination is arguably what got Linux where it is now...
On a separate subject: I'm sure any progress on seL4 will attract new people.
John
Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
I would agree that the file system APIs should be included in the next release of the manual, now that the VFS has outgrown libc and Noux. If a volunteer is needed I could make draft.
Emery
On Sat, Jan 16, 2016 at 07:08:39PM -0700, Nobody III wrote:
I'll also add that supporting more kernels (perhaps aside from seL4) is unlikely to help anywhere near as much as stabilizing and documenting the Genode API. Right now, it takes a lot of searching to get even a basic familiarity with any given part of the API (e.g. filesystems).
On Sun, Jan 17, 2016 at 11:38:56AM +0100, Emery wrote:
I would agree that the file system APIs should be included in the next release of the manual, now that the VFS has outgrown libc and Noux. If a volunteer is needed I could make draft. Emery
Hey Emery,
Sorry to hijack this thread but I've been meaning to ask: Have you considered using Guix instead of Nix? I feel that as we all get more and more serious when it comes to a package manager for Genode, Nix may be a bit of a low level choice. Guix is fairly new but has a few key features that may be of interest to the Genode project in particular:
- A focus on reproducible builds.
Guix has a focus on reproducible builds meant to be shared with others in a decentralized fashion, rather than from a single source. For a system with a focus on security, it'd be nice to see a package manager that keeps us honest.
- An architecture intended to be composable.
I think this is probably the biggest gap between Nix and Guix in philosophy. Guix is intended to be composable- you can use it within existing systems, VMs, disk images, containers and on bare metal. Rather than being just a package manager, it's a complete environment manager that you can use to bootstrap.
On a technical level, support for more fine grained environments like HURD is going to happen in the future, so perhaps it'd be interesting to follow that.
- Support for importing from other package managers.
Something you've hinted at before is having a Noux builder and perhaps down the track a Genode builder- Guix has things like Python, Ruby, Haskell and Nixpkgs import tools that download packages and fill in some boilerplate to make creating new packages easier. In addition it separates build systems like Cabal, Autotools and CMake which are defined on a package level.
I'd love to see a Genode system using a tool like Guix to have 'one' system running across VMs, containers, and languages with all the inter-language dependencies figured out and a common store.
I understand it's a huge leap from Nix to Guix, but I thought it might be nice to dream a little. Guix still uses Nix under the hood, so it can still benefit. Of course, Nix could provide all this eventually but then you just get Guix.
Cheers, Jookia.
I'm not really interested in Guix because Emacs gave me an aversion to Gnu projects involving lisp. My impression also has been that Guix is a duplication of effort for the sake of political correctness, and thus has a social and political dimension I'd rather not deal with.
Emery
Norman, and fellow Genodians-
I think that Genode's embrace of the BSD rump driver stack will continue to be a key lever towards the I/O drivers available for use within Genode (hopefully without any Source code modification).. In that sense it seems the Genode team has already had it's mot critical of Ah-Ha moments... and perhaps, continuing to leverage this Rump Kernel approach will remain a key strategy that could keep Genode developers focused where they can truly bring the value..as a properly secured OS kernel / programming model..
So I'm voting that the Genode team continues to grow it's working relationship with the BSD Rump Kernel camp... (so that the BSD Rump team might also make any needed adjustment to the Rump Kernel API, such that Genode's needs are more efficiently met)... and in the longer run... the BSD camp begins to see the wisdom of utilizing Genode's OS kernel API as a potential programming model for use in parts of the future BSD OS... Yes I know I'm dreaming here....but who says Genode can not have it all...it certainly has something to offer (ie. SYSTEM SECURITY)..
All the best -Peter
Hi everyone,
just before Norman finalizes the roadmap I want to make some remarks myself. It is always easy to point out single features or even applications that one might want to have in or use on Genode and those differ between people. So rather than providing my personal wishlist of features x y z I want to give some brief insight after using Genode on a daily basis for a few months (referring to a *slightly* modified system configuration based on the Turmvilla scenario):
It works incredibly well (stable and fast) for what it is configured, as long as you do not try anything fancy. You can interactively start and stop subsystems and change them at run-time. You can also change the configuration of several components, which makes the whole system usable and even quite fun to work with or rather on. But at its core, the system configuration is still static. You have to know beforehand which component provides which service and you have to route components (e.g. which hard disk should be used etc) accordingly. By all means, in some use cases that is a good thing but might be a limiting factor when considering, e.g., desktop computing.
Ideally, it would be nice if most of the configuration but the essential one could be done at run-time. Unfortunately, not all components in question support reporting their state or allow configuration changes at run-time — we definitely should improve that.
Therefore, identifying an essential core system configuration, that provides enough functionality to bootstrap a subsequent system might be reasonable. I know, such a configuration always depends on the use case at hand but still. For debugging purposes we currently depend on some kind of interface that provides us with the much needed LOG output. For better or worse not all machines provide such an interface and that might limit the application area of Genode — maybe we can come up with some neat component/configuration that gets rid of this dependency?
Relating to some points discussed earlier in this thread, the lack of hardware support is for me personally not such a big deal. At the end of the day it is either the vesa_fb_drv, intel_fb_drv or maybe even a mali_fb_drv that provides the Framebuffer service. What is more important at the moment is how the components are linked together. Also, w/o the help of the community focusing too much an supporting interesting hardware platforms will probably draw manpower off from more pressing tasks (I would love to ditch x86 for some “better” platform but I do not see that happing any time soon). On that account, I also do not see the current lack of appealing GUI agents as such an important site — than again I fancy Plan 9 and should probably not raise my voice on this topic anyway ;-)
So, in contrast to 2015 where among other things stability was the most prominent field of activity, 2016 should probably be the year where we try to make the most of the system, by providing means to ease the task of configuring and integrating a Genode based system.
Regards Josef
I agree with Josef. I've actually been thinking about this issue myself. Here are my ideas for how we could solve the issue without too much difficulty: 1. Make init terminate all children and restart when the config is updated. (for nested init configurations; possibly already implemented) 2. Create servers to forward/virtualize services and to quietly reconnect (if possible) whenever they lose their connection to the service they forward. (The vfs server fills this role, though it would be nice for it to handle filesystem driver failure as smoothly as possible.) 3. Allow init to provide services from its children.
On Mon, Jan 11, 2016 at 2:08 PM, Josef Söntgen < josef.soentgen@...1...> wrote:
Hi everyone,
just before Norman finalizes the roadmap I want to make some remarks myself. It is always easy to point out single features or even applications that one might want to have in or use on Genode and those differ between people. So rather than providing my personal wishlist of features x y z I want to give some brief insight after using Genode on a daily basis for a few months (referring to a *slightly* modified system configuration based on the Turmvilla scenario):
It works incredibly well (stable and fast) for what it is configured, as long as you do not try anything fancy. You can interactively start and stop subsystems and change them at run-time. You can also change the configuration of several components, which makes the whole system usable and even quite fun to work with or rather on. But at its core, the system configuration is still static. You have to know beforehand which component provides which service and you have to route components (e.g. which hard disk should be used etc) accordingly. By all means, in some use cases that is a good thing but might be a limiting factor when considering, e.g., desktop computing.
Ideally, it would be nice if most of the configuration but the essential one could be done at run-time. Unfortunately, not all components in question support reporting their state or allow configuration changes at run-time — we definitely should improve that.
Therefore, identifying an essential core system configuration, that provides enough functionality to bootstrap a subsequent system might be reasonable. I know, such a configuration always depends on the use case at hand but still. For debugging purposes we currently depend on some kind of interface that provides us with the much needed LOG output. For better or worse not all machines provide such an interface and that might limit the application area of Genode — maybe we can come up with some neat component/configuration that gets rid of this dependency?
Relating to some points discussed earlier in this thread, the lack of hardware support is for me personally not such a big deal. At the end of the day it is either the vesa_fb_drv, intel_fb_drv or maybe even a mali_fb_drv that provides the Framebuffer service. What is more important at the moment is how the components are linked together. Also, w/o the help of the community focusing too much an supporting interesting hardware platforms will probably draw manpower off from more pressing tasks (I would love to ditch x86 for some “better” platform but I do not see that happing any time soon). On that account, I also do not see the current lack of appealing GUI agents as such an important site — than again I fancy Plan 9 and should probably not raise my voice on this topic anyway ;-)
So, in contrast to 2015 where among other things stability was the most prominent field of activity, 2016 should probably be the year where we try to make the most of the system, by providing means to ease the task of configuring and integrating a Genode based system.
Regards Josef
Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
Hi,
Last year Genode gained support to run as a subject on top of the Muen Separation Kernel (SK) [1]. We are very happy about the results and how well the collaboration with the Genode developers worked out!
We would like to take the opportunity to share our goals and ideas for 2016.
Our motto for 2016 is to enable the Turmvilla scenario on hw_x86_64_muen. To achieve this goal we have identified the following work items:
* Finalize hw_x86_64_muen VirtualBox support
Last year we ported the Genode VirtualBox support to hw_x86_64_muen and are now able to successfully virtualize 32-bit guest operating systems including Windows versions 7-10. Proper VirtualBox support is a requirement to enable Turmvilla as it depends on running a rich OS alongside regular Genode components.
In order to submit these enhancements for upstream inclusion, we need to cleanup the development history and split the work into reviewable chunks.
* Support for 64-bit VirtualBox guest VMs
Once the VirtualBox support is accepted into staging, we intend to add support for 64-bit guests. We expect this to be straight forward as Muen already provides an execution environment for 64-bit Linux VMs using a Virtual Machine Monitor (VMM) written in SPARK 2014.
* Distributed Genode
Having a complete Genode system running in one Muen subject does not bring much benefit as it does not leverage the isolation properties provided by the SK. Therefore we intend to add support for executing multiple Genode subjects as a distributed environment while providing inter-subject communication channels. In order for the Genode instances to exchange data, the concept of so-called Proxy components as suggested by the Genode developers will be applied.
The proxy concept uses two components: a server and a client. Both of them implement a specific Genode service interface and communicate using a Virual Hardware (VHW) abstraction layer to create a bridge between the separated components. The following diagram illustrates the concept for the Log service:
log() -> Proxy:Client -> | VHW | -> Proxy:Server -> Log_Server
For Muen, we will implement the VHW functionality using shared memory channels and events.
While we plan to tackle the items listed above in 2016, we would also like to mention areas of interest that we will not be able to actively pursue for now:
* Support for Ada
It would be nice to have support for native Ada components in Genode. Initially, the Genode toolchain would need to be built with Ada enabled.
To make Ada a first class citizen of the Genode ecosystem, Ada bindings for Genode services would need to be provided.
* Investigate Intel GVT on Genode
Since Genode has ported the Linux i915 graphics driver, it would be interesting to see whether it is possible to apply the Intel Graphics Virtualization Technology (Intel GVT) concept [2] for mediated graphics device passtrough.
These are our inputs for the Genode roadmap 2016, we are looking forward to an eventful year!
Cheers - reto
[1] - http://genode.org/documentation/release-notes/15.08#Genode_on_top_of_the_Mue... [2] - https://01.org/igvt-g
On Tue, Jan 12, 2016 at 04:32:20PM +0100, Reto Buerki wrote:
- Distributed Genode
Having a complete Genode system running in one Muen subject does not bring much benefit as it does not leverage the isolation properties provided by the SK. Therefore we intend to add support for executing multiple Genode subjects as a distributed environment while providing inter-subject communication channels. In order for the Genode instances to exchange data, the concept of so-called Proxy components as suggested by the Genode developers will be applied.
The proxy concept uses two components: a server and a client. Both of them implement a specific Genode service interface and communicate using a Virual Hardware (VHW) abstraction layer to create a bridge between the separated components. The following diagram illustrates the concept for the Log service:
log() -> Proxy:Client -> | VHW | -> Proxy:Server -> Log_Server
For Muen, we will implement the VHW functionality using shared memory channels and events.
This seems like a really interesting idea, and something I've thought about for a while. Could you elaborate a little more on what you think the VHW functionality might entail? Is it specific to the kernel?
I somewhat imagine a Genode system running over a network with different kernels per-device, so in that scenario it'd be a network VHW.
On another note, this is some really great work being done, I'm excited to see what comes of it- Perhaps the next Qubes OS for x86_64.
Cheers
- reto
Cheers, Jookia.
On 01/12/2016 09:56 PM, Jookia wrote:
On Tue, Jan 12, 2016 at 04:32:20PM +0100, Reto Buerki wrote:
- Distributed Genode
Having a complete Genode system running in one Muen subject does not bring much benefit as it does not leverage the isolation properties provided by the SK. Therefore we intend to add support for executing multiple Genode subjects as a distributed environment while providing inter-subject communication channels. In order for the Genode instances to exchange data, the concept of so-called Proxy components as suggested by the Genode developers will be applied.
The proxy concept uses two components: a server and a client. Both of them implement a specific Genode service interface and communicate using a Virual Hardware (VHW) abstraction layer to create a bridge between the separated components. The following diagram illustrates the concept for the Log service:
log() -> Proxy:Client -> | VHW | -> Proxy:Server -> Log_Server
For Muen, we will implement the VHW functionality using shared memory channels and events.
This seems like a really interesting idea, and something I've thought about for a while. Could you elaborate a little more on what you think the VHW functionality might entail? Is it specific to the kernel?
VHW is the abstract term for the transport medium and might be specific to the kernel. On Muen the medium is shared memory in combination with events. One can think of it as a virtual device in the sense that state is visible in memory (analogous to device MMIO) and IRQs are used to signal the availability of new state.
Whether VHW should use a uniform interface needs to be seen, maybe it will be easier to just implement different proxy components per use case.
I think we will start off with a simple log client where the write() implementation stores the log buffer contents in shared memory and informs the server via an event about new data. The server will be a native Muen component running as separate subject.
I somewhat imagine a Genode system running over a network with different kernels per-device, so in that scenario it'd be a network VHW.
In the network VHW case the implementation might not be kernel specific, the proxy components would just reuse existing networking facilities.
On another note, this is some really great work being done, I'm excited to see what comes of it- Perhaps the next Qubes OS for x86_64.
Thanks, I'm glad you like our work.
Regards, - reto
Hi Reto,
thank you for this very valuable input!
- Distributed Genode
... The proxy concept uses two components: a server and a client. Both of them implement a specific Genode service interface and communicate using a Virual Hardware (VHW) abstraction layer to create a bridge between the separated components. The following diagram illustrates the concept for the Log service:
log() -> Proxy:Client -> | VHW | -> Proxy:Server -> Log_Server
The discussed approach for a "distributed Genode" system is actually appealing in several different contexts in addition to Muen, for example:
* Letting Genode components on a host machine interact with Genode components running on the USB Armory.
* Transparently move individual components of a Genode system to a cloud service provider.
* By running Genode on RISC-V-based FPGA softcores, a Genode system could be distributed over several softcores, thereby enforcing stronger (because hardware-based) isolation between components than a microkernel could.
Those examples could even be combined, which would open up completely new scenarios.
For Muen, we will implement the VHW functionality using shared memory channels and events.
That sounds very good. I look forward to the outcome of this line of work.
- Support for Ada
- Investigate Intel GVT on Genode
I find both ideas intriguing. I won't mention them on the road map, but maybe we are lucky and still find some time to investigate them a bit. :-)
Cheers Norman
Norman and fellow Genodeians,
I'm pretty much ready to hop into the Genode turmvila scenario, But it would be great to do so on a more capable (higher screen res) laptop... It would be great if there was full agreement at which fairly modern Laptop will be the next to be fully supported by Turmvila.... That is, it would be crazy to try to support all to many.... So if you folks can agree on a Model I'll make a point of acquiring the same. My vote would be for a Dell XPF_13 or XPF_15... but I'm flexible... just let me know what Laptop I should order for doing Genode development ??.... and I'm in... !!!... as a side note I will also plan on working with Genode on Zynq 7000 via SnickerDoodle http://krtkl.com/... so I'm jazzed...may my fellow Genodians have a good year!!
-Peter
I have a Dell Latitude 6430u (i7-3687U, 8GB RAM, ~250GB SSD), and everything except the sound works. Also, feel free to ask for help with configuration, since the Turmvilla run script assumes you have a specific partition configuration and a VirtualBox machine configured to run on Genode. I found setting up the VM fairly awkward, but I can send you my VM (and Genode configuration too, if you'd like), using Google Drive.
On Sat, Jan 16, 2016 at 8:13 PM, Peter Lindener <lindener.peter@...9...> wrote:
Norman and fellow Genodeians,
I'm pretty much ready to hop into the Genode turmvila scenario, But it would be great to do so on a more capable (higher screen res) laptop... It would be great if there was full agreement at which fairly modern Laptop will be the next to be fully supported by Turmvila.... That is, it would be crazy to try to support all to many.... So if you folks can agree on a Model I'll make a point of acquiring the same. My vote would be for a Dell XPF_13 or XPF_15... but I'm flexible... just let me know what Laptop I should order for doing Genode development ??.... and I'm in... !!!... as a side note I will also plan on working with Genode on Zynq 7000 via SnickerDoodle http://krtkl.com/... so I'm jazzed...may my fellow Genodians have a good year!!
-Peter
Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
Hi Peter,
On 17.01.2016 04:13, Peter Lindener wrote:
That is, it would be crazy to try to support all to many.... So if you folks can agree on a Model I'll make a point of acquiring the same. My vote would be for a Dell XPF_13 or XPF_15... but I'm flexible... just let me know what Laptop I should order for doing Genode development ??.... and I'm in... !!!...
within Genode Labs, we recently settled on the Lenovo Thinkpad x250 as modern reference machine. So this would be the best to get. When ordering, please make sure that the machine is configured with AMT support (to obtain low-level logs via serial-over-line).
Cheers Norman
I'm a little frustrated that the Lenovo Thinkpad x250 only haas one DIM slot, and is thus limited to 8Gb of ram... not all so good if one is apt to be compiling FPGA code.. It seems the X260 would fill the bill, but it wont be available until approx Feb 2016..
Is running Genode on a x260 likely to much of an issue with respect to I/O drver code and the like ?
-Peter
On Mon, Jan 18, 2016 at 4:41 AM, Norman Feske <norman.feske@...1...> wrote:
Hi Peter,
On 17.01.2016 04:13, Peter Lindener wrote:
That is, it would be crazy to try to support all to many.... So if you folks can agree on a Model I'll make a point of acquiring the same. My vote would be for a Dell XPF_13 or XPF_15... but I'm flexible... just let me know what Laptop I should order for doing Genode development ??.... and I'm in... !!!...
within Genode Labs, we recently settled on the Lenovo Thinkpad x250 as modern reference machine. So this would be the best to get. When ordering, please make sure that the machine is configured with AMT support (to obtain low-level logs via serial-over-line).
Cheers Norman
-- Dr.-Ing. Norman Feske Genode Labs
http://www.genode-labs.com · http://genode.org
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
Hello Peter,
On Mon, Jan 18, 2016 at 10:05:15PM -0800, Peter Lindener wrote:
I'm a little frustrated that the Lenovo Thinkpad x250 only haas one DIM slot, and is thus limited to 8Gb of ram... not all so good if one is apt to be compiling FPGA code..
You're right with the single DIMM slot, but it can be configured with one DIMM of 16 GiB like I did for the notebook on my desk:
*-memory description: System Memory physical id: 8 slot: System board or motherboard size: 16GiB *-bank description: SODIMM DDR3 Synchronous 1600 MHz (0.6 ns) product: 16KTF2G64HZ-1G6A1 vendor: Micron physical id: 0 serial: 17538116 slot: ChannelA-DIMM0 size: 16GiB width: 64 bits clock: 1600MHz (0.6ns)
So, the x250 could still fit your requirements.
It seems the X260 would fill the bill, but it wont be available until approx Feb 2016..
Is running Genode on a x260 likely to much of an issue with respect to I/O drver code and the like ?
I do not have information about the hardware the x260 will bring but you may expect standard peripherals (e.g., AHCI and keyboard/trackpoint) to work after some tweaking while the Intel graphics driver and audio will potentially need more effort. I suggest you to stick to the x250 for now except you're willing to potentially spend significant effort to enable the more stubborn devices.
Regards
Hello,
thanks to everyone who participated in the road-map discussion. You can find the resulting plan at the updated road-map page:
http://genode.org/about/road-map
Cheers Norman
-
unknown＠example.com -
Christian Helmuth -
Emery -
Emery Hemingway -
Guido Witmond -
John Lee -
Jookia -
Josef Söntgen -
Nobody III -
Norman Feske -
Peter Lindener -
Pontus Åström -
Reto Buerki -
Vladimir Lushnikov