I have added some comments inline on Normans reply. Maybe we can continue the discussion at FOSDEM? After all, the 2015 roadmap broke down after summer so it does not seem to be set in stone even if published next week just ahead of FOSDEM :)
See you,
Pontus
On Fri, Jan 8, 2016 at 6:11 PM, Norman Feske <norman.feske@...1...> wrote:
Hi Pontus,
welcome to the list and thank you for the thought-provoking comments!
First, I must say that I still have some trouble understanding the use case for Genode as a desktop OS. Not that I don't like the idea but I think this is such an enormous undertaking that a believe might be like aiming too far or maybe like missing lower hanging fruit.
I will try to clarify the goal and motivation.
The goal is to enable the productive use of Genode by a narrow group of users with reasonably with well-defined requirements. I gave two examples of such groups, namely us developers and the users of a disposable TOR-browser appliance. The immediate goal is *not* to create a desktop OS for the masses. This would be unrealistic. But with the goal narrowed like this, the aspiration is feasible.
The motivation behind this goal is manifold:
The use of Genode on the desktop stresses the point that Genode scales well.
The dynamic workloads of desktop computing present much stronger requirements with respect to the quality of Genode (in particular regarding stability, flexibility, and performance), compared to the domain of low-complexity static embedded systems. We want to satisfy those requirements, not avoid them. If we successfully overcome the challenges of dynamic desktop workloads, I am positive that we implicitly cover the weaker requirements of the embedded space.
By having us developers use Genode every day, we create the natural motivation to work on important problems that are otherwise easily dismissed as boring or as "minor details". Since I switched to Genode as my day-to-day OS, I discovered and fixed many of such issues that remained unnoticed in the years before.
By constantly stressing the system by a growing number of people, we get a certain kind of quality assurance for free. I am particularly thinking of device drivers (like graphics) for which no automated tests exist. In the past, regressions in those parts of the system sometimes remained unnoticed for a shockingly long time. By constantly using those parts, they stay healthy.
I am able to spontaneously demonstrate the Genode concepts live on the system that I use. This carries a much stronger message than bunch of slides.
It makes my geek heart very happy.
This answer makes perfect sense. In particular when I hear that the ones that pay for your services have this goal. Keep going.
It might be better to shorten the hops to the future in easier and faster achievable milestones. Genode then gets truly usable faster, albeit within a smaller community, but one that truly cares. It also gets tried in production sooner, which is very important to gain stability and real user feedback.
As a matter of fact, one particular "community that truly cares" are the ones who currently invest in the development on Genode by commissioning paid projects to Genode Labs. The majority of such project work is actually geared towards the desktop use case.
I can follow your argument. From the business perspective of a solution provider, it makes perfect sense to pick low-hanging fruits first. But that is not Genode's role. Genode is not picking fruits. Genode's role is the one of a ladder that allows you to pick high-hanging fruits also.
I like the metaphor "Genode is the ladder that allows the rest of us to pick high hanging fruit". But still I think it makes sense with some economics thinking like calculating the current value of investments. If you for instance can enable some new application domain with low effort it makes sense to do so. Here I think Vladimirs proposal regarding disk encryption makes sense. How much would it take for Genode to enable a stateless design according to Joannas ideas in http://blog.invisiblethings.org/papers/2015/x86_harmful.pdf ? Maybee not so much and it would open an new domain with lots of possibilities. Just an example.
One such step might be to make Genode a viable and usable platform for certain domains within embedded computing. This is a much easier undertaking as embedded systems not have the same requirements regarding ease of use and completeness regarding features. The GUI subsystem goes away and hardware support need not be as broad as for desktop OSes. I also think that Genode has several additional properties that are highly sought after in various embedded domains like responsiveness, stability and power efficiency. These characteristics should be exploited together with the main benefit of Genode as a secure and safe OS to gain maximum impact.
The interesting question is: What is missing?
Application examples and white papers outlining unknown areas where Genode shines and then maybe some polishing to make them shine better. Well designed examples kick-starts developers imagination in the various domains and that matters. For instance, if it can be shown that embedded control can be performed both securely and with very low latency, say submillisecond latency, a new domain opens up in sensitive control systems, say power plants. Today the solution is to cut of internal networks from the Internet due to internal control communication being totally insecure and unencrypted (and you don't trust the endpoints). These plants would gain very much if they could connect internal networks to the internet to upload and download all kinds of stuff. Today the solution is to put truly awkward filters in the path or maybe a data diode that only allows for incoming traffic. A big disadvantage when considering that what you really want is to mine data out of the plant.
I am glad to see that you have found the state machine being a usable pattern. I happen to believe that it would be very beneficial for Genode to mostly switch to a fully asynchronous communication paradigm based on independent cooperative state machines. Such a paradigm would maybe need some enhancements to the asynchronous communication protocol to be truly efficient. For instance I believe it might be beneficial if parameters could be added to signals in a similar fashion to that of RPCs. I have seen the motivation for not adding parameters to signals, but I fail to see why the various components shouldn't be able to donate dataspace for such storage like they may donate dataspace for RPC call parameter storage.
Regardless of how buffers are dimensioned and who pays for them, they can overrun. The crux is how to deal with this situation?
Look at the Internet, it is not failsafe and neither need your framework be. Not all applications depend on all signals going forward. Sometimes only the latest data matters like for control applications. Older control values simply doesn't matter. In games the result will for instance be some little glitch in an update sequence. In practice not even that happens as the missing signal is detected and the game runs some smoothing filter over the missing signal.
If you know the channel isn't failsafe that knowledge will have to be built into the application like it is for TCP/IP. My idea (taken from [1]) is to let asynchronous signals with data go via ring buffers. If these overflow you lose the oldest data and so it be. If a consumer is concerned with loosing data then just donate some additional ring buffer capacity to reduce the risk. Given that Genode can be finely tuned with respect to process priorities, CPU access slots and memory allocation this problem should not be very big.
If the signal sender is responsible to respond to such an overrun, signals would lose their fire-and-forget semantics because the sender may need to deal with the unexpected consequences of submitting a signal. How would that be any better than the use of synchronous RPC in the first place? If the receiver is responsible, it becomes prone to denial-of-service problems. In either case, we would create an inter-dependency between both parties that is not desired.
Denial of service is a risk yes. But so is sidechannel attacks, the physical hardware and many other things which Genode doesn't solve. Pick your poison sort of choice. I think some design guides for asynchronous signals with data outlining the dangers and pitfalls might be an option. Remember that the risk profile differs among users. Sometimes data leakage is what matters, with service availability a distant second. If you cannot support them all (because of complexity maybe) you should at least know why you chose one over the other.
The feature you describe (signals with arguments) is actually accommodated by the packet stream, which uses a combination of signals with a shared dataspace. But it escalates the buffer-overrun problem to the user of the API.
The current packet stream does not allow for several consumers, only one from what I understand. But yes, it looks like a usable option to start with if we care less for efficiency. But likely only little need to be added to allow different consumers to have their own read pointers in a circular buffers. Additional logic is also required to handle the case of overflows and the accounting for donated dataspace from different consumers. Hopefully not too complicated stuff.
I checked the loopback example you provided regarding a server component written according to the state machine pattern. I find it somewhat rudimentary and full of if then statements. Now the task at hand is simple so a simple pattern makes sense. ... I believe that Genode would gain in efficiency and clarity by integrating a solid state machine framework supporting UML statecharts to some degree including hierarchical state machines, ...
I followed the pointers you provided but remain unconvinced about the value that UML state charts bring to the table. I guess, this is caused by my lack of experience with such tools. It goes without saying that I am all for reducing complexity and increasing clarity. Right now, I simply fail to grasp how the FSM frameworks you cited would improve the situation. It looks like I will need to study the book you referred to.
Alternatively, have you considered to prove your assertion by modeling the NIC-loopback example with the tools you suggest? Maybe this example would help me to understand the benefit.
I am planning an example but not for the NIC loopback as it is too trivial to show on any advantages. The current line of work is to build the FSM from scratch to make better use of modern C++, avoid licensing issues and better integrate the solution with the existing signal mechanism. The idea is to allow for several cooperating FSMs to coexist in a protection domain that also can talk to FSMs in other components via the existing Genode communication infrastructure. An early prototype might be the loopback component, the real test is likely a new ip or udp stack layer.
My recommendation now is that Norman and the Genodians take a deeper look at the state of the art regarding state machine design and consider if there is some merit in making use of some existing frameworks or standards, most notably the UML statecharts standard.I can of course implement the required functionality myself but I believe that there is real benefit in having full integration into Genode to ease integration with built in components.
Thank you for the recommendation. To come to a conclusion how this information may benefit Genode, someone would first need to investigate, understand, and experiment. Admittedly, I am unable to connect the dots right now. But I don't want to sit on the critical path here. You seem to see a clearer picture. It certainly would be the best for you to go forward and share your findings.
Having experts taking a look at the various trade offs regarding state machine framework integration into Genode within the areas of signaling, execution, implementation patterns and so on also is of real benefit.
I'd be happy to review such work from Genode's perspective.
Some changes have been initiated to make Genode more secure and less error prone. I think of the abolition of unnecessary pointers and the planned removal of global state. These are standard recommendations in a number of prominent style guides [4, 5].
I skimmed over the Misra document [5], which is indeed very useful. It turns out that we happen to intuitively follow most of the rules. In the few cases where we don't comply, we have a rationale. Thanks for the good pointer!
I would like to see a more systematic approach to the improvement of Genode. As Bruce Schneier once stated; "Security is a process, not a product" [6]. With this I mean that objectives first should be decided. Is security and safety important? Is it important to gain the trust of people with little knowledge of Genode internals? If yes to both questions Misra C++ compliance makes sense as it is the standard in the area. A subset of Misra C++ should then be selected for implementation over time. Rules not implemented should be motivated very well as to why they not pose a security threat. Then the rules should be ordered in a list and gradually implemented and thereafter checked for adherence by tools like Flexelint. This is the process that some day might lead to Misra C++ compliance. The same reasoning apply to the upcoming C++ core guidelines or anything else worth adhering to.
I sense that this discussion is rooted in your desire to give Genode a stamp (aka certification) that expresses a certain code quality standard
- a stamp that non-technical people can understand.
Technically, while acknowledging that our code can always be improved, I don't think that code quality holds Genode back. In contrary, the feedback we get about the code is often quite enthusiastic. The real questions seem to be: What are the steps needed to obtain the desired stamp, and who invests the effort to take those steps?
I am not at all opposed to adopting the Misra C++ rules as I agree with them (a few exceptions notwithstanding). But personally, I miss a strong-enough incentive to pursue this direction on our own account.
Personally I have only been involved in projects over the last couple years that do require Misra compliance. That said, I believe you first should ask your current customers if Misra compliance adds anything of value, if not Misra can wait. I also believe that you might miss out on opportunities where requirements state Misra compliance but you cannot provide it, a pity when it looks like you actually can do it quite easily.
The benefits of Misra compliance it is that you get an independent judgement regarding certain qualities of your code that matter to many customers. If it is worth the effort aiming for compliance depends on your opinion of what your future customers will want.
Regarding the upcoming update of the book Genode Foundations. What I miss in the book is a connection the real code and examples. I believe that new readers much easier understand and remember how to use Genode if the connection to code and the inner workings are made clearer. Here I would take the opportunity to again recommend [1] as a book that makes this connection very clear. OK, it is easier to describe state machines but I still believe that Norman could find some inspiration in that book for the upcoming release of Genode Foundations.
I don't plan to change the existing book structurally. The update will merely adjust the content to the changes of the code (API, source-code layout, internals).
Do you think that your needs regarding code examples could be satisfied with a series of tutorial-like articles like I suggested in my original email?
I don't believe that you efficiently first read theory for 300 pages and then start reading another 300 pages of examples. Better is to mix theory with practice. All efficient teaching makes use of that formula. Just check some classes on Khan Academy or EdX and you will see that they switch frequently between theory and practice. Exactly this is done by Samek in [1] in quite an elegant way for the same topic, i.e. the presentation of an open source framework.
See you at FOSDEM,
Very cool! :-)
I'd like to come back to the original topic - the road map. Do you have concrete plans that you are willing to share with us?
I have a vision but it is not a plan until I master Genode. First thing first. 2016 is in my roadmap a travel from novice to guru... and beyond :)
Cheers Norman
-- Dr.-Ing. Norman Feske Genode Labs
http://www.genode-labs.com · http://genode.org
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main