My setup/environment: - Source: Branch: staging commit 17f3b559735027bcbde233ebacbb1f9d173a566c Date: Thu Aug 3 09:44:17 2017 -0500 - Target-genode-System: nova x86_64 - Host-System: genode-x86-gcc (GCC) 6.3.0 Ubuntu 17.04 QEMU emulator version 2.8.0(Debian 1:2.8+dfsg-3ubuntu2.3)
Hi,
I try to understand the difference between cli_monitor, launcher, launchpad and qt-launchpad.
I looked into the source code and I understand following:
- app/cli_monitor is the command line version to start child processes. It exists in the low-level OS components repo. - app/launcher is the UI version of cli_monitor and uses some headers from cli_monitor. It exists in the gems repo.
- app/launchpad is the UI version to start child processes. Launchpad is not using the cli_monitor headers. It exists in the demo repo. - app/qt5/qt_launchpad is the Qt version of launchpad. It uses lib/launchpad from demo repo. It exists in the libports repo.
Both cli_monitor headers and lib/launchpad using the Genode::Child and Genode::Child_policy API to managed the child compoments as described in section 8.17.2. Child management [1].
What I see between the cli_monitor/launcher and launchpad approach, that the configuration of the subsystems is different. But is there even more differences ?
May I asked, what are the future plans of these "launchers"? Cli_monitor is in the OS repo and sounds like the API is what I should use to implement some "own" launchers. Launch app in gems repo, as what I understand from the email from Norman (see Friday, 4. August 2017 11:56:38 CEST from Norman Feske) this is outdated. Launchpad is in the demo repo and what I see in the git commit history, this is also the older approach to "launch" subsystems in genode, or?
Here some background information, why I asked... I will use genode as a desktop system and for such a use case I need something to launch applications. I notice, that genode is currently implementing a package management and it sounds like for me, in the future I need a launcher that are flexible enough to start an application after I installed a package in genode :-) ... but step by step.
I'm a C++ developer and using STL/boost etc. but for UI I use Qt/QML. The idea, I will try to develope a launcher in Qt/QML (I know there is a launchpad in Qt ... but not in QML). I still try to find out, which approach is the best: Use the lib/launchpad or cli_monitor API or develope from scratch as described in section 8.17.2. Child management [1]. Maybe I will still using the current aprroaches (launchpad etc.), it depends on the email reply :-)
Cheers, Jörg
[1] http://genode.org/documentation/genode-foundations-17-05.pdf
Hi Jörg,
first, your understanding of the different components is correct. Thank for taking the time for the thorough investigation of the existing components.
What I see between the cli_monitor/launcher and launchpad approach, that the configuration of the subsystems is different. But is there even more differences ?
The launchpad and qt_launchpad were intended as quick demos, not a serious solutions. In fact, launchpad was one of the very first programs developed for Genode - just to showcase the dynamic creation/destruction of subsystems. CLI monitor is similar, but uses a textual interface.
Design-wise the disadvantage of the launchpad as well as CLI monitor (and even more so qt_launchpad) is the fact that the component is potentially complex (as it implements a UI) while being part of the TCB of each launched subsystems.
With the launcher, I explored the idea to separate the complex parts (i.e., the widget rendering) into a separate sandboxed component called menu view. This helps to significantly reduce the complexity of the launcher - and thereby the TCB of the launched subsystems. So software-design wise, the launcher is much more advanced.
However, all these solutions share one problem that holds them back for general-purpose scenarios: They can start and stop subsystems but do not allow the user to define any interaction between the subsystems. It is not possible to dynamically launch a new service to be used by another subsystem.
To overcome this limitation, I am currently investigating the use of a dynamic init instance as the host for subsystems. With this approach, the launcher merely generates configurations for the init instance and consumes reports generated by init. It goes without saying that the lessons learned from the menu-view approach can be applied in this scenario as well. Please refer to a previous discussion [1] that we had on the list earlier this year.
[1] https://sourceforge.net/p/genode/mailman/message/35787965/
May I asked, what are the future plans of these "launchers"? Cli_monitor is in the OS repo and sounds like the API is what I should use to implement some "own" launchers. Launch app in gems repo, as what I understand from the email from Norman (see Friday, 4. August 2017 11:56:38 CEST from Norman Feske) this is outdated. Launchpad is in the demo repo and what I see in the git commit history, this is also the older approach to "launch" subsystems in genode, or?
The launchpad is meant to stay as a quick-start example for exploring Genode. However, the other launchers will eventually be replaced by the dynamic init approach, once it is there.
Here some background information, why I asked... I will use genode as a desktop system and for such a use case I need something to launch applications. I notice, that genode is currently implementing a package management and it sounds like for me, in the future I need a launcher that are flexible enough to start an application after I installed a package in genode :-) ... but step by step.
I am currently working in the same direction. If you are interested to follow my work, you may have a look at the corresponding branch [2]. In my opinion, two particular interesting bits are the depot-query scenario [3] and the dynamic-drivers subsystem [4]. Both facilitate the dynamic init approach.
[2] https://github.com/nfeske/genode/commits/nitro [3] https://github.com/nfeske/genode/commit/cc2b7935fbdbe0bd3b342495bcc9d1da36e6... [4] https://github.com/nfeske/genode/commit/8046e4d2be50c183ad7ad3ba6e68d6e0a724...
But as a word of caution, this is work in progress. ;-)
Cheers Norman
Here's my work on a desktop environment for Genode: https://github.com/NobodyIII/genode-desktop-environment
So far, I have a working file manager and text editor. Feel free to use and modify my code.
On Fri, Aug 18, 2017 at 7:58 AM, Norman Feske <norman.feske@...1...> wrote:
Hi Jörg,
first, your understanding of the different components is correct. Thank for taking the time for the thorough investigation of the existing components.
What I see between the cli_monitor/launcher and launchpad approach, that the configuration of the subsystems is different. But is there even more differences ?
The launchpad and qt_launchpad were intended as quick demos, not a serious solutions. In fact, launchpad was one of the very first programs developed for Genode - just to showcase the dynamic creation/destruction of subsystems. CLI monitor is similar, but uses a textual interface.
Design-wise the disadvantage of the launchpad as well as CLI monitor (and even more so qt_launchpad) is the fact that the component is potentially complex (as it implements a UI) while being part of the TCB of each launched subsystems.
With the launcher, I explored the idea to separate the complex parts (i.e., the widget rendering) into a separate sandboxed component called menu view. This helps to significantly reduce the complexity of the launcher - and thereby the TCB of the launched subsystems. So software-design wise, the launcher is much more advanced.
However, all these solutions share one problem that holds them back for general-purpose scenarios: They can start and stop subsystems but do not allow the user to define any interaction between the subsystems. It is not possible to dynamically launch a new service to be used by another subsystem.
To overcome this limitation, I am currently investigating the use of a dynamic init instance as the host for subsystems. With this approach, the launcher merely generates configurations for the init instance and consumes reports generated by init. It goes without saying that the lessons learned from the menu-view approach can be applied in this scenario as well. Please refer to a previous discussion [1] that we had on the list earlier this year.
[1] https://sourceforge.net/p/genode/mailman/message/35787965/
May I asked, what are the future plans of these "launchers"? Cli_monitor is in the OS repo and sounds like the API is what I should
use
to implement some "own" launchers. Launch app in gems repo, as what I understand from the email from Norman (see Friday, 4. August 2017 11:56:38 CEST from Norman Feske) this is outdated. Launchpad is in the demo repo and what I see in the git commit history, this is also the older approach to "launch" subsystems in genode, or?
The launchpad is meant to stay as a quick-start example for exploring Genode. However, the other launchers will eventually be replaced by the dynamic init approach, once it is there.
Here some background information, why I asked... I will use genode as a desktop system and for such a use case I need something to launch applications. I notice, that genode is currently implementing a package management and it sounds like for me, in the future I need a launcher that are flexible enough to start an application after I installed a package in genode :-) ... but step by step.
I am currently working in the same direction. If you are interested to follow my work, you may have a look at the corresponding branch [2]. In my opinion, two particular interesting bits are the depot-query scenario [3] and the dynamic-drivers subsystem [4]. Both facilitate the dynamic init approach.
[2] https://github.com/nfeske/genode/commits/nitro [3] https://github.com/nfeske/genode/commit/cc2b7935fbdbe0bd3b342495bcc9d1 da36e61f55 [4] https://github.com/nfeske/genode/commit/8046e4d2be50c183ad7ad3ba6e68d6 e0a7240644
But as a word of caution, this is work in progress. ;-)
Cheers Norman
-- Dr.-Ing. Norman Feske Genode Labs
http://www.genode-labs.com · http://genode.org
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main
On 08/18/2017 07:14 PM, Nobody III wrote:
Here's my work on a desktop environment for Genode: https://github.com/NobodyIII/genode-desktop-environment
So far, I have a working file manager and text editor. Feel free to use and modify my code. ...
May be the
https://github.com/rumpkernel/rumprun-packages/
might be of interest to userland developers.
Hi Norman,
thanks for your quick answer and also for the detail information.
Last days, I tried quickly to get an overview in my less spare time :-) of the many information (read the discussion and "Package management on Genode" documentation, study the source code etc) so please take into account when I misunderstood something.
Am Freitag, 18. August 2017 15:58:49 CEST schrieb Norman Feske:
However, all these solutions share one problem that holds them back for general-purpose scenarios: They can start and stop subsystems but do not allow the user to define any interaction between the subsystems. It is not possible to dynamically launch a new service to be used by another subsystem.
Hm, ok. I was expected it was... I saw the "Genode as Desktop OS" presentation from you at FOSDEM 2016. You show there the Turmvilla scenario and demonstrate via the CLI_monitor (I think it was the CLI_monitor... the video resolution and the tone has not the best quality) how you can dynamically change the configuration of for example the screen resolution. Ok to launch dynamically a new service you didn't show there.
To overcome this limitation, I am currently investigating the use of a dynamic init instance as the host for subsystems. With this approach, the launcher merely generates configurations for the init instance and consumes reports generated by init.
With "init instance" and "by init" ... you mean here the "dynamic init instance" ?
It goes without saying that the lessons learned from the menu-view approach can be applied in this scenario as well. Please refer to a previous discussion [1] that we had on the list earlier this year.
[1] https://sourceforge.net/p/genode/mailman/message/35787965/
I have to admit, that I partially understand this discussion but not in detail. I think I need more experience in Genode :-)
Here some background information, why I asked... I will use genode as a desktop system and for such a use case I need something to launch applications. I notice, that genode is currently implementing a package management and it sounds like for me, in the future I need a launcher that are flexible enough to start an application after I installed a package in genode :-) ... but step by step.
I am currently working in the same direction. If you are interested to follow my work, you may have a look at the corresponding branch [2]. In my opinion, two particular interesting bits are the depot-query scenario [3] and the dynamic-drivers subsystem [4]. Both facilitate the dynamic init approach.
[2] https://github.com/nfeske/genode/commits/nitro [3] https://github.com/nfeske/genode/commit/cc2b7935fbdbe0bd3b342495bcc9d1da36e6... [4] https://github.com/nfeske/genode/commit/8046e4d2be50c183ad7ad3ba6e68d6e0a724...
I investigate in the source that depot_query component is query the package depot directory. When they found a package with "runtime" file they announced this via "blueprint" label to depot_deploy component. They generates the config for the "dynamic init" instance (here subinit) and the subinit is starting the "test-fs_report-2017-08-23" process (or subsystem?!) with the generates config from depot_deploy component. Is my investigation/conclusion correct ?
This means the "subinit" in the depot-query scenario should be the "dynamic init" instance what we are talking about, or is this "test-fs_report-2017-08-23" the "dynamic init" instance ?
This means we have three running init instances, yes ? [init -> subinit -> test-fs_report-2017-08-23 -> ...] But only the "root" init get the full ressoruces from the "core" or?
I didn't get yet, the "full picture", how the launcer can be involved yet in the depot-deploy scenario, because depot_query start via depot_deploy the process (or subsystem?!) immediately or the depot_query can also be the launcher ?
But as a word of caution, this is work in progress. ;-)
Ok, I see but it looks create for me. Also the dynamic-driver is a create solution I think. I decided to stop my work/idea for the "launcher stuff" now and waiting/try to use the depot-query solution :-).
Maybe I found a little "resource/performance" issue.... I noticed that depot_deploy announce an "init.config" three times, but I think only the last one with the "pkg" config is important. The first and second init.config only announce the "static" content configuration. Maybe we should prevent this and verify via an if statement if the "blueprint xml" has a "pkg" xml node. I test it and for me it looks like it works.... but this is only a hint :-), maybe I didn't see some "side effects".
Cheers Jörg
Hello Jörg,
To overcome this limitation, I am currently investigating the use of a dynamic init instance as the host for subsystems. With this approach, the launcher merely generates configurations for the init instance and consumes reports generated by init.
With "init instance" and "by init" ... you mean here the "dynamic init instance" ?
yes. Actually, there is no difference between "init" and "dynamic init". It is the same program. The only difference is that the latter gets its configuration from a ROM service that produces new versions of the configuration at runtime. The first init is always static because it obtains its configuration from core's ROM service (as a boot module).
I investigate in the source that depot_query component is query the package depot directory. When they found a package with "runtime" file they announced this via "blueprint" label to depot_deploy component. They generates the config for the "dynamic init" instance (here subinit) and the subinit is starting the "test-fs_report-2017-08-23" process (or subsystem?!) with the generates config from depot_deploy component. Is my investigation/conclusion correct ?
Yes.
This means the "subinit" in the depot-query scenario should be the "dynamic init" instance what we are talking about, or is this "test-fs_report-2017-08-23" the "dynamic init" instance ?
This means we have three running init instances, yes ? [init -> subinit -> test-fs_report-2017-08-23 -> ...]
You understood everything correctly.
But only the "root" init get the full ressoruces from the "core" or?
Yes. In the run depot_deploy.run script, most memory remains unused. But by assigning an overly large quantum of RAM to 'subinit', init will transfer the maximum amount possible to 'subinit'. From this point on, all the memory is at the disposal of the dynamic system.
I didn't get yet, the "full picture", how the launcer can be involved yet in the depot-deploy scenario, because depot_query start via depot_deploy the process (or subsystem?!) immediately or the depot_query can also be the launcher ?
There are two topics, which are somewhat intertwined. (1) creating subsystems out of packages stored in a depot, and (2) an interactive front end for generating init configurations. The depot-deploy scenario showcases (1) only and leaves (2) out of the picture. Conversely, one might construct a scenario that does not use the depot but interactively spawns subsystems by the means of updating init configurations at runtime.
To follow the topic (2), one fun experiment would be to let the user edit the configuration of a dynamic init by using Vim (running in a Noux runtime). The configuration for the dynamic init would come from a 'config' file stored in a 'ram_fs'. The RAM file system is mounted in the Noux instance. Its content is also made available in the form of ROM modules via the 'fs_rom' service. By letting the dynamic init obtain its 'config' ROM from this 'fs_rom' service, we feed it the content of the file stored in the RAM fs. For starting a new subsystem, the user would add a '<start>' node and save the file. The saving of the file results in a ROM-module update, which triggers the dynamic init instance to process the modified version.
The visibility of the Noux instance could be toggled via a global key (like the X-ray mode is handled in the demo.run script). By implementing this simple scenario, you would actually get a quite powerful "shell". ;-)
With this scenario in place, I think the use of the depot will appear as just another step.
Maybe I found a little "resource/performance" issue.... I noticed that depot_deploy announce an "init.config" three times, but I think only the last one with the "pkg" config is important. The first and second init.config only announce the "static" content configuration. Maybe we should prevent this and verify via an if statement if the "blueprint xml" has a "pkg" xml node. I test it and for me it looks like it works.... but this is only a hint :-), maybe I didn't see some "side effects".
In principle, I agree. However, for now, I deliberately leave such artifacts there to put stress on components that consume intermediate states. It is important that such components can gracefully deal with many (and sometimes arguably silly) state changes without stumbling.
Cheers Norman
Hello Norman,
Am Montag, 28. August 2017 12:05:38 CEST schrieb Norman Feske:
To overcome this limitation, I am currently investigating the use of a dynamic init instance as the host for subsystems. With this approach, the launcher merely generates configurations for the init instance and consumes reports generated by init.
With "init instance" and "by init" ... you mean here the "dynamic init instance" ?
yes. Actually, there is no difference between "init" and "dynamic init". It is the same program. The only difference is that the latter gets its configuration from a ROM service that produces new versions of the configuration at runtime. The first init is always static because it obtains its configuration from core's ROM service (as a boot module).
Ok, I read "4.6.3 Dynamic component reconfiguration at runtime" in the Genode documentation, but I forgot that the "root config" from core is read only.
I didn't get yet, the "full picture", how the launcer can be involved yet in the depot-deploy scenario, because depot_query start via depot_deploy the process (or subsystem?!) immediately or the depot_query can also be the launcher ?
There are two topics, which are somewhat intertwined. (1) creating subsystems out of packages stored in a depot, and (2) an interactive front end for generating init configurations. The depot-deploy scenario showcases (1) only and leaves (2) out of the picture. Conversely, one might construct a scenario that does not use the depot but interactively spawns subsystems by the means of updating init configurations at runtime.
To follow the topic (2), one fun experiment would be to let the user edit the configuration of a dynamic init by using Vim (running in a Noux runtime). The configuration for the dynamic init would come from a 'config' file stored in a 'ram_fs'. The RAM file system is mounted in the Noux instance. Its content is also made available in the form of ROM modules via the 'fs_rom' service. By letting the dynamic init obtain its 'config' ROM from this 'fs_rom' service, we feed it the content of the file stored in the RAM fs. For starting a new subsystem, the user would add a '<start>' node and save the file. The saving of the file results in a ROM-module update, which triggers the dynamic init instance to process the modified version.
The visibility of the Noux instance could be toggled via a global key (like the X-ray mode is handled in the demo.run script). By implementing this simple scenario, you would actually get a quite powerful "shell". ;-)
That's sounds like a challenge for me :-) ...
Ok, now I get the "picture". Thanks for your time to explain in detail about the problems and solutions.
Cheers Jörg
-
Jörg-Christian Böhme -
Martin Vahi -
Nobody III -
Norman Feske