Hello everyone,
in his most recent posting, Credric already noted that now is the time of year again to reflect on our achievements of this year and to anticipate topics to pursue next year. To keep up with this fine tradition, let me kick off our road-map discussion with my personal perspective.
Reflections of 2023
As indicated by the overarching theme of the last roadmap "Rocking the platforms we support!", Genode's four releases of 2023 had a strong focus on low-level platform work. This has been especially visible on modern PC platforms like the Gen12 Framework laptop I have under my fingertips right now. At the beginning of 2023, Sculpt OS was in principle working on this machine, but with compromises that spoiled the user experience: fan noise, an erratic touchpad (using the firmware's PS/2 emulation), Fn key having no effect, strange issues when re-plugging an external display, and no indication of the battery state. In the meantime, not only are all these rough edges gone, but we even gained the ability to exercise precise control over the machine' performance/frequency/temperature/power characteristics using an interactive GUI. I think it's fair so say that Genode advanced far beyond the state of "working" and has entered the territory of "rocking".
I wish to name four further personal highlights of the year:
First, we got the mobile version of Sculpt OS into the hands of a pilot group of users who provided instructive feedback to us. From my perspective, the system-update mechanism that I created for this purpose turned out to be an almost pivotal point in the evolution of Sculpt OS because it reduces the effort and risk of test-driving experimental versions to almost zero. It was a pleasure to see how e.g., Johannes leveraged this new way of gathering feedback for his IOMMU line of work. Providing system images for testing has become a second nature.
Second, the road map for 2023 envisioned Sculpt OS running on our custom base-hw kernel on the PC. We identified DMA protection and virtualization support as the two remaining showstoppers. With much excitement, I followed how both of these deeply technical topics got covered over the course of the year.
Third, Goa finally emerged from a personal project of mine to an official Genode project led by Johannes. I'm stunned how much the project benefited from this change. All of the remaining backlog of my vague plans - I'm thinking of the index-project support or bash completion - got eventually realized in a way true to the spirit of the project. The fate of the Goa tool makes me immensely happy.
Fourth, during the first half of the year, I found myself intensively working on Genode's new debug-monitor component, pursuing the idea to implement a debugging instrument as a specialized version of init augmented with the GDB protocol. This engagement was pretty much motivated by a customer. The result of my initial work then seamlessly transitioned into the hands of Christian Prochaska who did a marvelous job with steadily advancing this line of work towards our joint vision of on-target debugging on Sculpt OS. The technical feats notwithstanding, I found the frictionless way of collaborating a pure joy.
Besides the highlights above, one topic close to my heart was the creation of the dialog API that I designed as necessity to make the code of Sculpt's administrative user interface maintainable and easy to extend in the longer run.
Plans for 2024
After concentrating so intensively on topics below the surface, I now long for reaping user-visible rewards. Speaking of the dialog API just mentioned, I see potential in using this new infrastructure for Genode-native applications and immediately think of the file manager that I already wanted to tackle this year. But I also have plenty of ideas to make Sculpt OS more user friendly. What about presenting the README files of software packages directly in the GUI? Making the component graph scrollable? Allowing the user to select an arbitrary directory as a file system to a component? Buttons for saving the current deployment or the settings?
Beyond Sculpt's administrative user interface, I'd also like to attend the GUI stack. I think of refining the GUI-session interface to remove tearing artifacts, to better support desktop-UI-paradigms like drag'n'drop, and to explore the opportunity of using nitpicker's mechanisms at the application-level, not only at the window-composition level.
Hence, I'd condense my ambition for the next year to "Sculpt OS usability".
Device-wise, I'm going to continue my engagement with the PinePhone and look forward to the upcoming MNT PocketReform laptop.
Above I presented my personal view. How about your's? I would very much appreciate you sharing your feedback, ideas, concerns, and plans regarding Genode.
How are your interests aligned with the perspective shared above?
Do you see specific pain points that deserve the attention of Genode's core developer team?
What is your perspective on Genode's past year's accomplishments?
Can you share your ambitions or even concrete plans?
How and where would you like to see Genode at the end of 2024?
Cheers Norman
On Thu, 21 Dec 2023 at 16:58, Norman Feske norman.feske@genode-labs.com wrote:
Hello everyone,
in his most recent posting, Credric already noted that now is the time of year again to reflect on our achievements of this year and to anticipate topics to pursue next year. To keep up with this fine tradition, let me kick off our road-map discussion with my personal perspective.
Reflections of 2023
As indicated by the overarching theme of the last roadmap "Rocking the platforms we support!", Genode's four releases of 2023 had a strong focus on low-level platform work. This has been especially visible on modern PC platforms like the Gen12 Framework laptop I have under my fingertips right now. At the beginning of 2023, Sculpt OS was in principle working on this machine, but with compromises that spoiled the user experience: fan noise, an erratic touchpad (using the firmware's PS/2 emulation), Fn key having no effect, strange issues when re-plugging an external display, and no indication of the battery state. In the meantime, not only are all these rough edges gone, but we even gained the ability to exercise precise control over the machine' performance/frequency/temperature/power characteristics using an interactive GUI. I think it's fair so say that Genode advanced far beyond the state of "working" and has entered the territory of "rocking".
I wish to name four further personal highlights of the year:
First, we got the mobile version of Sculpt OS into the hands of a pilot group of users who provided instructive feedback to us. From my perspective, the system-update mechanism that I created for this purpose turned out to be an almost pivotal point in the evolution of Sculpt OS because it reduces the effort and risk of test-driving experimental versions to almost zero. It was a pleasure to see how e.g., Johannes leveraged this new way of gathering feedback for his IOMMU line of work. Providing system images for testing has become a second nature.
Second, the road map for 2023 envisioned Sculpt OS running on our custom base-hw kernel on the PC. We identified DMA protection and virtualization support as the two remaining showstoppers. With much excitement, I followed how both of these deeply technical topics got covered over the course of the year.
Third, Goa finally emerged from a personal project of mine to an official Genode project led by Johannes. I'm stunned how much the project benefited from this change. All of the remaining backlog of my vague plans - I'm thinking of the index-project support or bash completion - got eventually realized in a way true to the spirit of the project. The fate of the Goa tool makes me immensely happy.
Fourth, during the first half of the year, I found myself intensively working on Genode's new debug-monitor component, pursuing the idea to implement a debugging instrument as a specialized version of init augmented with the GDB protocol. This engagement was pretty much motivated by a customer. The result of my initial work then seamlessly transitioned into the hands of Christian Prochaska who did a marvelous job with steadily advancing this line of work towards our joint vision of on-target debugging on Sculpt OS. The technical feats notwithstanding, I found the frictionless way of collaborating a pure joy.
Besides the highlights above, one topic close to my heart was the creation of the dialog API that I designed as necessity to make the code of Sculpt's administrative user interface maintainable and easy to extend in the longer run.
Plans for 2024
After concentrating so intensively on topics below the surface, I now long for reaping user-visible rewards. Speaking of the dialog API just mentioned, I see potential in using this new infrastructure for Genode-native applications and immediately think of the file manager that I already wanted to tackle this year. But I also have plenty of ideas to make Sculpt OS more user friendly. What about presenting the README files of software packages directly in the GUI? Making the component graph scrollable? Allowing the user to select an arbitrary directory as a file system to a component? Buttons for saving the current deployment or the settings?
Beyond Sculpt's administrative user interface, I'd also like to attend the GUI stack. I think of refining the GUI-session interface to remove tearing artifacts, to better support desktop-UI-paradigms like drag'n'drop, and to explore the opportunity of using nitpicker's mechanisms at the application-level, not only at the window-composition level.
Hence, I'd condense my ambition for the next year to "Sculpt OS usability".
Device-wise, I'm going to continue my engagement with the PinePhone and look forward to the upcoming MNT PocketReform laptop.
Above I presented my personal view. How about your's? I would very much appreciate you sharing your feedback, ideas, concerns, and plans regarding Genode.
How are your interests aligned with the perspective shared above?
Do you see specific pain points that deserve the attention of Genode's core developer team?
What is your perspective on Genode's past year's accomplishments?
Can you share your ambitions or even concrete plans?
How and where would you like to see Genode at the end of 2024?
I started out 2023 very ambitious, porting Genode to rockchip 3528. For various reasons I needed to work on other projects. My plan for 2024 is to add RK3399 as a sub target. That means Pinebook Pro and Pinephone Pro. I guess that those two targets are wanted.
I also would like to see a layer for USB controllers like EHCI and XHCI. I would be most happy if someone can walk me thru the pc usb stack. I would like to cut lose EHCI from it and use that in my ports. I believe that this is better than using linux driver all up to the actual stack.
I wish you all happy holidays!
Michael Grunditz
Hi Michael,
thank you for your response.
I started out 2023 very ambitious, porting Genode to rockchip 3528. For various reasons I needed to work on other projects. My plan for 2024 is to add RK3399 as a sub target. That means Pinebook Pro and Pinephone Pro. I guess that those two targets are wanted.
I guess so too, as I see the latter being picked by the open-source hardware community, i.e., Lukas Hartmann in his recent posting [1].
[1] https://mastodon.social/@mntmn/111649131139949948
I also would like to see a layer for USB controllers like EHCI and XHCI. I would be most happy if someone can walk me thru the pc usb stack. I would like to cut lose EHCI from it and use that in my ports. I believe that this is better than using linux driver all up to the actual stack.
I'd be curious how your goals are aligned with Stefan's ongoing redesign of Genode's USB infrastructure (explained at [2]) and his further plans down the road. I'm hopeful that you might benefit from his line of work.
[2] https://github.com/genodelabs/genode/issues/5021
Cheers and happy holidays!
Norman
In message ab8dc2c1-cf33-4069-bc42-d63439f2aaa4@genode-labs.com Norman Feske norman.feske@genode-labs.com wrote:
Hi Michael,
thank you for your response.
Replying with real email client :-)
I started out 2023 very ambitious, porting Genode to rockchip 3528.
Err should have been RK3588 :-)
For various reasons I needed to work on other projects. My plan for 2024 is to add RK3399 as a sub target. That means Pinebook Pro and Pinephone Pro. I guess that those two targets are wanted.
I guess so too, as I see the latter being picked by the open-source hardware community, i.e., Lukas Hartmann in his recent posting [1].
A much needed upgrade for the MNT Reform laptop. i.MX8M is a bit slow for use in a consumer device for general computing.
I also would like to see a layer for USB controllers like EHCI and XHCI. I would be most happy if someone can walk me thru the pc usb stack. I would like to cut lose EHCI from it and use that in my ports. I believe that this is better than using linux driver all up to the actual stack.
I'd be curious how your goals are aligned with Stefan's ongoing redesign of Genode's USB infrastructure (explained at [2]) and his further plans down the road. I'm hopeful that you might benefit from his line of work.
I don't know. The plans linked is above what I want to do, if I understand it right. I come from RISC OS lands , and in our rom we have EHCI,OHCI and XHCI in seperate controller drivers. This all connects to what we call USBDriver which is the main USB stack.
In Genode I would like to have something similar. It makes it very easy to bring up on new platforms. In my case, Rockchip, I start USB ( power/phy) from U-Boot. After that I only need to assign address and irq to the controller driver. If Genode doesn't have native controller drivers at all (in pc port) ( sorry haven't checked yet), I could try to bring in controler drivers from BSD.
Cheers and happy holidays!
Norman
Happy end of 2023!
Michael
Hello,
On Fri, Dec 29, 2023 at 12:53:37PM +0100, Michael Grunditz wrote:
I also would like to see a layer for USB controllers like EHCI and XHCI. I would be most happy if someone can walk me thru the pc usb stack. I would like to cut lose EHCI from it and use that in my ports. I believe that this is better than using linux driver all up to the actual stack.
I'd be curious how your goals are aligned with Stefan's ongoing redesign of Genode's USB infrastructure (explained at [2]) and his further plans down the road. I'm hopeful that you might benefit from his line of work.
I don't know. The plans linked is above what I want to do, if I understand it right. I come from RISC OS lands , and in our rom we have EHCI,OHCI and XHCI in seperate controller drivers. This all connects to what we call USBDriver which is the main USB stack.
In Genode I would like to have something similar. It makes it very easy to bring up on new platforms. In my case, Rockchip, I start USB ( power/phy) from U-Boot. After that I only need to assign address and irq to the controller driver. If Genode doesn't have native controller drivers at all (in pc port) ( sorry haven't checked yet), I could try to bring in controler drivers from BSD.
I think using OHCI/EHCI/XHCI controller drivers written from scratch, or ported from another OS then Linux is an orthogonal topic to the recent attempts of re-designing the USB session. Driver implementation can be done either by using the current session API or the envisioned one. Anyway it might be good to target the newer session API, to limit the overall efforts.
I was thinking of creating an USB multiplexer, independent of the actual controller that serves either USB controller drivers acting as clients as well as USB client device drivers. Thereby, we could survive restarting USB host controller drivers in case of malfunctions, which would increase robustness for all driver variants. I'm not sure whether we shall schedule this vaque idea already for the roadmap 2024, but I'm open to it.
Regards Stefan
Hello Norman and all fellow Genodians
In this I only speak of my private achievements / plans, the company I'm working for, will write a separate reply.
On 21.12.23 16:57, Norman Feske wrote:
Do you see specific pain points that deserve the attention of Genode's core developer team?
For my private work I do see none.
What is your perspective on Genode's past year's accomplishments?
When testing out my new Toy (MinisForum UM790 Pro, AMD CPU with an integrated GPU), I was pleasantly surprised, that Sculpt did run out of the box on it. I did no deeper analysis what drivers it did use, but I was able to run the Falcon browser preset.
Can you share your ambitions or even concrete plans?
Reflections on 2023
I did some initial testing with the first image of Sculpt for the Pinephone. But as my setup at home doesn't want to connect the ADB to my development virtual machine, I never came around to answer Normans questions regarding my results, sorry for that. Hopefully I will be able to figure this out at the 2024 Hack'n'Hike.
I did some progress with my NIC-driver for the Intel-IGC NIC driver. It did start, but no communication yet. Around the Sculpt 23.10 release I started the integration of IGC support in to the new nic_drv for the PC platform. This in its current state sends out DHCP requests, but for some reason the responses aren't sent back to the `nic_router`.
My other plans were to port Neovim using Goa and porting the frame-buffer driver for AMD GPUs. On these I didn't really work as my time was used up otherwise.
Plans for 2024
For this year I would like to do the following: - finish the integration of the IGC driver and provide it upstream - Re-start the development of the framebuffer driver - tryout how to bring multi monitor support for Sculpt
How and where would you like to see Genode at the end of 2024?
If the GUI stack improvements would bring multi monitor support I would be very happy.
Cheers and all the best, Pirmin
Hi Pirmin,
thank you very much for joining our road-map discussion.
When testing out my new Toy (MinisForum UM790 Pro, AMD CPU with an integrated GPU), I was pleasantly surprised, that Sculpt did run out of the box on it. I did no deeper analysis what drivers it did use, but I was able to run the Falcon browser preset.
Given that we don't stress Sculpt on AMD, this is indeed pleasantly surprising. Thanks for sharing this happy anecdote. :-)
Can you share your ambitions or even concrete plans?
Reflections on 2023
I did some initial testing with the first image of Sculpt for the Pinephone. But as my setup at home doesn't want to connect the ADB to my development virtual machine, I never came around to answer Normans questions regarding my results, sorry for that. Hopefully I will be able to figure this out at the 2024 Hack'n'Hike.
Can't wait to stick our heads together at the Hack'n'Hike!
I did some progress with my NIC-driver for the Intel-IGC NIC driver. It did start, but no communication yet. Around the Sculpt 23.10 release I started the integration of IGC support in to the new nic_drv for the PC platform. This in its current state sends out DHCP requests, but for some reason the responses aren't sent back to the `nic_router`.
My other plans were to port Neovim using Goa and porting the frame-buffer driver for AMD GPUs. On these I didn't really work as my time was used up otherwise.
Even if you haven't put Goa into practice for the actual Neovim porting, your contributions to Goa - in particular the shared-library topic - certainly played in important role for the tool's evolution in the past year.
Plans for 2024
For this year I would like to do the following: - finish the integration of the IGC driver and provide it upstream - Re-start the development of the framebuffer driver - tryout how to bring multi monitor support for Sculpt
How and where would you like to see Genode at the end of 2024?
If the GUI stack improvements would bring multi monitor support I would be very happy.
You are not alone with this sentiment about multi-monitor support. So we should definitely aim to get it covered in 2024. It will be a challenging topic because it touches the whole GUI stack (drivers, nitpicker, wm, decorators, window layouter, GUI clients). But we could hardly speak of "Sculpt OS usability" in 2024 with a straight face without having a solid implementation of this feature.
Cheers Norman
Howdy Norman, all, A little screenshot to warm up, then my perspectives of daily Genode usage.
Mail: Here's a little screenshot of a quick port of the (Be)Mail GUI. https://tts-genode.neocities.org/tts-img/2023-12_BeMail_nano3d_ftppositive.p... This one specific app is probably not a most promising prospect for Genode/Sculpt, due to limited feature set compared to big names like ThunderBird and (more importantly) due to its requiring a custom version of Genode with xattr support, plus I didn't port its network back-end yet. But it could serve as inspiration for related projects, who knows. Doing that kind of small-to-medium size Be ports to Genode is quite easy now that most pieces of the Be puzzle are in place. Heck, after learning about the new debug-monitor in 23.08(?) I had to resist the urge to slap this GUI on top of it: https://www.haiku-os.org/files/images/debugger_return_value.png It would probably be a couple orders of magnitude more work than porting (Be)Mail... But knowing myself, I'll probably port it eventually, though it will have to wait a year or three. I rarely use that debugger/GUI in Be and also on Genode I tend to resort to tracing analysis. But sometimes it can make a big difference to dump an app into a debugger, e.g. if some threads are deadlocked, you save yourself many iterations of refining tracing to find out which thread stops where and when. Anyway, as I port new stuff in the future new screenshots will appear (always better than the wall'o'text I tend to post! /grin)
Stations: This coming year 2024 will be pivotal for us at tunetracker and our stations. Nothing that's of direct concern to the genodian community, but will mention it for the curious. After the elapsed time and the promises we made to our stations of providing them with a platform that doesn't crash several times per day, one of twho things could happen : stations could tell us they're still interested in our software and want to try it out now that it's getting to alpha stage on a rock solid platform (Genode!), or they'll tell us they've moved on to other pastures (free-as-in-beer/speech radio software running on Linux etc is gaining market shares against commercial concerns like ours : why pay for software suite X if you can get Y for free and it's "good enough" ?). In the latter case that's fine, sort of. Genode will still have a lot of staying power in my computing life as a 'daily driver', which has huge significance to me.
Past years "push" versus future "daily driver": My take on this kind of mirrors Norman's remarks : those past years my feedback often revolved around 'wishes' (what I needed/wished from Genode, and progress I must make), now in my case it's (the beginning of) harvest time, very happy about that. Now that SculptOS-style depot packages are supported in H/on/Genode, including Falkon, I've started booting into H/Genode just to browse the news in Falkon, and will be striving to extend use-cases next year. For instance it might be possible to migrate my asciidoc writing to Genode, at least the part of asciidoc that depends on Python, if not the half that depends on Ruby. Will have to reboot for that one (or I could try out the Seoul and VirtualBox packages ? Sounds like a plan!). And I'll keep porting stuff from the Be world that I need to be productive in Genode too. This is all a 'sea change' for me, for years I've been growing increasingly confident about Genode's capabilities as I became more familiar with it, and now I'm in new territory, starting to actually do the stuff I wanted to do in Genode. One way to cut this pie is to look at my daily usage as 1) programming-related and 2) all the rest. Short term, this coming year I'll be curious to see if anything in set #2 resists my migration efforts. I'm not looking at #1 yet as that's a perspective for the future, being very demanding in terms of FS usage. But Genode currently looks like it could handle an awful lot of part #2 -- provided I port the necessary software of course.
Future/miscellaneous/"stream of consciousness": - now that things are calming down at my end I'm going to resume my DFS (depth first search) 'walk' of the Genode source tree started a few years ago, and interrupted much too quickly/early ^^. I want to create a mental map of it, pick up things of interest. This also ties in with my wish to migrate my computing to Genode. If it's a given that Genode has 'all the right stuff' then it'd be a matter of transferring all my use-cases to Genode one by one, working on each case when I hit bumps. And that'll be easier if the required features/hooks/APIs inside Genode present themselves to me sequentially ("for each Genode bit: how can I use it ?"), rather than go the other way around ("foreach thing I need: try to hunt for it in the documentation or in the source code"). You guys are super diligent and ruthless in removing "dead wood" from the tree (oooh the mixed metaphor), so "walking the (christmas *g*) source tree" always had lots of appeal to me. - if Goa is increasingly used to add new ports to genode-world or to other decentralized repos, I probably won't be the only one looking forward to it and downloading packages to try them out! Packages need to be "advertised" one way or another though : when a a new package is mentionned in Genodians I take notes, when it goes through the GitHub genode-world timeline I takes notes, but if in neither of these, I'm not going to be aware of its existence, so something to keep in mind. - re drag'n'drop, curious to see what the Genode approoach would be. It might save me the need to roll my own ad-hoc implementation in H/Ge. - same thing with (if I understand correctly) creating a "nitpicker_primitives.lib.so" that can be re-used by other components : color me curious and interested.
FS optims: Maybe the one immediate item remaining on my Genode wish-list will be vfs related: Even if I'm in "reaping the fruits of past work" mode, there might still be one type of ticket I might file on github yet, related to FS xattr performance. Reading attributes in H/o-g is quite slow, maybe a couple dozen reads per sec. It's too soon to tell where the optimization potential is located though. Given the nature of the current xattr implementation (lots of back-and-forth between fs client and vfs server to process ad-hoc/virtual paths like /Station/Music/.PinkFloyd/.attributes etc) it comes to mind as a primary suspect, but gotta be careful to pinpoint accurately which part of which component is too slow and in need of optimization (on my side or on Genode's). It's easy enough to get lost in the weeds even for skilled devs, even more so for me 8-). My trip down that rabbit hole will probably begin in a few weeks or months, once the other ToDos are reasonably complete.
So to sum up, december was a solid start for my personal "daily driver" goal, the moment of truth's coming for my commercial stuff as we're going to probe/survey customers, and I might start a ticket or discussion re. fs optimization but I can't say yet what the specifics will be.
Here's to a productive new year to all genodians,
Cedric
Hi Cedric,
thanks for sharing your unique perspective. It's always a joy getting new insights into your experiences and ambitions. I'm grateful seeing that you hold our work in such high regard.
On 2023-12-22 12:03, ttcoder@netcourrier.com wrote:
Howdy Norman, all, A little screenshot to warm up, then my perspectives of daily Genode usage.
Mail: Here's a little screenshot of a quick port of the (Be)Mail GUI. https://tts-genode.neocities.org/tts-img/2023-12_BeMail_nano3d_ftppositive.p... This one specific app is probably not a most promising prospect for Genode/Sculpt, due to limited feature set compared to big names like ThunderBird and (more importantly) due to its requiring a custom version of Genode with xattr support, plus I didn't port its network back-end yet. But it could serve as inspiration for related projects, who knows. Doing that kind of small-to-medium size Be ports to Genode is quite easy now that most pieces of the Be puzzle are in place.
The screenshot nicely shows how far your work has come by now. Speaking of the porting of Be applications, I wonder how Goa could be applicable in these cases? Should we succeed to eventually bridge the gap between your customized version of Genode (xattr) and the regular Sculpt OS while fostering the use of Goa for such porting efforts, developers who enjoy casual porting work may become able to join the fun, building upon your H/o/G layer as a library.
Heck, after learning about the new debug-monitor in 23.08(?) I had to resist the urge to slap this GUI on top of it: https://www.haiku-os.org/files/images/debugger_return_value.png It would probably be a couple orders of magnitude more work than porting (Be)Mail... But knowing myself, I'll probably port it eventually, though it will have to wait a year or three. I rarely use that debugger/GUI in Be and also on Genode I tend to resort to tracing analysis. But sometimes it can make a big difference to dump an app into a debugger, e.g. if some threads are deadlocked, you save yourself many iterations of refining tracing to find out which thread stops where and when. Anyway, as I port new stuff in the future new screenshots will appear (always better than the wall'o'text I tend to post! /grin)
It is so nice seeing you drawing connections between Haiku and our working topics, even for the debug monitor.
Once we have fully realized our vision of on-target debugging via GDB, you will hopefully be able to follow the lines of this working example. Now is too early. But as you cheerfully noted, we aren't in a hurry.
Stations: This coming year 2024 will be pivotal for us at tunetracker and our stations. Nothing that's of direct concern to the genodian community, but will mention it for the curious. After the elapsed time and the promises we made to our stations of providing them with a platform that doesn't crash several times per day, one of twho things could happen : stations could tell us they're still interested in our software and want to try it out now that it's getting to alpha stage on a rock solid platform (Genode!), or they'll tell us they've moved on to other pastures (free-as-in-beer/speech radio software running on Linux etc is gaining market shares against commercial concerns like ours : why pay for software suite X if you can get Y for free and it's "good enough" ?). In the latter case that's fine, sort of. Genode will still have a lot of staying power in my computing life as a 'daily driver', which has huge significance to me.
I sense your mixed feelings from this paragraph, and I'm crossing fingers.
Past years "push" versus future "daily driver": My take on this kind of mirrors Norman's remarks : those past years my feedback often revolved around 'wishes' (what I needed/wished from Genode, and progress I must make), now in my case it's (the beginning of) harvest time, very happy about that. Now that SculptOS-style depot packages are supported in H/on/Genode, including Falkon, I've started booting into H/Genode just to browse the news in Falkon, and will be striving to extend use-cases next year. For instance it might be possible to migrate my asciidoc writing to Genode, at least the part of asciidoc that depends on Python, if not the half that depends on Ruby. Will have to reboot for that one (or I could try out the Seoul and VirtualBox packages ? Sounds like a plan!). And I'll keep porting stuff from the Be world that I need to be productive in Genode too. This is all a 'sea change' for me, for years I've been growing increasingly confident about Genode's capabilities as I became more familiar with it, and now I'm in new territory, starting to actually do the stuff I wanted to do in Genode. One way to cut this pie is to look at my daily usage as 1) programming-related and 2) all the rest. Short term, this coming year I'll be curious to see if anything in set #2 resists my migration efforts. I'm not looking at #1 yet as that's a perspective for the future, being very demanding in terms of FS usage. But Genode currently looks like it could handle an awful lot of part #2 -- provided I port the necessary software of course.
Wow! The stars seem to be aligning. :-) With that I mean that my personal ambition to streamline the use of Sculpt OS on the desktop will hopefully benefit your ambition. Since you have integrated Genode's depot/deploy mechanism now, you can finally draw on the full potential of our community (given the increasing arsenal of delightful packages maintained by individuals). Vice versa, I'd be thrilled to be able to fetch an index of nice Be applications from your depot one day.
- if Goa is increasingly used to add new ports to genode-world or to other decentralized repos, I probably won't be the only one looking forward to it and downloading packages to try them out! Packages need to be "advertised" one way or another though : when a a new package is mentionned in Genodians I take notes, when it goes through the GitHub genode-world timeline I takes notes, but if in neither of these, I'm not going to be aware of its existence, so something to keep in mind.
The discoverability remains unaddressed. The existing "components overview" [1] does not do justice to the current (and emerging) situation where more and more Genode software resides outside our central repository. Maybe a community-managed plain list of links to projects (be it in the form of Goa projects or components hosted at the world repository or in other forms) would be good start? Still, someone would need to assert the responsibility to host and curate such a list.
[1] https://genode.org/documentation/components
- re drag'n'drop, curious to see what the Genode approoach would be. It might save me the need to roll my own ad-hoc implementation in H/Ge.
I was indeed thinking of you when I mentioned drag'n'drop.
- same thing with (if I understand correctly) creating a "nitpicker_primitives.lib.so" that can be re-used by other components : color me curious and interested.
The foundational work is already done and found its way into the 23.11 release [2] but I deliberately don't overly advertise it to avoid wrong expectations. However, if you are curious about the use of this approach, you may risk a look at gems/src/app/sculpt_manager/ and the view/ sub directory in particular. Over the next year, I plan to successively complement the API with the features needed for more general use. One key feature will be the ability to integrate existing nitpicker clients into applications (similar to xembed), thereby clearing the path to augment existing full-screen applications (think of mupdf or avplay or VMMs) with UI controls. I think that this idea will fit Genode's capability-based component architecture like a glove.
[2] https://genode.org/documentation/release-notes/23.11#Dialog_API_for_low-comp...
FS optims: Maybe the one immediate item remaining on my Genode wish-list will be vfs related: Even if I'm in "reaping the fruits of past work" mode, there might still be one type of ticket I might file on github yet, related to FS xattr performance. Reading attributes in H/o-g is quite slow, maybe a couple dozen reads per sec. It's too soon to tell where the optimization potential is located though. Given the nature of the current xattr implementation (lots of back-and-forth between fs client and vfs server to process ad-hoc/virtual paths like /Station/Music/.PinkFloyd/.attributes etc) it comes to mind as a primary suspect, but gotta be careful to pinpoint accurately which part of which component is too slow and in need of optimization (on my side or on Genode's). It's easy enough to get lost in the weeds even for skilled devs, even more so for me 8-). My trip down that rabbit hole will probably begin in a few weeks or months, once the other ToDos are reasonably complete.
I'm with you on that. You already helped weeding out one performance issue [3] in the past, for which I'm grateful. We will surely get to the bottom of this one together as well.
[3] https://github.com/genodelabs/genode/issues/4603
Thank you again for the captivating write-up.
All the best!
Norman
Hi Norman,
that kind of small-to-medium
size Be ports to Genode is quite easy now that most pieces of the Be
puzzle are in place.
The screenshot nicely shows how far your work has come by now. Speaking of the porting of Be applications, I wonder how Goa could be applicable in these cases? Should we succeed to eventually bridge the gap between your customized version of Genode (xattr) and the regular Sculpt OS while fostering the use of Goa for such porting efforts, developers who enjoy casual porting work may become able to join the fun, building upon your H/o/G layer as a library.
Was hoping you'd suggest so! This could be mutually beneficial. This morning I'm pushing a series of commits to chiselapp with a couple of simple apps/applets (a desktop calculator...) to get our feet wet, should we proceed. One of the commits hides the call to xattr-patched-Genode vfs behind an #ifdef. When running the test suite I see that this breaks only what it is supposed to break and the rest of the test-suite is in the green, and the applets still run fine.
I'm going to open a ticket in the appropriate place (probably genode-world ?) with a possible game plan, titled "Goa: add support for Jam and h/o/g ?". Anyone who is tickled curious by that can pick up on it any time they like, I'll be there.
Once the initial hurdles are overcome, now that I have a bit more time on my hands I'd be able to spend time 'scouting and porting', i.e. looking for apps (IRC client ? utilities ?) that don't require the whole Be shebang with xattr et al but would still be useful additions to the Genode depot and useful to the community.
(and one could imagine a third step in the mid to long term aiming at getting access to the remainder of Be apps : maybe the xattr dependancy should be moved to a "file system add-on", just like the implementation for socket() currently resides in vfs_lxip.lib.so, if doing so resolves the filedesc-to-file-path-mapping-is-Private-access-only conundrum ?)
But Genode currently looks like it could handle an awful lot of part #2 --
provided I port the necessary software of course.
Wow! The stars seem to be aligning. :-) With that I mean that my personal ambition to streamline the use of Sculpt OS on the desktop will hopefully benefit your ambition. Since you have integrated Genode's depot/deploy mechanism now, you can finally draw on the full potential of our community (given the increasing arsenal of delightful packages maintained by individuals). Vice versa, I'd be thrilled to be able to fetch an index of nice Be applications from your depot one day.
Right now I'm in a bit of a bind for Goa : there is so far marginal Be support for 'expect' and 'tcl', which I understand are needed by Goa for building, packaging, and maintaining the depot. So I'd need to team up with someone on Linux (where those tools can be safely taken for granted) and let them publish the packages, do the Goa legwork at the end of the journey, after I've done the C++ and Jam porting work.
But I'm not giving up on being a "first class citizen" so to speak, some day I'll run all those tools myself, either in a VM hosted on Genode or in Genode itself ^^.
The discoverability remains unaddressed. The existing "components overview" [1] does not do justice to the current (and emerging) situation where more and more Genode software resides outside our central repository. Maybe a community-managed plain list of links to projects (be it in the form of Goa projects or components hosted at the world repository or in other forms) would be good start? Still, someone would need to assert the responsibility to host and curate such a list.
That list's a good case in point : it does not mention Falkon or Morph ;-) Just thinking off the top of my head : what if Genodians.org could contribute to the discoverability ? After all, it is to HTML documents what the depot is to .tgz packages : a decentralized aggregation. That is to say, multiple providers agree to publish their contents (html or tgz) in a selected room of the "mothership". Well, more so for genodians article which all share the same base URL. But that's the point indeed :
What if people with a repo decided to also create a genodians profile (if not already done), publish an article in there titled "My Depot Packages (Index)" or such, and used the editability feature ? If I'm not mistaken, a page published on Genodians can be edited and modified ad infinitum after its initial publication. So there'd be a "My Packages" page maintained by the official Genode Labs account, one by cproc, and so on. There could even be a special-handling of those pages, where they can be aggregated all-in-one by clicking a special link (currently if you click a particular author you get an aggregation of their articles, so something orthogonal to that, where you select a theme and get an aggregation of the one post of each author that shares that title/theme).
That would immediately help with discoverability for the "power user" types (and even the public at large might find it helpful as is, or if later augmented with screenshots and the like... but you'll probably want to wait until it is "worth it" investing that kind of time).
Even if each page was just HTML code generated automatically from the XML index file of each repo/author (i.e. no further work after the initial time invested in writing the generating scripts), I could see that providing big discoverability benefits -- provided every depot creator plays ball and creates a genodians account with an "My Index" page invoking that script.
Or.... use the "wiki" feature of GitHub ? That is, each dev maintains a wiki. On the other hand, on Genodians all "index" style articles would be close to each other, easily accessible in sequence, now sure how to produce the same effect on github (and is it possible to run scripts on wikis ?)
By contrast, a one-stop-shop centralized page with a curated list has more potential for user-friendlyness than a decentralized one, but requires solving responsibility/times issues indeed.
- same thing with (if I understand correctly) creating a
"nitpicker_primitives.lib.so"
that can be re-used by other components : color me curious and
interested.
The foundational work is already done and found its way into the 23.11 release [2] but I deliberately don't overly advertise it to avoid wrong expectations. However, if you are curious about the use of this approach, you may risk a look at gems/src/app/sculpt_manager/ and the view/ sub directory in particular. Over the next year, I plan to successively complement the API with the features needed for more general use. One key feature will be the ability to integrate existing nitpicker clients into applications (similar to xembed), thereby
At a glance, the wording sounds familiar to me, like the "Replicants" add-ons of the Be API.
clearing the path to augment existing full-screen applications (think of mupdf or avplay or VMMs) with UI controls. I think that this idea will fit Genode's capability-based component architecture like a glove.
Was curious of what Dialog looks like for an API 'end-user' like me so I took a look at test/main.cc, It's out of the beaten path indeed ^^. Though many of the paradigms will be familiar to all UI coders, like implementing a click() hook for when a button is invoked. While exploring the Depot last month I noticed some components are already making use of the dialog package if I'm not mistaken (like the power management ones), planning to try these packages out.
Anyhow, making µpdf or AVPlay use that new lightweight UI instead of qt5 is something I'm looking forward to. Will nudge me into making them part of my 'distro'. Consider yourself gently "nudged" forward ;-)
Thank you again for the captivating write-up.
All the best!
And all the best to you and your team of fellow ass-kickers, if I can be so bold :-)
Cedric
P.S. Also curious to learn more about that new Ge/Linux project discussed yesterday. The teaser video looks good.
Hi Norman,
On 29/12/2023 14:26, Norman Feske wrote:
The discoverability remains unaddressed. The existing "components overview" [1] does not do justice to the current (and emerging) situation where more and more Genode software resides outside our central repository. Maybe a community-managed plain list of links to projects (be it in the form of Goa projects or components hosted at the world repository or in other forms) would be good start? Still, someone would need to assert the responsibility to host and curate such a list.
when reading this, I remembered your original posting in which you mentioned presenting the README files of software packages in the GUI. Maybe we should make it a habit to add the URL of the original source-code repository in those README files. This way, a list of software packages and their origins could be extracted from the depot.
Johannes
Hello Norman, Hello Genodians,
right before Christmas Eve it's about time to share my personal view of 2023 and sketches for 2024.
On Thu, Dec 21, 2023 at 16:57:48 CET, Norman Feske wrote:
Reflections of 2023
Smirkingly, I have to consent Norman's list of 2023 highlights :-) I was personally involved in many low-level developments, some of them on winding paths (e.g., the not yet finished renovation of our USB infrastructure). This meant a lot of testing, learning, discussions, and amending decisions that did not quite fit the requirements. I'd like to thank all members of the team for the fruitful collaboration and the patience that must have been required to explain intentions or refactor code yet another time.
Plans for 2024
Theming the year 2024 with "Sculpt OS usability" sounds great to me, because usability is essential to gain foothold in the "rocking territory". But unlike Norman's, my perspective still remains from the low-level up. As Michael (RockChip USB) and Pirmin (IGC NIC, multi-monitor) noted in their postings, our already good platform and device support can get even better, which will support usability improvements indirectly. In concrete, I'd draft the following four ambitions.
* In a supporting role, I'd be glad to advance the full integration of the kernel-agnostic IOMMU support and base-hw virtualization. Both projects are stepping stones to a Sculpt PC variant on base-hw.
* I myself plan to push forward our PC platform discovery and ACPI support to a future-proof solution for a broad range of recent Intel-based devices. This is a prerequisite for clean integration of platform gadgets like Intel GPIO, I2C human interface devices.
* Also, I'm ready to do my share for consistent multi-monitor support in our GUI stack from the ground up.
* Last, I look forward to the full integration of the diversity of touch-based devices like touchpads, graphic tablets, and touch screens into our input-event stack.
How and where would you like to see Genode at the end of 2024?
It would be great to conquer more devices of more people next year.
Regards and Happy Christmas
Christian
Hello Norman and Genodians,
I want to share my plans for the first time. In 2024, I plan to finally start using Sculpt OS. The only thing keeping me is the dependency on some familiar Linux software. I would like to have a headless virtual machine with Linux, and to have Linux applications as native Nitpicker windows, to experience something similar to Qubes OS. I already have an early prototype working on Genode Linux, forwarding windows from the host system (see short video [1]). However, there's still a lot to be done before publishing. This is my plan for the year 2024.
[1] https://vanner.me/genode_wayland.mp4
-- Best Regards Ivan Loskutov
-- Best Regards Ivan Loskutov
On Thu, Dec 21, 2023 at 4:58 PM Norman Feske norman.feske@genode-labs.com wrote:
Hello everyone,
in his most recent posting, Credric already noted that now is the time of year again to reflect on our achievements of this year and to anticipate topics to pursue next year. To keep up with this fine tradition, let me kick off our road-map discussion with my personal perspective.
Reflections of 2023
As indicated by the overarching theme of the last roadmap "Rocking the platforms we support!", Genode's four releases of 2023 had a strong focus on low-level platform work. This has been especially visible on modern PC platforms like the Gen12 Framework laptop I have under my fingertips right now. At the beginning of 2023, Sculpt OS was in principle working on this machine, but with compromises that spoiled the user experience: fan noise, an erratic touchpad (using the firmware's PS/2 emulation), Fn key having no effect, strange issues when re-plugging an external display, and no indication of the battery state. In the meantime, not only are all these rough edges gone, but we even gained the ability to exercise precise control over the machine' performance/frequency/temperature/power characteristics using an interactive GUI. I think it's fair so say that Genode advanced far beyond the state of "working" and has entered the territory of "rocking".
I wish to name four further personal highlights of the year:
First, we got the mobile version of Sculpt OS into the hands of a pilot group of users who provided instructive feedback to us. From my perspective, the system-update mechanism that I created for this purpose turned out to be an almost pivotal point in the evolution of Sculpt OS because it reduces the effort and risk of test-driving experimental versions to almost zero. It was a pleasure to see how e.g., Johannes leveraged this new way of gathering feedback for his IOMMU line of work. Providing system images for testing has become a second nature.
Second, the road map for 2023 envisioned Sculpt OS running on our custom base-hw kernel on the PC. We identified DMA protection and virtualization support as the two remaining showstoppers. With much excitement, I followed how both of these deeply technical topics got covered over the course of the year.
Third, Goa finally emerged from a personal project of mine to an official Genode project led by Johannes. I'm stunned how much the project benefited from this change. All of the remaining backlog of my vague plans - I'm thinking of the index-project support or bash completion - got eventually realized in a way true to the spirit of the project. The fate of the Goa tool makes me immensely happy.
Fourth, during the first half of the year, I found myself intensively working on Genode's new debug-monitor component, pursuing the idea to implement a debugging instrument as a specialized version of init augmented with the GDB protocol. This engagement was pretty much motivated by a customer. The result of my initial work then seamlessly transitioned into the hands of Christian Prochaska who did a marvelous job with steadily advancing this line of work towards our joint vision of on-target debugging on Sculpt OS. The technical feats notwithstanding, I found the frictionless way of collaborating a pure joy.
Besides the highlights above, one topic close to my heart was the creation of the dialog API that I designed as necessity to make the code of Sculpt's administrative user interface maintainable and easy to extend in the longer run.
Plans for 2024
After concentrating so intensively on topics below the surface, I now long for reaping user-visible rewards. Speaking of the dialog API just mentioned, I see potential in using this new infrastructure for Genode-native applications and immediately think of the file manager that I already wanted to tackle this year. But I also have plenty of ideas to make Sculpt OS more user friendly. What about presenting the README files of software packages directly in the GUI? Making the component graph scrollable? Allowing the user to select an arbitrary directory as a file system to a component? Buttons for saving the current deployment or the settings?
Beyond Sculpt's administrative user interface, I'd also like to attend the GUI stack. I think of refining the GUI-session interface to remove tearing artifacts, to better support desktop-UI-paradigms like drag'n'drop, and to explore the opportunity of using nitpicker's mechanisms at the application-level, not only at the window-composition level.
Hence, I'd condense my ambition for the next year to "Sculpt OS usability".
Device-wise, I'm going to continue my engagement with the PinePhone and look forward to the upcoming MNT PocketReform laptop.
Above I presented my personal view. How about your's? I would very much appreciate you sharing your feedback, ideas, concerns, and plans regarding Genode.
How are your interests aligned with the perspective shared above?
Do you see specific pain points that deserve the attention of Genode's core developer team?
What is your perspective on Genode's past year's accomplishments?
Can you share your ambitions or even concrete plans?
How and where would you like to see Genode at the end of 2024?
Cheers Norman
-- Dr.-Ing. Norman Feske Genode Labs
https://www.genode-labs.com · https://genode.org
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users
On Sat, Dec 30, 2023, 5:13 PM Ivan Loskutov loskutov.ivan@gmail.com wrote:
Hello Norman and Genodians,
I want to share my plans for the first time. In 2024, I plan to finally start using Sculpt OS. The only thing keeping me is the dependency on some familiar Linux software. I would like to have a headless virtual machine with Linux, and to have Linux applications as native Nitpicker windows, to experience something similar to Qubes OS. I already have an early prototype working on Genode Linux, forwarding windows from the host system (see short video [1]). However, there's still a lot to be done before publishing. This is my plan for the year 2024.
[1] https://vanner.me/genode_wayland.mp4
-- Best Regards Ivan Loskutov
I usually keep quiet around here, and was intending to do so now as well, but this is too exciting.
This annual thread is one I always look forward to and which reliably brings me quite some joy to read. This year is no exception.
Last year I recall my hope and excitement being primarily captivated by the aspirations and intent for suspend/resume on x86 to become a reality. In the time since, my excitement on that topic has waned somewhat, due partly to the acceptance that that road is even longer and more difficult than I already anticipated, and partly due to the much anticipated rise of prevalence of realistic contenders to take x86's increasingly tenuous throne, which have favorable power characteristics to begin with.
This year, so far, your video takes the cake for me personally.
This may be the most teasing video I have seen all year! In my excitement I showed it to the nearest person available who, predictably, found it entirely mundane and my excitement unrelatable. Alas, our niche may be narrow and true appreciators few and far between, but I can assure you the merit and potential of this line of work is most certainly not unnoticed by all! ;)
The marriage of ideas and approaches found in Qubes with something providing a more solid foundation for the future is a topic I have been passionate about for a long time, and I am eager to follow your work. If you wish to discuss the design space, I would be very happy to share whatever knowledge and perspective I have developed over the years while working on Qubes and similar efforts. Regretfully I cannot commit to more than that at this time (which is why I was otherwise intending to stay quiet), but I am truly excited that someone else is pursuing this and happy to support your efforts how I can.
What an exciting way to bring in the New Year :)
Kind regards, Jean-Philippe
Hi Jean-Philippe,
thanks for your kind words.
If you wish to discuss the design space, I would be very happy to share whatever knowledge and perspective I have developed over the years while working on Qubes and similar efforts.
I will be happy to get feedback from you and all other Genodians. I am working on cleaning and publishing all my code.
Frankly, I don't work on the design for this project. It is a more iterative process for me. I only have a little free time and want a working solution with minimum effort from my side, using as many existing components as possible. I learn new things, go deeper, and change my approach if it makes sense.
The current solution is quite simple. On the Linux side, it is a customized Wayland compositor based on dwl (simple wlroots-based tiling window manager). And a server that captures screens and inputs event injection using Wayland protocol extensions. The server now provides a simple TCP-based interface.
On the Genode side, it is a simple app that gets framebuffer data, shows it, and sends input events to the server.
Now, I am trying to improve performance. I want to implement some virtio device to remove TCP and reduce the number of copies for framebuffers data.
So, stay tuned :)
-- Best Regards Ivan Loskutov On Sun, Dec 31, 2023 at 11:23 PM Jean-Philippe Ouellet jpo@vt.edu wrote:
On Sat, Dec 30, 2023, 5:13 PM Ivan Loskutov loskutov.ivan@gmail.com wrote:
Hello Norman and Genodians,
I want to share my plans for the first time. In 2024, I plan to finally start using Sculpt OS. The only thing keeping me is the dependency on some familiar Linux software. I would like to have a headless virtual machine with Linux, and to have Linux applications as native Nitpicker windows, to experience something similar to Qubes OS. I already have an early prototype working on Genode Linux, forwarding windows from the host system (see short video [1]). However, there's still a lot to be done before publishing. This is my plan for the year 2024.
[1] https://vanner.me/genode_wayland.mp4
-- Best Regards Ivan Loskutov
I usually keep quiet around here, and was intending to do so now as well, but this is too exciting.
This annual thread is one I always look forward to and which reliably brings me quite some joy to read. This year is no exception.
Last year I recall my hope and excitement being primarily captivated by the aspirations and intent for suspend/resume on x86 to become a reality. In the time since, my excitement on that topic has waned somewhat, due partly to the acceptance that that road is even longer and more difficult than I already anticipated, and partly due to the much anticipated rise of prevalence of realistic contenders to take x86's increasingly tenuous throne, which have favorable power characteristics to begin with.
This year, so far, your video takes the cake for me personally.
This may be the most teasing video I have seen all year! In my excitement I showed it to the nearest person available who, predictably, found it entirely mundane and my excitement unrelatable. Alas, our niche may be narrow and true appreciators few and far between, but I can assure you the merit and potential of this line of work is most certainly not unnoticed by all! ;)
The marriage of ideas and approaches found in Qubes with something providing a more solid foundation for the future is a topic I have been passionate about for a long time, and I am eager to follow your work. If you wish to discuss the design space, I would be very happy to share whatever knowledge and perspective I have developed over the years while working on Qubes and similar efforts. Regretfully I cannot commit to more than that at this time (which is why I was otherwise intending to stay quiet), but I am truly excited that someone else is pursuing this and happy to support your efforts how I can.
What an exciting way to bring in the New Year :)
Kind regards, Jean-Philippe
Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users
Hi Ivan,
On 2023-12-31 00:13, Ivan Loskutov wrote:
I want to share my plans for the first time.
what a nice surprise! Long time no see. :-)
In 2024, I plan to finally start using Sculpt OS. The only thing keeping me is the dependency on some familiar Linux software. I would like to have a headless virtual machine with Linux, and to have Linux applications as native Nitpicker windows, to experience something similar to Qubes OS. I already have an early prototype working on Genode Linux, forwarding windows from the host system (see short video [1]). However, there's still a lot to be done before publishing. This is my plan for the year 2024.
I believe this is a dream hedged by quite a few, as evidenced by Jean-Philippe's enthusiastic response.
Your idea to leverage Wayland for this purpose is very cool!
Closely related, some years ago, Daniel Collins implemented an Xpra client for Genode. Although it has remained at a prototype stage [2], the code is accompanied with nicely entertaining documentation [3] (practical experiences, discussing X-isms, various ideas).
[2] https://github.com/zorodc/genode-world/commits/xpra/ [3] https://github.com/zorodc/genode-world/blob/d59879cdd33c5e035e80fb53cd36d927...
Thanks for sharing this teasing preview, which definitely wets the appetite!
Cheers Norman
Hi Norman,
Closely related, some years ago, Daniel Collins implemented an Xpra client for Genode. Although it has remained at a prototype stage [2], the code is accompanied with nicely entertaining documentation [3] (practical experiences, discussing X-isms, various ideas).
Thanks for the pointer. I remember you talked about it, but I've never tried searching for sources. I may find some new ideas there.
Best Regards Ivan Loskutov
Hi,
in 2023 I spent more time with the optimization of the Morph web browser performance on the PinePhone with the help of the "trace recorder" and "Eclipse Trace Compass" tools, which went as far as implementing a "Genode lock analysis" feature in Trace Compass to investigate why Jitsi RTP audio packets are often not processed fast enough. Then it was time for another tool chain update and since then I've been mainly working on the new debug monitor with the goal of on-target debugging on Sculpt OS.
For 2024, I'm interested in continuing with the debug monitor and tracing topics and also in porting more development tools that are needed to build Genode on Sculpt OS without depending on a Linux VM. We could also consider updating Qt to Qt6 to fix the current Python 2 dependency of QtWebEngine (since Python 2 is deprecated and apparently not even provided by newer Ubuntu versions anymore) and to get a newer Chromium engine version for the Falkon and Morph web browsers.
Christian
Hi Chriatian,
On 2024-01-02 12:58, Christian Prochaska wrote:
For 2024, I'm interested in continuing with the debug monitor and tracing topics and also in porting more development tools that are needed to build Genode on Sculpt OS without depending on a Linux VM. We could also consider updating Qt to Qt6 to fix the current Python 2 dependency of QtWebEngine (since Python 2 is deprecated and apparently not even provided by newer Ubuntu versions anymore) and to get a newer Chromium engine version for the Falkon and Morph web browsers.
that is a perfectly sensible plan. Thank you for both the concrete work items (the updates of Qt and QtWebEngine), and also for the steady continuation of the dev-tooling-on-Genode topic. In particular, I think that the ability to run Goa directly on Genode would be quite a pinnacle.
Cheers Norman
Hello everyone,
here is my input to the roadmap discussion:
First of all, many thanks to all of you, who have put so much energy into Genode in 2023, making it a pleasure to use it on a daily basis!
On Thu, Dec 21, 2023 at 04:57:48PM +0100, Norman Feske wrote:
Plans for 2024
I share the ambition of "Sculpt OS usability". Personally, I would love to have an e-mail client that can be used natively on Genode. I know, we had this on the Roadmap already years ago, and it didn't went out, because of limited time resources. But nowadays we have hopefully more efficient approaches to simplify the porting of mail handling tools and libraries, due to the recent improvements of Goa. More lighweight toolkits, like LVGL, might help to build a surrounding GUI.
Beside any visible improvements, I see some leftover works like the USB session re-design, or HW kernel optimizations (e.g. scheduling) to use it as default kernel in Sculpt OS. Moreover, we should keep the Linux driver ports up-to-date. Thereby, I would like to re-think the IRQ session to support more than one MSI/MSI-x interrupt per device, which will help to use certain drivers/devices the way they are used in their original context.
Although, the ARM virtualization is more-or-less feature complete, we're lacking integration to easily bootstrap and use a Linux guest OS.
There exists a Linux audio driver port for newer Intel cards outside the Genode mainline repository. It contains support for SoF (Sound Open Firmware), which is used in i.MX 8MQ's sound landscape too (to my knowledge). I would appreciate to enable sound for MNT Reform 2 using this driver approach, when the before mentioned driver becomes mainline.
This is a longer wishlist, and potentially I won't be able to address all these points personally, but maybe we find synergies together ;-).
Best regards Stefan
Hi Stefan,
thank you for sharing you ideas and plans. I concur to each point 100%.
It would be great to revive the email user-agent topic. Could this endeavour be a welcome opportunity to stress Goa's new Rust support by experimenting with (existing?) Rust-based protocol implementations (IMAP, SMTP...) as building blocks? Ben wondered about a natural place for Rust in our picture. I sense that this use case could provide us with a tangible way to let the combination of Rust with Genode shine.
There exists a Linux audio driver port for newer Intel cards outside the Genode mainline repository. It contains support for SoF (Sound Open Firmware), which is used in i.MX 8MQ's sound landscape too (to my knowledge). I would appreciate to enable sound for MNT Reform 2 using this driver approach, when the before mentioned driver becomes mainline.
This goes hand in hand with the audio line-of-work that I'm currently pursuing. If everything works out as I hope for, the new Linux-based audio driver can become our new default soon. If this clears the path towards i.MX8 sound, all the better! Personally, I'd be pretty excited about audio on the MNT Reform as I'm using this machine for Atari hobby activities that are most fun when loud.
Cheers Norman
Hi everybody,
It would be great to revive the email user-agent topic. Could this endeavour be a welcome opportunity to stress Goa's new Rust support by experimenting with (existing?) Rust-based protocol implementations (IMAP, SMTP...) as building blocks? Ben wondered about a natural place for Rust in our picture. I sense that this use case could provide us with a tangible way to let the combination of Rust with Genode shine.
I haven't personally used it, but runt[1], an isync[2]-like IMAP <-> Maildir synchronization tool, could maybe be a useful building for Stefan's (mutt) setup on Genode. However it seems that runt is being even less actively developed than isync. Moving towards a custom GUI, himalaya[3] is a CLI email client that could probably be wired up similarly to the recent SIP client GUI[4]. Finally, rust-imap[5] as a library is rumored to be of good quality. As Michael R. Elkins observed[6], it's not easy to build a decent email client. I think on the one hand, email is indeed a use case where Genode-style decoupling of components really could shine. On the other hand, I believe it's important to set a reasonably narrow scope, especially if we're building our own GUI like Stefan suggested.
As for my Rust aspirations, I guess how deeply using Rust in this context will touch native Genode interfaces strongly depends on the runtimes chosen in the different building blocks (porting of a Posix program vs. native library, use of something like the Terminal Crosslink vs. Genode IPC and Dataspaces etc.) I think a deeper exploration of the email client roadmap idea would best be done in person, it sure seems an interesting way to explore native Genode components and Rust!
Best wishes,
Ben
[1] https://github.com/mordak/runt [2] https://isync.sourceforge.io/ [3] https://github.com/soywod/himalaya [4] https://genodians.org/jws/2023-11-16-sip-client-for-genode [5] https://github.com/jonhoo/rust-imap [6] "All mail clients suck. This one just sucks less." -me, circa 1995 http://www.mutt.org/
Hi Norman, Hi Genodians,
Reflections of 2023
Reflecting on the last year, I'm very pleased with the results. Even though, I had other plans in mind, I really enjoyed what eventually landed on my plate.
Taking over Goa's maintenance role sparked a plethora of ideas. It was fun implementing some of those like the index-project support and bash completion; I am thrilled to see the tool mature further. I appreciate all the contributions, especially Pirmin's shared-library support, which I regard as a game-changer w.r.t. Goa's autonomy.
I spent a great part of the year working on DMA protection. Initiated by my work on the Zynq-7000 SoC, for which I developed a custom DMA guard IP core, I integrated support for IOMMU-like devices into the platform driver. Later that year, I could build upon this to integrate Intel DMA remapping support into the platform driver, which bit off a big chunk of the kernel-agnostic IOMMU support.
Plans for 2024
I share Norman's ambitions for focusing on Sculpt OS usability. I think Goa can play an important role for the development of utility applications. I'm thinking of a USB passthrough GUI, a NIC router port-forwarding GUI, or a file launcher GUI. As mentioned in last year's roadmap, it would be great being able to download a PDF with Falkon and open it with mupdf. My plans are to pick one or two to further test Goa's practicality for app development and share the process on genodians.org. My ultimate goal would be to assemble a third Genode book that focuses on application development.
Besides these app-focused topics, I still have a few low-level topics on my plate. First, there is the kernel-agnostic IOMMU support that still lacks IRQ-remapping support to be complete. Second, I'm expecting new hardware devices on which I'd like to test-run Sculpt. One device is a ZimaBlade single-board server, which I'd like to use as a new VM host for my home server. I believe it could make an interesting use case for a headless Sculpt deployment. The other device is a StarLite tablet, which I will mainly use for browsing and which should serve for testing Sculpt's usability on tablet-like devices.
Another intriguing idea is to try running Goa natively on Genode. This would make a pretty cool demo, yet, I'd not like to schedule it on this year's roadmap but rather keep it in mind for the next year(s).
Best regards Johannes
Hi All, Happy New Year to everyone. As always it's fun to read about all the stuff going on. Personally, my reflection on 2023 goes up just to July, as since then I haven't had as much time to play with Sculpt/Genode :( But in the first half of 2023 I did have a lot of fun working on USB wifi dongle drivers, and I did get both the ath9k and rt2x00 based devices to work. In 2024, I may take another look at this once I get my head around the changes to the USB system. It seems like a lot of exciting changes are in the works that could be very helpful. For example with a USB multiplexer, I hope there will be a smoother interface to connect a driver with a whole range of vendor/product IDs. In the meantime, I may also take a look again at the issue with Apple hardware and PS2 drivers, which is a minor annoyance. I think the ideal way would be for the PS2 driver to consume an ACPI report somehow and shut itself off in the event the PS2 hardware is not present, but I have to see how hard this would be to implement, as I have no experience with ACPI.
Regards, CP
Hi Colin,
On Tue, Jan 02, 2024 at 16:46:49 CET, Colin Parker wrote:
In the meantime, I may also take a look again at the issue with Apple hardware and PS2 drivers, which is a minor annoyance. I think the ideal way would be for the PS2 driver to consume an ACPI report somehow and shut itself off in the event the PS2 hardware is not present, but I have to see how hard this would be to implement, as I have no experience with ACPI.
I desire a similar feature since ages as we also have a test PC that behaves stranges on the PS/2 port (mouse protocol). Unfortunately, it is not trivial to detect PS/2 resources from ACPI and, in fact, requires to run the _STA/_CRS AML method for requesting the "function" state and current resources.
During my exploration of improved boot-time ACPI discovery, I had promising results when checking PNP0F13/PNP0F03 ACPI devices. In the case of modern notebooks, those devices only get state "functioning" if the PS/2 mouse emulation was enabled in BIOS. My plan includes to enhance the platform devices boot-time information with data about detected ACPI devices and their resources. This information could than be used by the platform_drv to amend the platform session of the PS/2 driver with device information. As a result, the driver may initialize only those devices (mouse or keyboard) detected on boot.
In the meantime, you may check the ACPI information for the device in Linux or MacOS. Look out for devices with HID/CID PNP0F13 or PNP0F03 (mouse) resp. PNP0303 (keyboard).
Greets
Hi Johannes,
thank you for sharing your view. In fact, I remember you putting the usability topic on the agenda one year ago. So I somewhat expected you resonating with my proposal, and I'm not disappointed. :-)
It is great reading that you already have the natural continuation of the IOMMU work in the back of your head. The plan about the application-centered book is splendid. This is definitely a missing piece of the puzzle today.
What you write about the new devices on your plate invokes quite a few cross-connections to other topics we discussed. In particular, I welcome another demanding use case for a practical touch UI besides the phone. Your tablet would fit just right. This is very much in line with John's desire about PC touch-screen support. Since I've put the revisiting of the GUI-stack on the schedule, this aspect must be part of it (e.g., making the window manager touch-aware, interplay of multi-monitor support and touch).
The vision of self-hosted Goa on Genode is mutual. However, I agree that adverting it on the road map for this year would be over-ambitious.
Cheers Norman
On 12/21/23 10:57, Norman Feske wrote:
Hello everyone,
[snip]
How are your interests aligned with the perspective shared above?
I failed miserably last year, but my main goal for the new year (again) is to make Genode my "daily driver" OS. This obviously fits perfectly with the "Year Of Sculpt OS Usability" motif, and I will be a willing and eager tester/adopter of whatever this brings.
Do you see specific pain points that deserve the attention of Genode's core developer team?
Not a pain point, but a wish - absolute pointer support for PC touchscreens. The touch interface works surprisingly well on the small phone screen, but would really shine on a larger screen.
(In a similar vein, screen rotation would also be nice, but I don't know if there are architectural difficulties there.)
What is your perspective on Genode's past year's accomplishments?
Congratulations to the entire team and community for another impressive year! Along side the driver system improvements, "rocking" the PinePhone was really impressive (and fun!), but the biggest highlight for me was the new system update mechanism. That was a usability game-changer!
Can you share your ambitions or even concrete plans?
As mentioned above, my #1 goal for the year is to move to Sculpt as my "daily driver" OS. This will start with running VMs in VirtualBox, but I really want to migrate to native apps where possible. As I progress on this road, I will try to write some "newbie" articles for Genodians.org, etc.
There are so many facets of native Genode development that I would love to explore, but considering my time constraints, I will keep my expectations realistic. :^( But I will try!
How and where would you like to see Genode at the end of 2024?
On every alternative OS enthusiast's computer (and phone), of course! ;^)
Seriously, IMO most if not all of the building blocks are in place for wider adoption; if the Year Of Usability lowers a few barriers to entry, Genode may finally get the attention it has deserved for years.
Happy Sculpting!
John J. Karcher devuser@alternateapproach.com
Hi,
Reflections of 2023
I spent the year mostly around the usual topics of driver development and maintenance, mainly on the PinePhone and DDE Linux in general. Bringing the Linephone client and a GUI to the PinePhone together with Sebastian was a nice intermission in the last quarter of the year. Apart from that my personal projects did not far as well as most of them, namely the amdgpu_fb_drv, stalled by the mid of the year as I got occupied with other non-computer related things and the one or other one-shot was merely a gimmick.
Plans for 2024
To be honest I do not have any elaborate plans for the year but there certainly will be one or the other topic I will work on in a supporting role.
Best regards Josef
Hello,
with 2023 I continued the former PC suspend/resume work done in 2022 for the NOVA kernel and integrated the feature also to base-hw. Unfortunately, after the low level kernel work was in shape, various higher rated work items led in the course of the year to insufficient time to push this item appropriately, namely adjust all our device drivers to play well after resume.
Beside quite a lot of customer triggered work, worth mentioning is the update of seL4 to 12.1 and the update of the Intel device driver and the subsequent backend integration changes towards usage of the DRM related interfaces in our port. Additionally, I contributed various acpica and usb-c attached display related fixes which also improved the Gen12 framework notebook support.
My former spare time developed and maintained x86 CPU Intel/AMD power/frequency extension for the NOVA kernel got generalized and streamlined to the Genode framework and is part of Sculpt OS.
In my Genode related spare time I maintained the experimental multihead extension of the Intel display driver, which I use day-to-day. Beside that, the actual most interesting and partially strenuous topic was for me to take a deep dive into the x86 world to extend the Seoul VMM for 64bit guests. With the end of the year I got all pieces together and was happy to operate my disposal 64bit Firefox VM for Intel and AMD. But the actual icing on the cake was shortly before Christmas, where I could replace vbox by Seoul to operate my day-to-day main developer VM - _grin_.
For 2024, presumably the PC suspend/resume topic related to our drivers will keep me busy. Additionally, in general I'm interested to get/add a general multi-head support for Genode, a port of Qemu to Genode with goa?, a port of the AMD display driver part, e.g. pick up the work of Josef of amdgpu_fp_drv, and various Seoul VMM improvements which improves life further.
Cheers,
Alex.
On 12/21/23 4:57 PM, Norman Feske wrote:
Hello everyone,
in his most recent posting, Credric already noted that now is the time of year again to reflect on our achievements of this year and to anticipate topics to pursue next year. To keep up with this fine tradition, let me kick off our road-map discussion with my personal perspective.
Reflections of 2023
As indicated by the overarching theme of the last roadmap "Rocking the platforms we support!", Genode's four releases of 2023 had a strong focus on low-level platform work. This has been especially visible on modern PC platforms like the Gen12 Framework laptop I have under my fingertips right now. At the beginning of 2023, Sculpt OS was in principle working on this machine, but with compromises that spoiled the user experience: fan noise, an erratic touchpad (using the firmware's PS/2 emulation), Fn key having no effect, strange issues when re-plugging an external display, and no indication of the battery state. In the meantime, not only are all these rough edges gone, but we even gained the ability to exercise precise control over the machine' performance/frequency/temperature/power characteristics using an interactive GUI. I think it's fair so say that Genode advanced far beyond the state of "working" and has entered the territory of "rocking".
I wish to name four further personal highlights of the year:
First, we got the mobile version of Sculpt OS into the hands of a pilot group of users who provided instructive feedback to us. From my perspective, the system-update mechanism that I created for this purpose turned out to be an almost pivotal point in the evolution of Sculpt OS because it reduces the effort and risk of test-driving experimental versions to almost zero. It was a pleasure to see how e.g., Johannes leveraged this new way of gathering feedback for his IOMMU line of work. Providing system images for testing has become a second nature.
Second, the road map for 2023 envisioned Sculpt OS running on our custom base-hw kernel on the PC. We identified DMA protection and virtualization support as the two remaining showstoppers. With much excitement, I followed how both of these deeply technical topics got covered over the course of the year.
Third, Goa finally emerged from a personal project of mine to an official Genode project led by Johannes. I'm stunned how much the project benefited from this change. All of the remaining backlog of my vague plans - I'm thinking of the index-project support or bash completion - got eventually realized in a way true to the spirit of the project. The fate of the Goa tool makes me immensely happy.
Fourth, during the first half of the year, I found myself intensively working on Genode's new debug-monitor component, pursuing the idea to implement a debugging instrument as a specialized version of init augmented with the GDB protocol. This engagement was pretty much motivated by a customer. The result of my initial work then seamlessly transitioned into the hands of Christian Prochaska who did a marvelous job with steadily advancing this line of work towards our joint vision of on-target debugging on Sculpt OS. The technical feats notwithstanding, I found the frictionless way of collaborating a pure joy.
Besides the highlights above, one topic close to my heart was the creation of the dialog API that I designed as necessity to make the code of Sculpt's administrative user interface maintainable and easy to extend in the longer run.
Plans for 2024
After concentrating so intensively on topics below the surface, I now long for reaping user-visible rewards. Speaking of the dialog API just mentioned, I see potential in using this new infrastructure for Genode-native applications and immediately think of the file manager that I already wanted to tackle this year. But I also have plenty of ideas to make Sculpt OS more user friendly. What about presenting the README files of software packages directly in the GUI? Making the component graph scrollable? Allowing the user to select an arbitrary directory as a file system to a component? Buttons for saving the current deployment or the settings?
Beyond Sculpt's administrative user interface, I'd also like to attend the GUI stack. I think of refining the GUI-session interface to remove tearing artifacts, to better support desktop-UI-paradigms like drag'n'drop, and to explore the opportunity of using nitpicker's mechanisms at the application-level, not only at the window-composition level.
Hence, I'd condense my ambition for the next year to "Sculpt OS usability".
Device-wise, I'm going to continue my engagement with the PinePhone and look forward to the upcoming MNT PocketReform laptop.
Above I presented my personal view. How about your's? I would very much appreciate you sharing your feedback, ideas, concerns, and plans regarding Genode.
How are your interests aligned with the perspective shared above?
Do you see specific pain points that deserve the attention of Genode's core developer team?
What is your perspective on Genode's past year's accomplishments?
Can you share your ambitions or even concrete plans?
How and where would you like to see Genode at the end of 2024?
Cheers Norman
Hi all,
On 21.12.23 16:57, Norman Feske wrote:
Reflections of 2023
For me 2023 started out with optimizing Genode's GPU Session for our Intel GPU multiplexer. GPU clients, like Mesa, often make a lot of fine grained video memory allocations, which led to a very large number of capabilities required because each allocation led to the creation of a Buffer object in the multiplexer that needed to be mapped to the client. In order to improve the situation, I removed the Buffer concept from the GPU session and replaced it with the notion of Video RAM (VRAM). A Mesa client now allocates large chunks of video memory (16MB) and handles small allocation locally. This decreased capability usage by an order of magnitude. Additionally, I implemented support for GPU Resets (GPU versions GEN9 and GEN12), so the GPU can recover from hangs caused by malicious or buggy clients.
After that I took a deep dive into the Sailfish-OS SDK in order to use it for application development for Genode on the PinePhone. I learned a little QML and started implementing my own QML Plugin because the GUI part of the Sailfish SDK (Silicia) is not open source. While this effort eventually failed (we settled on Ubuntu Touch or Lomiri how it is called now) it proved to be a valuable experience later in the year.
Because we needed a SIP client for Voip on the PinePhone, Josef and myself ported the LinPhone SDK to Genode. Because the SDK uses CMake as a build system, we could take full advantage of our Goa tool, while adding some improvements there on the way. For the GUI part my previous Sailfish/Ubuntu UI experience came in handy, as we used Ubuntu Touch's "Linphone Simple" as a base on Genode which required Josef to write QML Plugins.
With our ongoing effort to port all legacy DDE-Linux drivers to our new approach, I took care of the USB client drivers (HID, network, and modem). Because the network driver and the LTE-modem driver (MBIM) are very similar I was able to merge them into one driver. The last remaining artifact now is our Linux based IP stack (lxip), which I am currently updating from Linux version 4.4.3 to 6.1.20. Because this port is so ancient, it is more like a complete re-write. We additionally intent to replace the vfs_lxip Plugin with a more generic one that handles lwip and lxip, so this effort might well continue into 2024.
I helped with the Goa Project where my expertise seemed fit and I really like the overall progress.
Plans for 2024
For this year I would like to finally get rid of all the legacy DDE-Linux projects. I would also like to update Mesa to the current version by using Goa. This requires Meson support in Goa, which might become my Hack'n'Hike project 2024.
Since I use an Alder Lake laptop now, 3D acceleration and proper audio driver support would be a nice to have for me personally. If there is spare time, I want to have a look into audio re-sampling, since this becomes an increasingly pressing matter.
Looks like it is going to be an interesting year for Genode,
Sebastian
Hi Sebastian,
thank you for the elaborate reply, which is perfectly aligned with my perspective. I'm especially grateful that you have taken the TCP/IP line of work under your wings and wholly anticipate your goal to retire the last traces of the legacy-DDE-Linux as soon as possible.
Plans for 2024
For this year I would like to finally get rid of all the legacy DDE-Linux projects. I would also like to update Mesa to the current version by using Goa. This requires Meson support in Goa, which might become my Hack'n'Hike project 2024.
Since I use an Alder Lake laptop now, 3D acceleration and proper audio driver support would be a nice to have for me personally. If there is spare time, I want to have a look into audio re-sampling, since this becomes an increasingly pressing matter.
Speaking of audio, I forget to mention that I'm currently right in the middle of designing and implementing new infrastructure (session interfaces, mixer) for audio on Genode. I have three goals: First, to make audio drivers pluggable (following the tracks of our pluggible network/display/input drivers). Second, to make audio routing flexible (similar to how the NIC router gives us so much flexibility for network traffic). And third, to overcome the uncertainties of the current solution with respect to drifting/buffering/latency/jitter. To solve the latter, I'm applying an adaptive re-sampling approach that gives audio producers and consumers the freedom to use sample rates they see fit. Hence, the re-sampling is part of the my current work. I'm going for b-spline-based interpolation (in the time domain) in my first take. But this is of course not set in stone.
I plan to have the first version of the new infrastructure ready for Genode 24.02. Should it be received favorable, maybe we can jointly aspire converting the existing audio drivers and applications (i.e., vfs_oss) until version 24.05?
Cheers Norman
Hello everyone!
Reflections of 2023
For me, 2023 was dominated by virtualization-related topics and getting Rust to work on Genode.
The first few months of the year I was busy wrapping up support for SVM in base-hw and still wrapping my head around bits and pieces of the Genode OS Framework. Preliminary SVM support got merged as part of the 23.05 release and the same release saw a proof of concept port of the Rust toolchain to Genode.
Through collaboration with my colleagues, notably Sebastian Sumpf, by release 23.08 we refined our Rust support to the point where we managed to use an unmodified FreeBSD toolchain, which greatly lowered the effort for adoption or maintenance.
Meanwhile, I ported a first complex Rust package, which thanks to the ever improving Goa tool went very smoothly. Building on the great work and advice by Alexander Böttcher and Christian Helmuth, I built a new-paradigm virtualization interface across all kernels and architectures supported by Genode. Tackling corner cases on all platform proved to be a bigger challenge than expected, but the new interface got released with Sculpt OS 23.10, and has proven itself in daily use for Genode release 23.11.
Testing my implementation of Intel's VMX virtualization in base-hw revealed a few shortcomings that still need to be addressed before we can take it for a spin with Sculpt OS, which brings me to my plans for this year:
Plans for 2024
First of all, I want to wrap up my virtualization work and adapt base-hw for more dynamic workloads to get it ready for use in Sculpt OS on x86. After that, I would like to look into worthwhile optimizations of its virtualization support.
Chiming in on the theme of usability, I would like to explore how we can refine resource allocation for Sculpt OS Components to scale from the Pinephone to a (potentially multi-monitor) 4k setup.
Last but not least, I'm still pondering in which direction to take our Rust support. I would love to explore how an idiomatic Rust component would interact with our framework, but as I have written previously[1], we currently don't have a strong enough use case to craft a Genode ABI for use with a no_std Rust component. That being said, I hope to contribute a little rusty streak to the colorful exploration of (GUI) usability for next year to widen our current scope of Rust- and multi-language support. Maybe a little helper application that combines our libc interface with a 3rd-party (graphical) library and a few select direct calls to Genode? I have to admit that I don't yet have a specific project in mind though.
I can only agree with Norman that it was a pleasure to see Genode mature in everyday use on our development devices, and I'm excited what the next developments in the GUI stack, (ACPI) platform discovery and input device, multi-monitor and suspend to RAM support will bring.
Best wishes,
Ben
[1] https://lists.genode.org/pipermail/users/2023-October/008825.html
Happy new year everyone!
On 21.12.23 16:57, Norman Feske wrote:
Reflections of 2023
In the last year, I mainly worked at the CBE / Tresor / File Vault ecosystem. Josef, Alexander, and I translated the former CBE library that was written in Ada/SPARK to C++ in order to simplify maintenance for our team and reduce the size of the code base. Thereby, the project was renamed Tresor. Beside that, I enhanced the File Vault for the PinePhone integration to be steerable without a GUI via configuration and report. The results of this work entered the Sculpt 23.04 release.
After that, I introduced a new framework that caused the Tresor code base to shrink further and made the core logic easier to grasp. As a side effect, the tests on the ecosystem were also strengthened. The effort is still ongoing but in its final stage.
Furthermore, I was involved in working on some scheduling issues with the base-hw kernel that were triggered by the more recent media workloads. Last but not least, I simplified and enhanced the Depot-Autopilot interface for writing tests, created a NIC/Uplink session adapter and did some maintenance work regarding the network stack and the MMIO framework.
Plans for 2024
Most importantly, I want to bring the Tresor / File Vault clean-up to a point where it can enter the next Sculpt release and where it can be maintained resp. adapted by everyone in our team. I'm also at adding upper-bound checks to the MMIO framework. Furthermore, there is a plan to adapt the File Vault to use the new Dialog API and separating logic and UI into different components. And since quite some time there are two open but non-trivial issues with the NIC router that I'd like to fix. From there on, I have no further plans yet for 2024.
Cheers, Martin
Hello everyone,
In 2023, the progress in Goa turned out to be extremely helpful to us, as was the maturing of the Linux DDE, base-hw, and the tracing infrastructure. We also very much appreciate the new debug monitor.
Gapfruit's ambition for 2024 is to deploy a fleet of industrial IoT gateways to several factory floors. To achieve this, we will work on our Azure device management agent, fully implement TPM-backed security features, and enable a new device based on an i.MX8MP SoC. The last part seems to align with the planned work on the MNT PocketReform laptop.
Our pain points include stability problems with both available network stacks. We would also like to improve our in-house expertise in enabling new SoCs. Another issue worthy of further work would be the addition of resource trading to the fs_rom and cached_fs_rom components.
Also, we would very much welcome any further work to make the debug monitor available on arm_v8a. Any advances in tooling for performance analysis of complex asynchronous components are also topics of interest.
Best regards Stefan for the Gapfruit Team
Hello Stefan,
thanks lot for sharing your view. I'm happy to read that the topics of last year were so much appreciated by you.
It is good to know the importance of i.MX8. This certainly reinforces our commitment to this SoC family. What you write about the deployment plans for 2024 sounds exciting!
Our pain points include stability problems with both available network stacks. We would also like to improve our in-house expertise in enabling new SoCs. Another issue worthy of further work would be the addition of resource trading to the fs_rom and cached_fs_rom components.
These are all valid and important points. Fortunately, Sebastian has already expressed his ambition give the network stacks the attention they deserve. At present, an lxIP stack based on the new DDE Linux is already under way.
I agree with you that the resource management of the ROM services should best undergo a revision. Maybe we can relieve your particular pain point by agreeing on an ROM-session interface change that would in principle enable you to implement a resource-management scheme that matches your requirements? E.g., one sensible change would be to let the `dataspace` call optionally return a `Session_resources` object instead of a dataspace capability (using the `Attempt` pattern), thereby telling the client that it is in need of the stated amount of RAM/caps. So the client can upgrade the session before issuing a `dataspace` call again. I think that this change is doable without much disruption. With this change in place, you could implement ROM services that let the client pay.
From my view, I would like to (sometime down the road) replace the cached_fs_rom by an implementation that leverages two features of Genode that are currently underutilized. First, Genode's on-demand page-fault handling could be used to fetch parts of ROM dataspaces on demand instead of loading each dataspace completely. This could speed up the start of components that rely on overly large ROMs (like QtWebEngine) while also allowing us to evict data, capping the RAM used for the cache. And second, the resource-balancing protocol (aka "balooning") of the parent interface could be put to good use for dedicating slack RAM for caching ROMs and freeing the RAM on resource pressure. However, as I wrote, I'm thinking of those ideas as being somewhat longer term.
Also, we would very much welcome any further work to make the debug monitor available on arm_v8a. Any advances in tooling for performance analysis of complex asynchronous components are also topics of interest.
That fits perfectly well with our - in particular Christian Prochaska's - present developments.
Thank you again for your participation in the road-map discussion.
Cheers Norman
A couple thoughts (hijacking?) re. this discussion:
client can upgrade the session before issuing a `dataspace` call again. I think that this change is doable without much disruption. With this change in place, you could implement ROM services that let the client pay.
From my view, I would like to (sometime down the road) replace the cached_fs_rom by an implementation that leverages two features of Genode that are currently underutilized. First, Genode's on-demand page-fault handling could be used to fetch parts of ROM dataspaces on demand instead of loading each dataspace completely. This could speed up the start of components that rely on overly large ROMs (like QtWebEngine) while also allowing us to evict data, capping the RAM used for the cache. And second, the resource-balancing protocol (aka "balooning") of the parent interface could be put to good use for dedicating slack RAM for caching ROMs and freeing the RAM on resource pressure. However, as I wrote, I'm thinking of those ideas as being somewhat longer term.
From my vantage point, I find that idea (on-demand fetch) quite intriguing. If I
understand it correctly, it seems this concept would end up enriching the Genode API with yet another "forte" (strong point), the same way that things like RM, and the super modular vfs system, and many other things, empower us when writing code for Genode.
If I put on my 'syst. integrator' hat, when crafting my system to run Falkon I found myself wondering where to set the bar for the cached_fs_rom RAM quota, so that it would be neither 'too little' nor 'too big/wasteful'. The prospect of just setting it to a 'low' value like 200M and letting the Genode class handle where and when to load/unload disk data to respond to read requests is enticing. (it could even go the extra mile and allow the setting to be changeable dynamically, rather than statically set once and for all at startup time).
(It would be like each app have their own "swap" that can be dialed up and down... Except it would be completely different (grin), as there would be no write-back-to-disk involved, only reading... but you get my drift : with that scheme, if an app has runaway memory usage, it's free to use this on-demand cache and slow itself, without slowing down the rest of the system ; it kind of meshes with the "each component have their *own* vfs and their own ram disk" philosophy).
And now with my API user hat on, providing a generic system which would be an 'answer' to things like mmap() or virtual-memory swapping used on other OSes but with the usual Genode benefits (guaranteed protection against malicious client code etc) sounds quite good.
So, I guess, something for the 2025 roadmap?, if it won't fit on the 2024 roadmap, would be my tentative encouragement ^^.
P.S. As a (quite marginal) secondary point, that hypothetical new infrastructure might also help in my future/long term project to port the Hpkg package format to Genode (i.e. the don't-extract-contents-to-disk-but-uncompress-on-the-fly-to-ram-instead virtual FS concept that I've grown addicted to since the mid-2000s).
Cedric
Hi Cedrik,
On 2024-01-17 09:42, Cedric At TTS via users wrote:
From my vantage point, I find that idea (on-demand fetch) quite intriguing. If I understand it correctly, it seems this concept would end up enriching the Genode API with yet another "forte" (strong point), the same way that things like RM, and the super modular vfs system, and many other things, empower us when writing code for Genode.
If I put on my 'syst. integrator' hat, when crafting my system to run Falkon I found myself wondering where to set the bar for the cached_fs_rom RAM quota, so that it would be neither 'too little' nor 'too big/wasteful'. The prospect of just setting it to a 'low' value like 200M and letting the Genode class handle where and when to load/unload disk data to respond to read requests is enticing. (it could even go the extra mile and allow the setting to be changeable dynamically, rather than statically set once and for all at startup time).
that's exactly my line of thinking. Personally, I encountered the limitation of the current implementation when using the MNT-Reform laptop with only 4 GiB of RAM. Once I start the Falkon browser, all the heavy-weight ROMs (Qt, WebEngine...) are loaded, inflating the depot_rom (cached_fs_rom) as expected. Now, when switching away from the browser (e.g., selecting another preset w/o Falkon), I want the depot_rom to deflate so that I can use the RAM otherwise. But this does not happen unless I do a full "un-use" "use" cycle of the Sculpt partition.
Both, the capping of RAM usage of depot_ram as well as the inflating/deflating can be realized with the lazy-loading / on-demand-eviction idea. Now, the devil of implementing it lies in the details (sensible eviction strategies, async fs access, granularity of I/O), for which reason I don't want to hasten it.
P.S. As a (quite marginal) secondary point, that hypothetical new infrastructure might also help in my future/long term project to port the Hpkg package format to Genode (i.e. the don't-extract-contents-to-disk-but-uncompress-on-the-fly-to-ram-instead virtual FS concept that I've grown addicted to since the mid-2000s).
I love how you draw so many connections between our low-level ideas and your use cases when wearing the Haiku-hat (guessing you are a hat person?). :)
Cheers Norman
Thanks to everyone who participated in the brain-storming!
Based on the plentiful input, I've now come up with the following plan for this year:
https://genode.org/about/road-map
I hope that it captures well the variety of interests and ambitions expressed during our discussion.
Cheers Norman
Hi Norman,
This is very exciting roadmap! I just have a couple of questions:
1. Could you say a few words about "Application-level compositing using Genode's dialog API" (24.05)?
2. Does "Multi-monitor window management" (24.11) include support for multiple "workspaces"? (Actually, does it already support this, and I just didn't notice?)
3. During the discussion, I completely forgot about screen rotation on tablets (not necessarily auto-detecting). Given the current design, is that a difficult item?
Happy Sculpting!
John J. Karcher devuser@alternateapproach.com
Hi John,
- Could you say a few words about "Application-level compositing using
Genode's dialog API" (24.05)?
the idea is to use one application as widget inside another. I mentioned it near the bottom of the posting [1].
[1] https://lists.genode.org/pipermail/users/2023-December/008850.html
- Does "Multi-monitor window management" (24.11) include support for
multiple "workspaces"? (Actually, does it already support this, and I just didn't notice?)
When using Sculpt's window-manager preset, you can switch between multiple virtual desktops via Super-1 ... Super-9. By default, new windows appear at screen 1. To move a window to another screen, you have to edit rw/recall/window_layouter/recall/rules (changing the 'target' attribute of the respective window). Note that the rules file is persistent as it resides in the recall_fs. So the system remembers the window configuration across reboots.
- During the discussion, I completely forgot about screen rotation on
tablets (not necessarily auto-detecting). Given the current design, is that a difficult item?
We should certainly aim for it. But I cannot yet predict the best approach.
Cheers Norman
-
Alexander Boettcher -
Benjamin Lamowski -
Christian Helmuth -
Christian Prochaska -
Colin Parker -
Duss Pirmin -
Ivan Loskutov -
Jean-Philippe Ouellet -
Johannes Schlatow -
John J. Karcher -
Josef Söntgen -
Martin Stein -
Michael Grunditz -
Norman Feske -
Sebastian Sumpf -
Stefan Kalkowski -
Stefan Thöni -
ttcoder＠netcourrier.com