Roadmap 2021

[Tomasz Gajewski]

Hello everyone,

annual discussion about roadmap has been started a little earlier than
last year and that forces me to say a little about my current work
around Genode which is not yet ready to be published (possibly I need a
week or two). More about it later in summary of current year and my
plans for next one.


Summary of current year

In my roadmap for 2020 I stated two areas in which I'm going to
work. First was continuing work for supporting different rpi
devices. Unfortunately this work did not progress much as the blocker
issue with usb handling is still unresolved. At some point in time I
felt powerless and left it. At least until now noone else had motivation
to fight with it so this work is stalled.

Another direction was to try to setup a working environment for Genode
development in some IDE - possibly in Eclipse. I started some work in
this direction and quickly found out that to be able to configure some
kind of project with which IDE will be able to help with programming I
need to provide to it as much information about build as possible
especially source files, include paths, compilation flags. Without this
information even features like syntax highlighting and identifiers
completion cannot work. So I started to look into internals of Genode
build system to find a way to extract those informations.

During this process once again I realized why I don't like build system
based on make and especially after I found out that recursive make in
case of Genode build system can be written with capitalized "R" in
recursive :-). Maybe I'm just incompatible with make but trying to
understand details of it was so hard for me, that one more time I
started thinking if it would be possible have this build system
implemented/converted to scons, in what ways it would be better and
what would be worse.

Sidenote: During last Hack'n'Hike in 2019 we had a discussion about
scons and make and that decision was made to use make. I also read a
paper about comparison of make and scons for Genode and I don't really
know what concrete elements/features were that the decision was to
choose make.

In consequence I started my own project to create a scons based build
system for Genode. My plan was to create one and use it myself for rpi
development, fix all issues that I really dislike about current build
and verify if there are some problems that I don't see that would reject
the whole idea. Given the simplicity of most of makefiles in Genode I
did not expect that this small part will be complex enough that
implementation will take me half of the year.

The most important things I do not like in current build system are:

 * big recompilations after touching makefiles and after switching
   branches

 * cases when leftovers in build directory caused hard to diagnose build
   problems

 * lack of easy ways to debug what is going on inside make (or maybe
   it's just my inability to comprehend it)

My main goals were:

 * implement build in a way that would be able to use current build
   description with as little changes in Genode repository as possible
   (currently only two symlinks have to be created and one "marker" file
   in each build directory to specify that this build directory is built
   with scons)

 * implement a verification mechanisms to be sure that scons build is
   100% correct replacement of current build (I build database of
   commands from both types of build and compare them list of targets
   and that commands are identical - after some normalization)

 * remove all absolute paths in compilation commands - they are used for
   all paths in current build - and they are bad for reproducible builds
   (which should be a goal for a project at some point in time), bad for
   using ccache or similar tools and bad for defining a project in IDE

 * be able to understand and retrieve all informations about build to be
   able to use it in external tools ide and possibly other like static
   analyzers

Current state and missing parts:

 * above goals are achieved for build of almost all target on linux
   build on sources from release 20.08 with repositories enabled by
   default (base, base-linux, os, demo). First attempts for arm6 shown
   only some minor inconsistencies in command flags so it should quickly
   go forward

 * missing any installation and usage documentation

 * need to verify on 20.11 release and current master

I hope to finish those parts quickly and make it public this year.



Plans for next year

I plan to further develop build system based on scons at least until it
fullfills my own whishes about it or until I find some blocker that I
won't be able to overcome. As originally I plan to use it for
development for rpi. I know that I'll have to add some support for ports
(I plan only wrapper for current descriptions), depots and possibly
something more.

Depending on results of this experiment and reception of Genode team and
community I will throw it away, use it myself or maybe develop further
features and fixes - I'm curious how it will be.

On the rpi front I still plan to work on it. I plan some work on rpi4
(which is not affected by current usb blocker) and hopefully other
variants too.



Comment on Norman's plans

All of your plans are really great.

Plan on supporting PinePhone is definitely appealing for broader
audience. I'll probably buy one in case of success and see how it works.

Optimization attempts also seem to be a great target. Improving
efficiency will probably further improve code quality and will require
more tooling that will allow to constantly test for more kinds of
regresssions - especially those related to efficiency.

But for me ideas around impremovements of DDE seem to be most
interesting. Especially that I had my own ideas that I queued to explore
for myself. During driver porting attempts I felt that amount of code
that has to be duplicated (even though it was just dummy stubs) was too
big. If I refresh this knowledge during work on rpi I'll definitely like
to participate in discussion about it. Porting device drivers from Linux
seem to me a very important aspect.


Regards
Tomasz Gajewski