build directory outside of Genode dir xor vim error

[Norman Feske]

Hi Paul,

> On foc?
> The make prepare was so long, that I decided I did not want to try to
> build that.
> The make prepare seems to prepare vim, download uclibc (with a libc
> that seems to prepare to run on Linux and others). At first, seeing
> some good documentation was fun. But seeing it download all the stuff
> in base-foc/contrib/l4/pkg make me afraid.
> I really don't understand why all that is needed for Genode.

frankly, it isn't. Genode only uses three tiny bits of that code, namely
Fiasco.OC (the kernel), bootstrap (a small program only used at boot
time), and sigma0 (a simple manager for physical memory). Unfortunately,
the latter two components depend on a few header files and library
functions of the other l4/pkg packages. For example, bootstrap uses
string functions provided by uclibc. Hence, uclibc must be downloaded
(and built) prior compiling bootstrap. But since there is no option to
only build the parts of uclibc that are actually needed by bootstrap,
the whole package is built. I agree that it feels a bit awkward to
download and build all the stuff just for using a microkernel. However,
the Fiasco.OC developers see no value in decoupling the Fiasco.OC kernel
(and bootstrap and sigma0 respectively) from their L4re userland. So
they leave it as is. Of course, we could fork it and cut off those
dependencies. But then, we needed to maintain the fork. So we can decide
whether to live with the inconvenience of downloading and building too
much stuff, or to maintain a fork. Up to now, we decided against the fork.

> On nova:
> paul at ...162...:~$ cd build.nova32
> paul at ...162...:~/build.nova32$ make run/noux_bash
> using run script --include

This looks like your build directory is somehow broken. Haven't you
built the run/demo for NOVA before? I wonder what happened to your build
directory. Please try again, taking one step after the other.

1. use tool/create_build_dir nova_x86_32 to create a fresh
   build directory
2. try out run/demo
3. enable the libports, ports, and gems repositories in your
   <build-dir>/etc/build.conf
4. try out run/noux_bash

If one of those steps fails, check back with me.

> On Pistachio.
> I liked the make prepare that got 5.6 MiB, was ok.
>   Program core/x86/core
> make[1]: Leaving directory `/home/paul/build.pistachio'
> genode build completed
> cp: cannot stat
> `/home/paul/build.pistachio/l4/util/kickstart/kickstart': No such file
> or directory

It seems that your build directory is in an inconsistent state. I guess
you have tried to do run/noux_bash, then noticed that your forgot to
prepare the base-pistachio repository, and then after preparing, tried
run/noux_bash again. To be on the safe side, please do a 'make cleanall'
in your build directory after preparing a kernel.

I will try to explain what is going on. The run tool does not build the
kernel on each run. Checking those dependencies would take a really long
time on some platforms such as Fiasco.OC. This time delays the workflow
but since the kernel rarely changes, it is largely wasted. Therefore,
the run tool issues the build of the kernel only once. If it detects
that there is already a 'kernel' directory, it skips building the
kernel. In your case, the first attempt to build pistachio failed but it
nevertheless left an incomplete 'kernel' sub directory in your build
directory. So the next time, you called run/noux_bash, the kernel wasn't
rebuilt.

I don't know a good solution of how to improve the situation. We could
refine the heuristics but they will still not capture all possible cases.

> And make run/noux_bash on okl4:
>     COMPILE  main.o
> In file included from /home/paul/mygenode/base/src/core/main.cc:24:0:
> /home/paul/mygenode/base/src/core/include/core_env.h:121:33: error:
> cannot declare field ‘Genode::Core_env::_rm_session’ to be of abstract
> type ‘Genode::Core_rm_session’

You are using staging, aren't you? This error is a good reason not to do
that.

Let me explain the role of the staging branch. The only user of this
branch is the automated building and testing infrastructure at Genode
Labs. Before we incorporate changes into the master branch, we put them
into the staging branch. Each night, this branch gets automatically
built for all base platforms and a series of automated tests is executed
(those run scripts listed at 'tool/autopilot.list') on almost all of the
kernels. Only if those build and runtime tests succeed, we merge those
chances into the master branch. Consequently, the staging branch is
almost expected to fail when doing complex changes that affect different
base platforms. If this happens, we fix up the corresponding commits in
the staging branch until all tests succeed.

In short, you hit a typical issue that will be fixed shortly at the
staging branch.

> While trying the different 'platforms' (kernels), the build system
> seems a bit unoptimized.
> To me, rebuilding libports, ports, gems in each different kernels
> don't make much sense.
> I don't expect them to apply patch platform-specifics, only
> Genode-specifics.
> So it would make sense to specify two directory on create_builddir: 
> one common for all platforms, and one for the platform.

That would be cool but it is not possible.

Even though those libraries look like being the same for each base
platform, they are different. At compile time, those libraries
ultimately depend on platform-specific header files (e.g., the kernel's
syscall bindings). So a library built for one kernel will typically not
work on another kernel because it would try to call kernel functions in
wrong ways. In theory, all those kernel-specific things could be
encapsulated in a single kernel-specific library with an interface that
hides all kernel details. But in practice, this would be extremely
difficult and the extra indirection needed for that would certainly
imply runtime overheads. In my opinion, the little added value of such a
feature does not justify opening a big can of problems.

I hope, my explanations are helpful to shed some light on how the build
system works. Thanks for your patience.

Cheers
Norman

-- 
Dr.-Ing. Norman Feske
Genode Labs

http://www.genode-labs.com · http://genode.org

Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden
Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth