Hi Norman-
I gota hand it to you, for the heavy weight Genode ABI work your pulling together! I'm sure its not easy keeping track of everything staying coherent in the context of Genode's dynamic linker. Then tis is exactly the kind of work that provides the foundation for a production strength OS.. and that seems to be Genode's future..
you mention: "** The packaging of kernel-agnostic binary packages has become within **close reach now.*" it would be great to hear you current reflection as to how a unified, repeatable build src / binary package management system might help keep the Genode OS package versioning well organized..
-Peter
On Fri, Jan 13, 2017 at 7:30 AM, Norman Feske <norman.feske@...1...> wrote:
Hello everyone,
I am happy to report that we wrapped up the first phase of our big API adaptation work today and introduced the unified build directories into the master branch. This posting is both a heads-up warning and an invitation to experiment with the new mechanism.
First, the warning part (quite literally):
With the commit "Warn about the use of deprecated env() function" [1], a compiler warning will appear each time the legacy 'env()' function is used, either explicitly or implicitly. This makes it easy to spot the places that need our attention because they still rely on global side effects. Since New Year, we have been very busy to overhaul our code base and made huge progress. Thanks to everyone involved! However, as you will see, there are still many places left unattended. Therefore, the warnings somehow spoil the impression when building Genode right now. Their main downside is that build errors and important warnings are more difficult to spot. Please bear with us. ;-) Don't worry though, the "deprecated" warnings are merely cosmetic and will gradually disappear.
[1] https://github.com/genodelabs/genode/commit/a7f40b24cad0a34f0ac0bd9256a528 5a6bf7cd94
Second, I proudly present you the fruit of my labor with respect to the Genode ABI. Genode has now become able to use different kernels from within the same build directory. Of course, this change comes with a sight change of the tooling, in particular to the 'create_builddir' tool, the build system, and the autopilot tool. Let's take the new version for a quick test drive.
Checkout the current master branch of Genode.
git clone https://github.com/genodelabs/genode.git
Invoke the 'create_builddir' tool to see the new options of build directories.
cd genode ./tool/create_builddir
As you can see, the original kernel-specific platform parameters are still there but they are marked as "deprecated". They will be removed shortly. The new unified build directories are the new way to go now. There is one option for each supported hardware platform, e.g., 'x86_64', or 'usb_armory'. Note that the kernel is left unspecified. Assuming you have a 64-bit Linux host system, let us create a build directory that matches your Linux system:
./tool/create_builddir x86_64
(if you are using a 32-bit Linux system, specify 'x86_32' instead) Now, enter the build directory and look at the build configuration:
cd build/x86_64 <editor> etc/build.conf
You may notice the changed "Run-tool configuration" section. Parts this configuration are dependent of a new 'KERNEL' variable. The variable has the following effects:
- It selects kernel-specific run-tool arguments 'KERNEL_RUN_OPT',
- It selects kernel-specific Qemu arguments, e.g. 'QEMU_OPT(nova)'
- It adds the kernel-specific 'base-<kernel>' directory to the list of source-code 'REPOSITORIES'.
Once you have your build.conf open in your editor, you may enable parallel build by uncommenting the corresponding line at the beginning of the file.
Let's build a Genode component, e.g., init:
make init
Wow, that was quick! The build system created the init binary without even building any parts of the Genode base system. How is that possible? The answer lies at 'repos/base/lib/symbols/ld', which is the binary interface provided by Genode's dynamic linker. The build system uses this information to build a pseudo shared object that is linked against init instead of the real dynamic linker. From the perspective of the init executable (just put yourself into the shoes of an ELF executable to follow me), it feels like being linked against Genode's dynamic linker that provides the base framework. But in reality, you got presented just an empty hull of Genode - a bunch of symbols. In case you are curious, you can find this ABI stub library at var/libcache/ld/ld.abi.so. If you look closely into the build subdirectory of init/, you will see that the used 'ld.lib.so' is actually a symlink to the stub.
At this point, the build directory is clear from any kernel- specific peculiarities.
Building a kernel
To build a kernel, the build system needs to know the kernel- specific parts of Genode, i.e., the corresponding 'base-<kernel>' source code repository must be added to the 'REPOSITORIES'. The default build configuration does this automatically if you specify a 'KERNEL' argument:
make kernel KERNEL=nova
At this point, the build system may point out that you will need to download the kernel source code. Just follow the proposed command to do these preparatory steps. If done, execute the above command again. In the particular case of NOVA, the result of the kernel build can be found at 'kernel/nova/'.
Executing a scenario on Linux
Let's build and execute Genode's graphical demo scenario on your Linux host system:
make run/demo KERNEL=linux
The build will take a while. Most of the produced components are kernel-independent. Only a few, such as core, fb_sdl, and the timer driver are Linux-specific. Also a Linux-specific version of Genode's dynamic linker will be built. It is named 'ld-linux.lib.so'.
Once the build completes, the scenario is executed. So you should see a window with the demo scenario popping up.
Switching to another kernel
Now, let's execute the same scenario on, let's say, NOVA. To use this kernel instead of Linux, a few actual drivers are needed. So let us enable the 'libports' and 'dde_linux' repositories in the build configuration.
At this point, you may have guessed the command to use:
make run/demo KERNEL=nova
You will be prompted to prepare a few ports of 3rd-party software, in particular, the libc, dde_linux (USB driver), x86emu (for VESA). After installing those ports, issue the above command again.
Note that the build process will build the additional drivers and NOVA-specific parts of Genode. But all the components that we built before when executing the scenario on Linux remain untouched now.
At the end of the build process, the scenario is stared in Qemu.
To sum it up, switching from one kernel to another has become just a matter of the 'KERNEL' argument. This has a number of benefits:
When using multiple kernels, there is no need to have multiple build directories. You waited an hour to build a Qt5-based scenario on NOVA but want to run it on Linux now? This is now possible in a few seconds.
If an executable has a bug, the bug will be there regardless of the kernel being used. To debug the problem, you can use the kernel with the most appropriate debugging instruments available.
The packaging of kernel-agnostic binary packages has become within close reach now.
The new ABI feature that works so well for Genode's dynamic linker can actually be leveraged by arbitrary libraries. For example, I just introduced an ABI for the libc (libports/lib/symbols/libc). So in the future we can even have multiple levels of ABI stability.
I hope that you will like the unified build directories. This being a very recent development, there may still be rough edges. Your feedback is very welcome. I hope that we will have smoothened those edges and removed most of the "deprecated" warnings by the time of releasing version 17.02 at the end of February.
Cheers Norman
-- Dr.-Ing. Norman Feske Genode Labs
http://www.genode-labs.com · http://genode.org
Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
Developer Access Program for Intel Xeon Phi Processors Access to Intel Xeon Phi processor-based developer platforms. With one year of Intel Parallel Studio XE. Training and support from Colfax. Order your platform today. http://sdm.link/xeonphi _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main