Hi Emery,
In what started as an effort to fix some simple problems when building Genode on NixOS, I have gone ahead and attempted to replace the Make build system with one implemented in Nix.
that is a seriously cool experiment - thanks for sharing it!
To start out with, here are some problems with a Nix build system:
- Nix must be ported to Genode if Genode shall be self hosting.
I was planning to investigate the use of Nix as a package-management solution on Genode. Porting Nix to Noux (sounds a bit funny) will be needed then anyway. So I don't see this as a problem.
- Each Make target will probably have to be re-expressed in Nix.
- Nix produces more arcane output than Make.
Admittedly, I haven't considered Nix as a build-system replacement because the current make-based build system works quite well. I could not see the added benefits that Nix would bring as a build tool. Maybe, your experiment will be eye-opening in this respect. :-)
One point not to forget is that Nix adds a new concept to the whole picture, which is alien to must people but must be understood by everyone exploring Genode then. Genode is already quite difficult to approach. I am a bit afraid that the mandatory use of Nix would further raise the bar. Maybe my fear is unfounded and Nix would actually simplify the status quo?
Some benefits as I see it:
- Nix is the only build system dependency.
To see this value, I'd first need to understand the dependencies of Nix itself. But coreutils + findutils + bash + make is not a very complicated software stack either, don't you agree?
- More build parallelism.
Is this really true? When enabling parallel builds in the Genode build system (by adding 'MAKE += -j4' to the 'etc/build.conf' file) the actual build stage gets executed in parallel already, managed by make. Are you sure that Nix can squeeze more performance out of the build process? I am a bit skeptical because of the following blog posting (the section about work):
http://allsoftwaresucks.blogspot.de/2014/06/minor-notes-about-recent-stuff.h...
- Lazy dependency evaluation means faster rebuilding.
- Binary component packages.
- NixOS can be nested in the Genode build system.
- Genode can be nested in the NixOS build system.
Those points sound very good, especially the possible integration with NixOS.
I'd like to note that the Genode build system actually employs an optimization to accelerate rebuilding. At the first stage, the build systems discovers the subset of libraries needed for the actual build and generates a makefile for those (the file is generated at 'var/libdeps'). The second stage executes all the targets contained in the libdeps file but omits everything else. Effectively, this mechanism prunes the dependency graph to those portions that are really needed. To accelerate the rebuild time, the first (lib-dependency-generating) stage can even be omitted by issuing 'make again'.
- Less boilerplate.
- Less user interaction.
Can you please substantiate those points a little? Are you referring to the installation of the tool-chain?
That said, my conversion is far from complete, my focus has been on getting the Linux and Nova implemenations working. I have core and init building for Linux and Nova as well as a number of tests. The Expect framework for executing Genode has been wrapped and works for Linux, but I will have to express the Nova kernel before moving on to Qemu support. I've been bypassing ARM to avoid getting into the details of the spec managment for now.
If anyone would like to try it out I'm working on the 'nix' branch at https://github.com/ehmry/genode.git
The only depency is Nix which you can get here: http://nixos.org/nix/
There is a script named 'try_me.sh' that will run the tests that have been defined.
If you wish to explore further the command that you will need to get aquainted with is 'nix-build'. For example, to build some components: nix-build ~/genode/specs/x86_64-nova.nix -A base.core nix-build ~/genode/specs/x86_64-nova.nix -A os.tests nix-build ~/genode/specs/x86_64-linux.nix -A run.thread
Nix-build will leave a symlink named 'result' in your current directory that contains the output of the Nix expression you call.
To remove the outputs that Nix has built, delete the result symlink and run 'nix-store -gc' to invoke the garbarge collector.
If you first want to know what Nix will do when building a run from scratch, here is a summary:
Install dependencies for the Nix build environment, Nix will likely not use anything in your current PATH.
Download and patch the Genode toolchain for the Genode build environment.
Compile the sources of each dependency into object files.
Merge objects into libraries.
Link libraries into applications.
Execute Genode and leave the build output and a log of the run in a directory in the Nix store.
If there are any questions I will be lurking as 'emery' in the IRC channel on Freenode.
Thank you for having taking the time to document the steps in such detail.
Right now, I'm busy with the upcoming Genode release. Once it is out, I'm looking forward to experiment with your branch to learn more about the possibilities of combining Nix with Genode.
Cheers Norman