Major revision of the C runtime (VFS)

[Norman Feske]

Hello,

I have just merged a significant change of the C runtime into our
staging branch. Because this change will likely affect users of the
framework (hopefully for the better ;-), I'd like to bring up this topic
here.

Since many years, our C runtime served us quite well, thanks to the
plugin architecture that allows us to combine different backends such
the lwIP socket API using libc_lwip_nic_dhcp, using LOG as stdout via
libc_log, or using a ROM dataspace as a file via libc_rom. However, the
original design of the plugin architecture is starting to show its
limitations.

Examples of the current limitations:

* There is a libc_fs plugin that allows a program to access files
  from a file-system server. But there is no way to allow one
  program to access two different file-system servers.

* There is a libc_block plugin that hands out a block session as a
  pseudo block device named '/dev/blkdev'. However, when combined with
  libc_fs, it is not defined which of both plugins will handle the file
  with this name. As a quick and dirty work-around, the libc_fs plugin
  explicitly black-lists '/dev/blkdev'.

* In general, if multiple plugins are combined, there is no consistent
  virtual file system structure exposed via getdirentries.

* There is no flexible concept for handling stdio. Most programs use
  libc_log to direct stdout to the LOG service. But what if we want to
  direct the output of such a program to a terminal? Granted, there
  exists the 'terminal_log' server to translate a LOG session to a
  terminal session but it would be much nicer to have this flexibility
  at the C runtime level.

* Many libc plugins look similar. There are quite a few dusty corners
  where duplicated code has been accumulated over the years. That said,
  the semantic details (e.g., the quality of error handling) differ
  from plugin to plugin.

Seeing the number of file systems (and thereby the number of added libc
plugins) growing, it became apparent that our original design would not
scale too well.

On the other hand, we have gathered very positive experiences with the
virtual file system implementation of Noux. It allows to us have a
coherent VFS with support for stacked file systems (similar to union
mounts) of various types. The VFS of Noux is stable and complete enough
to accommodate our tool chain. Wouldn't it be a good idea to reuse the
Noux VFS for the normal libc? This line of thoughts led to the following
issue:

  https://github.com/genodelabs/genode/issues/999

The result of this line of work has been merged into the staging branch
now. I adapted most of the run scripts and targets to the changed libc
but certainly missed a few bits. I.e., the following run scripts remain
untested and are expected to fail in one way of another:

* http_srv_tracing_nonblocking_panda
* qt_avplay (the qt4 version)
* netperf
* l4linux_dynamic
* noux_gdb_dynamic
* debug_nitpicker
* noux_gdb
* virtualbox
* virtualbox_auto

I would appreciate your help to rectify them, so that we can merge the
staging branch to the master branch soon.

How has the libc changed?

Each libc-using program can be configured with a program-local virtual
file system as illustrated by the following example:

  <config>
    ...
    <libc stdin="/dev/null" stdout="/dev/log" stderr="/dev/log">
      <vfs>
        <dir name="dev">
          <log/>
          <null/>
        </dir>
        <dir name="etc">
          <dir name="lighttpd">
            <inline name="lighttpd.conf">
              ...
            </inline>
          </dir>
        </dir>
        <dir name="website">
          <tar name="website.tar"/>
        </dir>
      </vfs>
    </libc>
  </config>

Here you see a lighttpd server that serves a website coming from a TAR
archive (which is obtained from a ROM module named "website.tar"). There
are two pseudo devices "/dev/log" and "/dev/null", to which the
"stdin", "stdout", and "stderr" attributes refer. The "log" file system
consists of a single node that represents a LOG session. The web server
configuration is supplied inline as part of the config. (Btw, you can
try out a very similar scenario using the 'ports/genode_org.run' script)

The VFS implementation resides at 'os/include/vfs/'. This is where you
can see the file system types that are available (look for
'*_file_system.h' files). Because the same code is used by Noux, we have
one unified and coherent VFS implementation throughout the framework now.

There are two things needed to adapt your work to the change.

* Remove the use of the libc_{rom, block, log, fs} plugins from your
  target description files. Those plugins are no more. As of now,
  the VFS is still internally a plugin, but it is always included with
  the libc.

* Configure the VFS of your libc-using program in your run script. For
  most former users of the sole libc_log plugin, this configuration
  looks like this:

  <config>
    <libc stdout="/dev/log" stderr="/dev/log">
      <vfs> <dir name="dev"> <log/> </dir> </vfs>
    </libc>
  </config>

  For former users of the other plugins, there are the 'block', 'rom',
  and 'fs' file-system types available.

I hope that you will like the new VFS infrastructure. If you have any
questions, bug reports, or critique, feedback is always welcome.

Cheers
Norman

-- 
Dr.-Ing. Norman Feske
Genode Labs

http://www.genode-labs.com · http://genode.org

Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden
Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth