[HEADS UP] Refactored run tool on staging

[Josef Söntgen]

Hello,

tl;dr we merged the modularized version of our run tool to the staging
branch. This version requires changes in the build directory, e.g.,
changes to 'etc/build.conf' and it is therefore advised to remake
all used build directories if one uses the current staging branch.
There is also 'tool/run/README', which explains the usage and the
architecture in more detail. In addition you can take a look at the
corresponding issue [1].


The run tool served us well over the years and by continously
extending its functionality it met our daily requirements. The way
it was extended up to now, however, was in an ad-hoc fashion that made
future changes and maintaining more time-consuming. We addressed these
problems by refactoring the run tool:

* Modularizing the run tool: each step (building/loading components,
  execution scenarios and so) is now implemented by a module. All of
  the common use cases, that were handled by the old run tool, are now
  covered by combining these modules. This makes the actual run tool
  more managable and makes future extensions easier.

* Unifying the argument handling of the run tool: each module is
  configured explicitly in the same fashion. That makes configuration
  failures more visible.

To just give an example on how the interface of the run tool changed,
here is a comparison of the old way and the new way to handle a common
use case:

In this case, we want to reset the target system and caputure the LOG
output via AMT. The target system loads all files it needs to execute
the scenario via TFTP:

* Old (changes in nova_x86_64/etc/build.conf):

RUN_OPT                     = --target amt --tftp-absolute

export AMT_TEST_MACHINE_IP  = 10.23.42.51
export AMT_TEST_MACHINE_PWD = secret
export PXE_TFTP_DIR_BASE    = /var/lib/tftpboot
export PXE_TFTP_DIR_OFFSET  = /x86

* New (changes in nova_x86_64/etc/build.conf):

RUN_OPT       = --include boot_dir/nova

AMT_HOST      = 10.23.42.51
AMT_PASSWORD  = secret
RUN_OPT      += --include power_on/amt \
                   --power-on-amt-host $(AMT_HOST) \
                   --power-on-amt-password $(AMT_PASSWORD)
RUN_OPT      += --include log/amt \
                  --log-amt-host $(AMT_HOST) \
                  --log-amt-password $(AMT_PASSWORD)
RUN_OPT      += --include load/tftp \
                  --load-tftp-base-dir /var/lib/tftpboot \
                  --load-tftp-offset-dir /x86 --load-tftp-absolute

The new way is arguably more verbose but also makes the configuration
more clear, which is more important in our opinion. In addition, it is
possible to capture the LOG output, i.e., via an USB-serial adapter by
replacing 'log/amt' with 'log/serial'. That is something, that was more
difficult to achieve with the old tool.
When creating a new build dir, a default RUN_OPT configuration is added
to the generated 'build.conf'. In most cases this means creating an ISO
image and starting the scenario in Qemu like the old tool did in the
default configuration.

As I have said before, there is a README file, that hopefully provides
all information that is needed to get you started with the new tool. If
you think there is something that is missing from this documentation,
please drop a note either via E-Mail or github.


Cheers
Josef

[1] https://github.com/genodelabs/genode/issues/1355