base-hw: Virtualbox thread priorities

[Adrian-Ken Rueegsegger]

Hello Martin,

On 07/07/2015 03:26 PM, Martin Stein wrote:
> * For comparison with NOVA, it might be a good idea to assign 100% of
> the CPU quota, because then, priorities are absolute also on base-hw.
> Later, you may switch the best-effort parts of the scenario back to a
> non-quota mode and see if the performance benefits from it.

I have adjusted the virtualbox.run scenario accordingly and split the
CPU quota between the components.

> * For a better debugging basis it might be useful to get the CPU
> utilization stats of the scenario. This can be achieved in the kernel
> pretty simple and unadulterately. You can find a basic implementation
> and demonstration on my branch [2] (all .* commits). The output is
> triggered via Kernel::print_char(0) that can be called from everywhere
> in the userland but you may also print it e.g. on console-input IRQ
> through THREAD_QUOTA_STATS_IRQ in the last commit. The printed "spent"
> values are the timer tics that a thread has used for its own or spent
> helping another thread. The printed "used" values are the timer tics
> that the thread was really executed (on its own or through a helper).

Based on your cpu quota stats commits I made some minor adjustments so
the output is in CSV format [1], which makes it more suitable for
processing by other tools. I wired the stats print to the keyboard IRQ.

The first set of numbers for the top 10 threads were collected when
pressing a key in the guest VM bootloader. So they basically cover the
base-hw/Virtualbox init and start:

pd_label             | label        | spent      | % of total
---------------------+--------------+------------+-----------
init -> virtualbox   | EMT          | 5283917631 | 39.8206%
init -> virtualbox   | virtualbox   | 4629194169 | 34.8865%
init -> rtc_drv      | rtc_ep       | 1439510978 | 10.8484%
init -> rtc_drv      | rtc_drv      |  698105922 |  5.2611%
init -> fb_drv       | vesa_ep      |  221176830 |  1.6668%
init                 | init         |  150973892 |  1.1378%
init -> timer        | timer_drv_ep |  142735932 |  1.0757%
init -> ps2_drv      | ps2_drv_ep   |  132941808 |  1.0019%
init -> ps2_drv      | ps2_drv	    |  125508496 |  0.9459%
init -> platform_drv | platform_drv |  107468792 |  0.8099%

The second set was collected once the Linux system has booted to the
command prompt:

pd_label           | label             | spent       | % of total
-------------------+-------------------+-------------+-----------
init -> virtualbox | EMT               | 97132063560 | 89.9964%
init -> virtualbox | virtualbox        |  4631019033 |  4.2908%
init -> timer      | timer_drv_ep      |  2135808255 |  1.9789%
init -> rtc_drv    | rtc_ep            |  1439510978 |  1.3338%
init -> rtc_drv    | rtc_drv           |   698105922 |  0.6468%
init -> virtualbox | Timer             |   313374324 |  0.2904%
init -> fb_drv     | vesa_ep           |   238903418 |  0.2214%
init -> virtualbox | alarm-timer       |   207690920 |  0.1924%
init -> timer      | timeout_scheduler |   199926872 |  0.1852%
core               | pager_activation  |   155622972 |  0.1442%

So the overwhelming majority of CPU time is spent by the Virtualbox EMT
thread, which looks sensible. I assume that the time spent in hwaccel
mode is all accounted to the EMT thread since it is the one that invokes
Vm_session::run.
I do not know the implementation details of rtc_drv but I did not expect
it to make the top 10.

A sidenote: The difference between quota used and spent (total used -
total spent) is 7029216 ticks. This value is identical for each measurement.

> According to my above writings, I think that we can still raise the
> performance through modifications at the application level. It would
> also be helpful to see your CPU-utilization stats if the scheudling
> remains a problem.

I experimented a bit with the CPU quotas in the virtualbox.run scenario
by giving Vbox 70/80/90% and dividing the remaining percentage among the
other components. The execution was only slightly faster with more CPU
time but the time differences were quite small in the range of one
second (compared to the total time of 33-35seconds).

Do you see anything suspicious with these numbers or a potential for
further optimization?

Regards,
Adrian

[1] -
https://github.com/codelabs-ch/genode/tree/hw_x86_64_muen-vbox-cpu_quota_stats