Benchmark Armv8 VMM (Stefan Kalkowski)

[Chris Hofer]

Hi Stefan,

Hi Chris, On Wed, May 06, 2020 at 01:23:59PM +0200, Chris Hofer wrote:

> Hi,
>
> to measure the performance overhead of the Armv8 Genode VMM, I ran some
> benchmarks (cyclictest and lmbench). I compared it against a native Linux
> installation on the Imx8-evk board (same kernel version). For some
> measurements (for example fork + executing /bin/sh from lmbench), I observe
> counter-intuitive results in the sense that the virtualized version runs
> *faster*  than the native one (Vm 1,1ms vs. native 2,5ms).
>
> My understanding is, that the virtual timer offset (cntvoff_el2) is not used
> and thus, the guest sees wall clock time. I would expect that the
> virtualization incurs an overhead and this overhead manifests itself in
> longer (wall clock) time measurements.
>   
>> Correct, but you assume that native Linux and virtualized version run
>> with the same frequencies (cpu + timer) - don't you?
>>
>> Actually, I think the timer frequency should be constant in both
>> scenarios, whereby at least the cpu will be controlled dynamically in
>> the native Linux case.
>>
>> When comparing such complex scenarios to avoid mistakes I would
>> strongly encourage you to not measure time on-target. It often leads
>> to wrong assumptions. It is better to measure long-term scenarios from
>> another host. If you need to measure micro-benchmarks, you might loop
>> the operation quite often and measure the whole time-span outside, and
>> divide the result.
>>
>> Alternatively, if you insist to measure on-target, you can instrument
>> Linux to use a constant CPU frequency (google for "linux cpu
>> governor"). Moreover, you might check whether the timer frequency is
>> the same in both settings (CNTFRQ_EL0) - well it should be.

Thank you for this advice; it makes perfect sense to capture the time 
off-target.


I finally found a big mistake in my setup to measure the duration of 
forking a process and calling /bin/sh. The native installation uses a 
dynamically linked shell and the guest has a statically linked binary. 
After using statically linked binaries on both systems I get reasonable 
results.


Nevertheless, I will take advice and verify the measurements off-target.


Best,

Chris

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.genode.org/pipermail/users/attachments/20200507/67fa1a42/attachment.html>