ARM TrustZone Shared Memory Crossworld

[Martin Stein]

Hi Stefan,

Am 09.12.2015 um 17:15 schrieb Stefan Brenner:
> 1) We use mmap(NULL, <size>, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANON,
> -1, 0) to allocate the buffer in normal world (Linux). Is there a better
> way to allocate uncached memory?

The only way I found, when working at Genode's TrustZone VMM on USB
Armory, was by allocating DMA memory. As far as I know, Linux doesn't
provide allocating non-cached DMA regions in the userland. However, you
can do this inside the kernel. You may have a look at the initialization
of genblk_buf in my para-virtualized block driver [1]. The resulting
address is used for shared RAM between secure and normal world. The
secure back end of the driver can be found in [2].

> 2) We've tried __cpuc_flush_user_all(), __cpuc_flush_kern_all() in order
> to invalidate all cache lines (later maybe only the affected cache lines
> once this works in general). Is this correct, to do this in normal world
> (i.e. Linux)?

I do not know whether this is sufficient, but recently there was a post
with the same idea [3] and maybe the author got it working at the end.
In the corresponding thread I also suggested a general design for memory
synchronization caused from within the VMM. This is useful when not
having non-cached memory _and_ the VMM is the transfer initiator. But I
assume that the latter is not the case in your scenario.

> 3) At the moment we assume that Genode doesn't have to invalidate/flush
> its cache lines because as it is the VMM it should be aware of the fact
> that the cache might be stale.

I'm afraid I do not understand what you mean. A RAM region shared
between the TrustZone worlds must always be located in non-secure RAM.
In the tz_vmm, the whole non-secure RAM is mapped as IOMEM [4], which is
- using ARM-speak - tagged as "Device Memory" and thus non-cached. So,
yes, the tz_vmm must not flush caches as it should always use these
non-cached mappings.

> Any help is appreciated. A short hint: Currently our idea is to send an
> hypercall to the Genode VMM after Linux has written to the buffer, so
> Genode will pretty quickly access the memory after it has been written
> to from Linux.

This is exactly what is done in the para-virtualized block driver
mentioned above after a write request has been set-up in the buffer [5].

Cheers,
Martin

[1]
https://github.com/m-stein/linux/blob/genode_hw_usb_armory_tz_vmm/drivers/block/genode.c#L338

[2]
https://github.com/genodelabs/genode/blob/master/repos/os/src/server/tz_vmm/block.cc

[3] http://sourceforge.net/p/genode/mailman/message/34667551/

[4]
https://github.com/genodelabs/genode/blob/master/repos/os/src/server/tz_vmm/include/vm_base.h#L69

[5]
https://github.com/m-stein/linux/blob/genode_hw_usb_armory_tz_vmm/drivers/block/genode.c#L204