Hi,

On 04/28/2015 02:26 PM, Stefan Brenner wrote:

Hi,

just a short question, I am using the TZ VMM example on i.MX53. How can I syncronize accesses to a shared memory range between secure world and normal world? I would be interested in a mutex that can be acquired both from Genode in secure world, as well as from Linux in normal world.

From the secure world's side, the synchronization is somehow implicitly

available. Whenever the VMM receives a VM exception signal, it knows that the VM is in pause state and cannot intercept it. Moreover, if the VMM receives other asynchronous events (e.g. events from some backend devices etc.) it can call pause() on the VM object. If the pause() call returns, the VMM can be sure that the VM is not scheduled anymore. By calling run() after a critical path the VMM makes the VM executable again.

From the normal world's perspective you might ask the secure world via a

hypercall explicitly when entering or leaving a critical section.

Of course, you can also implement an inter-world capable mutex laying within a shared memory region that is mapped uncached in both worlds. By now we did not need such a mutex for our experimental VMM. Therefore, there is no implementation existent.

Regards Stefan

Ciao

---

One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight. http://ad.doubleclick.net/ddm/clk/290420510;117567292;y _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main

Hi Stefan,

It would be more suitable for my app to not do the critical action in normal world, but ask the secure world to do it. Then it would be synchronized as well, but only one world switch back and forth necessary.

I think I have to bite the apple and implement what you mention, a mutex in a shared memory section. Do you have any further input for me according to this? Especially about the "mapped uncached" property?

I hope you don't mind me chiming in. I doubt that mutex-based synchronization of a shared buffer between both worlds is the best way forward. Instead, I would investigate a protocol similar to Genode's synchronous bulk transfer approach as described in Section 3.6.5 of the upcoming manual [1].

In short, while the normal world is active, it can freely access the memory without the need for synchronization. To hand over the data to the secure world, the normal world performs a secure method call (smc) to the secure world. While the secure world is active, the normal world cannot (by definition [2]) be active. So the secure world can safely consume the data (e.g., by copying out the data from the shared buffer to a private buffer). Once the secure world is finished, the smc call returns, which implicitly hands back the "ownership" of the shared buffer to the normal world. Would that approach possibly work for you?

Cheers Norman

[1] http://genode.org/files/53bcb8e33fe6602fed25edc3c7b922c5/manual-2015-04-27.p... [2] On the multi-processor system, this may not be true. Here, the normal world would need to perform the synchronization of smc calls. But this synchronization does not involve the secure world.

On 06.05.2015 10:26, Norman Feske wrote:

Hi Stefan,

...

It would be more suitable for my app to not do the critical action in normal world, but ask the secure world to do it. Then it would be synchronized as well, but only one world switch back and forth necessary.

I think I have to bite the apple and implement what you mention, a mutex in a shared memory section. Do you have any further input for me according to this? Especially about the "mapped uncached" property?

I hope you don't mind me chiming in. I doubt that mutex-based synchronization of a shared buffer between both worlds is the best way forward. Instead, I would investigate a protocol similar to Genode's synchronous bulk transfer approach as described in Section 3.6.5 of the upcoming manual [1].

In short, while the normal world is active, it can freely access the memory without the need for synchronization. To hand over the data to the secure world, the normal world performs a secure method call (smc) to the secure world. While the secure world is active, the normal world cannot (by definition [2]) be active. So the secure world can safely consume the data (e.g., by copying out the data from the shared buffer to a private buffer). Once the secure world is finished, the smc call returns, which implicitly hands back the "ownership" of the shared buffer to the normal world. Would that approach possibly work for you?

Thanks for your answer. While in short term we are actually working with imx53 which is a single core your assumption holds of course. Still I am worried about the impact of IQs and FIQs that might disturb us while consuming or writing data in both (secure and normal) worlds. In long term we would like to even move on to a multi core plattform.

So as far as I see it, I think in our case it is unfortunately not that easy. We can not just assume implicit synchronisation through TrustZone being either in secure or normal world at a time, but interrupts (and later the multi core arch) will make our life harder. Please correct me if I am wrong here (would be happy to be wrong, to be honest).

The only thing I see here is that interrupts - even though they will actually happen - dont affect our shared memory. Probably depends on what the ISR is doing, if it will just finish its job and return back to the world running before the interrupt happened, then it might be fine. No?

Cheers Norman

[1] http://genode.org/files/53bcb8e33fe6602fed25edc3c7b922c5/manual-2015-04-27.p... [2] On the multi-processor system, this may not be true. Here, the normal world would need to perform the synchronization of smc calls. But this synchronization does not involve the secure world.

On 10.05.2015 19:32, Norman Feske wrote:

Hi Stefan,

...

Thanks for your answer. While in short term we are actually working with imx53 which is a single core your assumption holds of course. Still I am worried about the impact of IQs and FIQs that might disturb us while consuming or writing data in both (secure and normal) worlds. In long term we would like to even move on to a multi core plattform.

So as far as I see it, I think in our case it is unfortunately not that easy. We can not just assume implicit synchronisation through TrustZone being either in secure or normal world at a time, but interrupts (and later the multi core arch) will make our life harder. Please correct me if I am wrong here (would be happy to be wrong, to be honest).

I don't think that interrupts are a concern here. As long as the tz_vmm is active with processing an SMC, normal-world interrupts are masked. The normal world will receive its next interrupt not before tz_vmm has finished its work.

Well, it seems like you are assuming a different system design than we actually have. I don't say our design is the "holy grail", but let me describe how things currently work:

(secure world=Genode, normal world=Linux) 1) normal world allocates a buffer in normal world memory 2) normal world issues an SMC and transmits the buffer's address to secure world where addresses are translated 3) secure world stores the address 4a) normal world writes something to the shared buffer and modifies some meta data structures. Basically, it writes to the metadata "Hey, secure world, there is something for you". 4b) secure world busy-waits (ugly) until it reads something new from the normal world, and starts processing it.

So, I think unfortunately in this design interrupts are definitely a concern, right?

Basically, we implemented the buffer as a message queue abstraction, so it will offer read() and write() messages, and it allows to send messages of (almost) unlimited size via the queue.

Maybe this is the wrong way approaching our goal. What we need is a way to communicate between secure (Genode) and normal (Linux) world such that we can send and receive any possible byte buffer.

On a multi-processor system, each CPU can be in the normal or secure world independently. Hence, it would be sensible to use a dedicated tz_vmm handler with a separate shared memory buffer for each CPU. Alternatively, the normal-world OS may synchronize SMC calls similar to how it usually synchronizes the access to a peripheral device. Neither of both designs would require mutex-based synchronization between the secure and normal world.

Could I clear up your doubts a bit? ;-)

Best regards Norman

Hi Stefan,

(secure world=Genode, normal world=Linux)

1. normal world allocates a buffer in normal world memory
2. normal world issues an SMC and transmits the buffer's address to

secure world where addresses are translated 3) secure world stores the address 4a) normal world writes something to the shared buffer and modifies some meta data structures. Basically, it writes to the metadata "Hey, secure world, there is something for you". 4b) secure world busy-waits (ugly) until it reads something new from the normal world, and starts processing it.

are you using the tz_vmm example at _os/src/server/tz_vmm/_ as blue print?

The points 1 to 4a are consistent with my suggestion. But instead of letting the tz_vmm poll for new information (4b), I propose to let the normal world issue an SMC call each time it has written something meaningful to the buffer. So the tz_vmm could simply block for SMC calls (wait_for_signal) and each time a signal comes in, it would process a new request. Not before finishing the work, it would pass control back to the normal world (by issuing _vm->run()). With "finishing the work" I mean that the tz_vmm applies changes to its internal state. It should not wait for a blocking operation between the reception of a signal and calling '_vm->run()' because this would stall the normal world.

Essentially, tz_vmm should be understood as a state machine. State changes are triggered by signals and applied in the '_handle_vm' function. An SMC request is such a signal.

Basically, we implemented the buffer as a message queue abstraction, so it will offer read() and write() messages, and it allows to send messages of (almost) unlimited size via the queue.

That sounds fine.

Cheers Norman

Hi,

On 05/12/2015 04:02 PM, Stefan Brenner wrote:

On 12.05.2015 15:47, Norman Feske wrote:

...

Hi Stefan,

...

(secure world=Genode, normal world=Linux)

1. normal world allocates a buffer in normal world memory
2. normal world issues an SMC and transmits the buffer's address to

secure world where addresses are translated 3) secure world stores the address 4a) normal world writes something to the shared buffer and modifies some meta data structures. Basically, it writes to the metadata "Hey, secure world, there is something for you". 4b) secure world busy-waits (ugly) until it reads something new from the normal world, and starts processing it.

are you using the tz_vmm example at _os/src/server/tz_vmm/_ as blue print?

The points 1 to 4a are consistent with my suggestion. But instead of letting the tz_vmm poll for new information (4b), I propose to let the normal world issue an SMC call each time it has written something meaningful to the buffer. So the tz_vmm could simply block for SMC calls (wait_for_signal) and each time a signal comes in, it would process a new request. Not before finishing the work, it would pass control back to the normal world (by issuing _vm->run()). With "finishing the work" I mean that the tz_vmm applies changes to its internal state. It should not wait for a blocking operation between the reception of a signal and calling '_vm->run()' because this would stall the normal world.

Ok, that sounds promising to me. Thanks for the hint. However, I see one problem for us here. If the secure world should handle long running tasks (large computation), using your proposed mechanism the normal world would be paused while computing in secure world, right?

But what we need is the ability for the normal world to send further requests to secure world and to respond to network packages during the secure world's computation. So we need a way for the secure world to notify back to normal world when the computation has finished and the response/result has been put to the message queue (we have another queue for the answers).

Afaik, in tz_vmm example (btw, yes, we have been starting from this example originally) the smc in the other direction (from secure to normal) unfortunately is not intended at the moment, right?

In our example there is no such use-case. Anyway, to signal from secure to normal world you can inject an interrupt. Of course a 'smc' instruction by definition (secure monitor call) would not work in this case. In case of the Freescale interrupt controller have a look at the Software Interrupt Trigger Registers (TZIC_SWINT) to trigger a software interrupt that you previously assigned to the normal world.

I did not had time to test it out on my own, but this way it should work.

Regards Stefan

...

Essentially, tz_vmm should be understood as a state machine. State changes are triggered by signals and applied in the '_handle_vm' function. An SMC request is such a signal.

...

Basically, we implemented the buffer as a message queue abstraction, so it will offer read() and write() messages, and it allows to send messages of (almost) unlimited size via the queue.

That sounds fine.

Cheers Norman