Conquering NUMA land

Norman Feske norman.feske at ...1...
Tue Mar 19 14:36:17 CET 2013

Hash: SHA1

Hi Udo,

> There are also use cases where you don't want to partition. One
> example is a multi-core VM, where each virtual CPU could run on a
> different physical core and yet all of those virtual CPUs share the
> same memory.

I agree that a vcore instance should definitely be able to manage sets
of CPUs. The partitioning policy should be up to the user.

> Rather than going for an extreme design point, where virtually
> nothing is shared (e.g., Barrelfish), I think it would be better to
> provide an interface where the user has precise control over what
> is shared and what isn't.
> I'd go for concurrent invocation of services first. Then you'll
> know what data structures you have contention on. And then you can
> decide whether you want that data replicated (read-mostly) or
> shared (frequently written).
> IMHO, dealing with replicas, distributed protocols, consensus and
> all that is a lot harder than implementing a few locks or atomic
> ops on pieces of shared memory. Especially now that we have HLE and
> TSX coming really soon.

The vcore approach should indeed not hold us back from making core
more scalable. The latter should be the ultimate goal and if new Intel
technologies can help us, that's great.

One thing left me wondering, don't you see the different access
latencies to local vs. remote memory in NUMA systems as a pressing
problem that needs a solution by the OS? The consideration of memory
locality was actually the driving motivation behind the vcore idea.


- -- 
Dr.-Ing. Norman Feske
Genode Labs ·

Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden
Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla -


More information about the users mailing list