Hi Norman, Yes, sorry I meant using the system calls rather than L4Re directly.

As far as design choice, I think you need both. When you can use an affinity mask for a set of threads that belong to a single CPU session, then setting this mask through the CPU session works. However, lets say you want to explicitly define each individual mapping of thread to core. In this case you either need to be able to define the affinity from the thread, or some how build a map of threads-to-affinity in the CPU session. The latter could be cleaner, but less conventional.

I did try to include the l4sys headers in my genode program, but it sort of blew up - it found the headers but not things like L4_PAGESHIFT.

Daniel

On 03/17/2011 09:37 AM, Norman Feske wrote:

Hello Daniel,

...

I'd like to be able to use the native Fiasco.OC L4re APIs (actually to set thread affinity) from within my Genode application. Its not clear to me how to set up the target.mk to do such things.

Genode is not built upon L4re. It is using only the Fiasco.OC kernel bindings (called l4sys) and two other small components of L4re, namely sigma0 and bootstrap. Logically, both sigma0 and bootstrap actually belong to the kernel but they are contained in the L4re source tree for historical reasons.

It is not possible to intermix L4re and Genode code but of course, you can issue Fiasco.OC system calls by including the corresponding kernel bindings. See 'base-foc/src/base/ipc/ipc.cc' as an example. As you can see in this library code, we use to include Fiasco.OC header files into the dedicated C++ namespace 'Fiasco'. I would recommend you to do the same.

Setting the thread affinity is not yet supported though Genode's API because until recently, we used Genode with SMP only on Codezero and Linux, both handling the thread affinity transparently. However, there are two natural ways to support Fiasco.OC's explicit assignment of thread affinities, as session-construction argument for CPU sessions or by adding a 'set_affinity' function to Genode's 'Cpu_session' interface. Both variants were successfully prototyped for Pistachio but have not been merged with the official Genode API.

Personally, I have a preference to specifying the affinity as CPU session argument such that all threads of this session are implicitly bound to the specified CPU. This enables the assignment of complete Genode subsystems to a specific CPU. Also it would preserve the notion of having a "CPU session". Furthermore, the affinity would be subjected to session policies imposed by the process hierarchy (similar to how RT priorities are handled now) - affinities could be "virtualized". In contrast, by adding a 'set_affinity' call to the 'Cpu_session' interface, different threads of one CPU session could be bound to different CPUs.

In both cases, the propagation of the thread affinity to the Fiasco.OC kernel must happen within Genode's core by submitting the affinity as argument when creating a new thread (at 'base-foc/src/core/platform_thread.cc').

Which variant of setting the affinity would make more sense from your perspective?

Best regards Norman

Hi Norman,

Yes, I got the L4 headers to work now.

What if one program would create multiple CPU sessions and specify the CPU session for each thread as constructor argument of the 'Thread' class? In the extreme case, each thread could be created via a distinct CPU session using a different affinity. The program would express its desired affinity for each CPU session as session argument. The parent then has the option to change the affinity when forwarding the session request towards Genode's core. This would be a quite powerful mechanism to assign any subsystem to arbitrary sets of CPUs without letting the subsystem know about it.

Yes I think this would work.

Daniel

On 03/17/2011 11:42 AM, Norman Feske wrote:

Hi Daniel,

...

As far as design choice, I think you need both. When you can use an affinity mask for a set of threads that belong to a single CPU session, then setting this mask through the CPU session works. However, lets say you want to explicitly define each individual mapping of thread to core. In this case you either need to be able to define the affinity from the thread, or some how build a map of threads-to-affinity in the CPU session. The latter could be cleaner, but less conventional.

What if one program would create multiple CPU sessions and specify the CPU session for each thread as constructor argument of the 'Thread' class? In the extreme case, each thread could be created via a distinct CPU session using a different affinity. The program would express its desired affinity for each CPU session as session argument. The parent then has the option to change the affinity when forwarding the session request towards Genode's core. This would be a quite powerful mechanism to assign any subsystem to arbitrary sets of CPUs without letting the subsystem know about it.

Currently, the 'Thread_base' implementation always uses the CPU session of 'Genode::env()->cpu_session()'. However, the replacement of this hard-wired policy by a construction argument would be a sensible way to enable the binding of threads to manually created CPU sessions.

...

I did try to include the l4sys headers in my genode program, but it sort of blew up - it found the headers but not things like L4_PAGESHIFT.

Normally, this shouldn't be a problem. Can you give the following program a try?

base-foc/src/test/headers/test.cc:

namespace Fiasco { #include<l4/sys/types.h> }

#include<base/printf.h>

int main(int argc, char **argv) { using namespace Fiasco;

 Genode::printf("L4_PAGESHIFT=%ld\n",
                (long)L4_PAGESHIFT);
 return 0;

}

base-foc/src/test/headers/target.mk:

TARGET = test-headers SRC_CC = test.cc LIBS = env cxx

base-foc/run/headers.run:

build "core init test/headers" create_boot_directory

install_config { <config> <parent-provides> <service name="ROM"/> <service name="LOG"/> </parent-provides> <default-route> <any-service> <parent/> </any-service> </default-route> <start name="test-headers"> <resource name="RAM" quantum="1M"/> </start> </config> }

build_boot_image "core init test-headers" append qemu_args "-nographic -m 64" run_genode_until "exited with exit value 0" 20

To execute the test, just issue 'make run/headers' from your build directory.

Cheers Norman