Hi Peter,
thanks for providing more insight into your work. With the workload on Genode getting more and more sophisticated with each release, profiling becomes increasingly important to us. So your work in this direction is much appreciated. :-)
I know too little about the structure of your profiling environment to give a clear advice on how to integrate it with Genode. The answer depends on the relationship between the user-level portion of your profiling environment and the other processes. If there is no interaction (only interaction with the kernel) and the profiling takes place at a global level (e.g., monitoring the IPC traffic between all processes), the profiling process could be started just alongside the other programs by the init process. But I assume that the profiling environment somehow has to interact with the rest of the system, for example for presenting the profiling result or for getting activated in the first place.
Another setting would be to apply profiling as a tool to analyse a Genode subsystem only. This could be achieved by letting the profiler play the role of the parent of this subsystem (similar to init). This way, all requests of the subsystem to the outside will become visible to the profiler at the Genode API level. For example, if the subsystem creates a new process, it will ask the parent (which is then the profiler) to create the corresponding CPU, PD, and RAM sessions. So the profiler can observe how the subsystem interacts with the outside and, in principle, virtualize all services used by the subsystem, or just delegate requests further to the profiler's parent. This way, the profiler could obtain much higher level information about the system than plain kernel events.
Regarding the question about how to integrate your source code with Genode, I recommend creating a dedicated repository for your work rather than modifying the existing source code. You are free to define the structure of your repository. Usually we have a layout such as
repo/include - public interfaces usable by other repositories repo/lib/mk - library description files usable from programs of other repositories repo/src/server - source codes of services repo/src/lib - source codes of libraries repo/src/app - source codes of applications repo/src/test - source codes for test programs repo/tool - tools and scripts specific for the repository repo/doc - documentation
Whether your profiler belongs to 'src/server' or 'src/app' depends on role it plays within Genode (does it provide a service RPC interface or not?).
The reason why I'm looking at genode was that there was an implementation of a HTTP server. I'd like to be able to use my language/profiling environment so that it can monitor activity in terms of CPU usage, and also network usage, given the IP stack. What do you think would be the best way to "link" my profiling environment with Genode, given that virtually all the profiling code is L4 based (there is some Iguana functionality as i mentioned before, but this code can easily be converted to L4 code). I was thinking that it could be implemented as part of the test suite (I've noticed a couple of test folders that provide implementations of pagers and what not), where profiling begins upon boot up, and then somehow providing a link between genode and my profiling environment.
Can you elaborate why the profiling code is tied to L4? Would it be possible to generalize it? I ask because a general profiling infra- structure would be valuable not only on OKL4 but also on the other platforms.
Apart from the networking example, I think there are plenty of workloads on Genode that are worth to inspect in detail. For example, how does Qt4 interact with Genode and are there opportunities for performance optimizations? Or how interplays Genode's signalling API with the OKL4 scheduler? Or how much latency overhead do we have for each interrupt delivered via Genodes IRQ service? A profiling framework could provide answers to these and many more interesting questions.
Regards Norman