Hi Johannes,
thanks for posting your idea. I share your sentiment that the debugging challenges increasingly move from individual programs to entire compositions of components. Here, the classical tools like GDB start to become inefficient.
On the other hand, I made some interesting observations while developing the GUI stack, which consists of many components that interact with each other. As you said, each of those components is typically quite simple. In fact, from a high-level view, a component like the window layouter is merely a state machine that consumes 'ROM' data and produces 'Report' data. Since I realized the propagation of state between the components via the report_rom server, a new opportunity for debugging appeared: By configuring the report_rom server to be verbose, each incoming report is being printed to the LOG. Since all the components report their internal states as reports, the output is a nice sequence of state transitions. Most problems turned out to be very easy to spot in this trace.
I can think of several ways to make this style of debugging even more usable for interactive scenarios. For example, a graphical tool could consume those reports and produce a graphical view of each report presented as the internal state of the originating component. Since the report_rom server also knows the consumers of each report (the ROM session clients), we could even show the information flow between the components graphically. This view could be dynamic, depending on clients known by the report_rom service.
The debug logic can probably be implemented as a debug library that implements the hooks. A component is being instrumented by adding calls to the library functions and registering callback functions that print the debug information. Debugging is disabled by linking against a dummy library. Maybe this could also be implemented as a trace policy.
Leveraging the tracing mechanism sounds like a very cool idea.
Cheers Norman