Hello Jilong,

Can anyone give me a hint on how to get rid of the "Could not allocate metadata" error msg? Or, how can I effectively increase the quota for that?

your question too generic to give you a definite answer. Your complete LOG output and a way to reproduce the problem (i.e., a run script) would help.

However, let me try to guide you for finding the problem yourself. The message "Could not allocate metadata" is printed by core's RAM service when there is not enough session quota left for holding meta data about allocated/free memory blocks. Hence, your attempt to upgrade the session quota using 'parent()->upgrade()' was actually spot-on. However, since you report that the upgrade did not solve your problem, I suppose that the session that produced the error message is not the one represented by the 'env()->ram_session_cap()' argument you passed to the upgrade function.

Unfortunately, the message printed by core leaves little insight about the client that triggered the problem. To reveal a bit more information, you may find the patch "Debug: print session labels if quota exceeds" in the following branch useful:

https://github.com/nfeske/genode/commits/quota_msg

Just cherry-pick the commit and adjust it as needed, e.g., you may enhance the error message with additional status information of the RAM session. Right now, it just appends the session label as additional information, which may already guide you to the right spot.

Another approach to find the troubling session it to let core block once the error condition occurs. I.e., just temporarily insert an infinite loop after the message is printed. This way, core will get stuck at that point and, consequently, not reply the RPC call that triggered the problem. Now, when the condition occurs, you can enter the Fiasco.OC kernel debugger and look up the thread that is currently doing an IPC call to core's entrypoint thread. Use the 'lp' command to list the threads and look at their respective states. Knowing the troubling thread will probably be helpful.

Good luck! If you manage to encircle the problem, I would greatly appreciate if you report the issue (and possibly your fix).

Cheers Norman

Hello Jilong,

thanks for describing your scenario in more detail. Apparently, the whole issue is actually not an issue at all because of the message "[init->P1] upgrading quota donation for Env::RAM session". To get a grasp what is going on, you will need to understand the quota concept of Genode more thoroughly.

Let me draw an analogy first. Imagine starting a small company that outsources its accounting work to a bookkeeping service company. You entrust the bookkeeper to track the records of all your financial transactions and even grant him access to your bank account. So for paying an invoice, you can conveniently send over the invoice with a short notice to the bookkeeper and he will take care of issuing the bank transaction within the due day of payment. It will also record the transaction in your books. Of course, for doing this work, the bookkeeper needs resources such as allowances, sheets of paper, ink, or coffee to stay awake while doing the boring paperwork. He gets paid according to a contract with you. Now, imagine that your business starts to take off, resulting in a huge amount of individual transactions reported to the bookkeeper. This will inevitably increase the work and expenditures of the bookkeeper. Even though he has your bank account's credentials, he will obviously not withdraw those costs from your bank account at his own discretion. Instead, he will give you a call to re-negotiate his terms.

How does this scenario relate to Genode? Core's RAM service plays the role of such a bookkeeper. Once a process has opened an account at the RAM service (by creating a RAM session at core), the RAM service keeps track of the RAM budget and all the RAM allocations withdrawn from the account. To keep those records, the RAM service needs a bit of RAM to cover its expenditures. This RAM is explicitly provided by the client by lending a bit of its RAM quota to the RAM service at session-creation time (the initial contract). The default amount of RAM attributed for the expenditures can be found in the 'Ram_connection' class:

https://github.com/genodelabs/genode/blob/master/base/include/ram_session/co...

64 KiB seem to be enough for a usual process. Please note that the RAM session for a process is not created by the process itself. It is created by its parent. If a process allocates an unusually high amount of RAM dataspaces, the 64 KiB won't be enough to hold all the records. So the RAM service will at some point throw an 'Out_of_metadata' exception, thereby signalling to the client that it's time to re-negotiate the terms. A normal Genode process will respond to such an exception by upgrading the terms. You can find the corresponding code at:

https://github.com/genodelabs/genode/blob/master/base/src/base/env/platform_...

The 'Expanding_ram_session_client' will receive the call from the bookkeeper about the unexpectedly increased costs for bookkeeping, print the message "upgrading quota donation for Env::RAM session" and send over another 8 KiB to make the bookkeeper happy.

When speaking of the RAM quota of a process, we refer to the budget of RAM of a process (like the balance on a bank account), assigned by its parent. E.g., when a process is created by the init process, the 'config' file tells the init process how to distribute init's own budget among its children.

When opening a session to a service, an initial payment of the service is performed. The terms of the "contract" between the client and the server are typically represented by the corresponding 'Connection' class. By calling the 'Parent::upgrade' function, a process requests its parent to increase the payment for a service (the RAM service is just one of those) after session-creation time. In both situations session creation and session upgrade, the parent will withdraw the specified amount from the budget of the client and transfer it to the server.

In short, your fix is not necessary because the 'Expanding_ram_session_client' already responds to the situation where core prints the "Could not allocate metadata" message. This message is not an error. In principle, we could remove the message. But in some situations where the client does not respond to such a condition in a meaningful way, this diagnostic message used to guide us to the problem. So we kept it. Sorry for the confusion!

process in Genode initially. In my upgrade() statement, I give 32M to each process.

That should not be needed. Genode's RAM service will hopefully never ever require 32 MiB for bookkeeping. ;-)

I hope that I could demystify the concept of Genode's RAM quotas a bit.

Cheers Norman