Hi Roman,

• How big is the initial quota of init?

init receives the quota for all the available physical memory from core. The amount depends on the physical memory of the machine. It is printed by core when init is started. Look for a message like this:

24 MiB RAM assigned to init

• Can I change that quota - or only the preserved slack memory?

If you are using Qemu, you may adjust the memory of the virtual machine by putting the following line (for 256 MiB) in your run script:

append qemu_args " -m 256 "

In your run script, the value is 64 MiB, which leaves merely 24 MiB (on 64 bit) for init.

• If not - is there another way to prevent the "Quota exceeded!" warning?

I looked a bit closer at your scenario. Here is what happens:

For each child started by init, init needs to create several capabilities (e.g., the parent capability presented to the child, or the capability for the local ROM session for providing the child's binary as "binary" ROM module). The allocation of those capabilities is performed via the 'Nova_native_pd::alloc_rpc_cap' RPC function. This function naturally consumes session quota of the corresponding PD session (init's PD session). At one point, the initial session quota (that was passed to core when init's PD session was created) is depleted. In this case, core prints the diagnostic message and returns an error to the client (init). Init responds to this error by upgrading the session quota of its PD session using the preserved slack memory. The session upgrading is handled at 'base-nova/src/lib/base/rpc_cap_alloc.cc'.

In your case, the message you see is merely a diagnostic message. The condition is handled properly. In cases where proper error handing of the 'Out_of_metadata' condition is missing, the message used to be quite valuable to spot the problem. So we decided to kept it.

Of course, you may opt to suppress the message by adjusting the code in 'base/include/base/allocator_guard.h'.

• Does the size of the metadata allocation for a child depends on

whether I'm using a 32 or 64 bit system?

Yes. I.e., your scenario produces the message only on 64 bit, not on 32 bit.

• Is a scenario with 19 or more components within one init considered

'large'?

The static part of the current scenarios (like the ones tested by our autopilot) typically consists of fewer components. Personally, I only hit the limit with the Turmvilla scenario.

That said, the limit turned out not to be a problem in your case. The message is a false-positive warning. The default limit becomes a problem not before the PD-session upgrade fails. I can trigger the problem with your run script when configuring Qemu with 64 MiB of memory and starting 76 children. Sorry that my previous email pointed you to the wrong direction.

Cheers Norman

I've had problems with init failing when a component was configured to use all remaining available RAM. I've had to alter some of the standard run scripts to make them work. I don't remember which ones right now, but I think arora was one of them. Maybe init should have an XML option to set its reserved RAM quota, or it could instead automatically calculate the required quota based on the number of components it starts. Also, it would be nice to have both reserved and maximum quotas for components in init's configuration.

On Wed, Oct 5, 2016 at 8:48 AM, Roman Iten <roman.iten@...453...> wrote:

Hi Norman

• How big is the initial quota of init?

init receives the quota for all the available physical memory from core. The amount depends on the physical memory of the machine. It is printed by core when init is started.

Thats what I thought. But I couldn't imagine how init's quota could possibly exceed on a x86-64 machine with several GiB physical memory :)

For each child started by init, init needs to create several capabilities (e.g., the parent capability presented to the child, or the capability for the local ROM session for providing the child's binary as "binary" ROM module). The allocation of those capabilities is performed via the 'Nova_native_pd::alloc_rpc_cap' RPC function. This function naturally consumes session quota of the corresponding PD session (init's PD session). At one point, the initial session quota (that was passed to core when init's PD session was created) is depleted. In this case, core prints the diagnostic message and returns an error to the client (init). Init responds to this error by upgrading the session quota of its PD session using the preserved slack memory. The session upgrading is handled at 'base-nova/src/lib/base/rpc_cap_alloc.cc'.

In your case, the message you see is merely a diagnostic message. The condition is handled properly. In cases where proper error handing of the 'Out_of_metadata' condition is missing, the message used to be quite valuable to spot the problem. So we decided to kept it.

How can I distinguish if the condition is handled properly or not? Are there any preceding or following log messages in either case?

• Does the size of the metadata allocation for a child depends on whether

I'm using a 32 or 64 bit system?

Yes. I.e., your scenario produces the message only on 64 bit, not on 32 bit.

Is it worth thinking about calculating the slack memory size based on Genode::addr_t? Or make the value even configurable?

That said, the limit turned out not to be a problem in your case. The message is a false-positive warning. The default limit becomes a problem not before the PD-session upgrade fails. I can trigger the problem with your run script when configuring Qemu with 64 MiB of memory and starting 76 children. Sorry that my previous email pointed you to the wrong direction.

It didn't. I'm trying to improve my understanding of memory configuration and allocation in Genode. So every hint helps ;)

Thanks, Roman

---

---

Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main

Hi

Thank you very much for this very good explanation.

On 07.10.2016 10:47, Norman Feske wrote:

In the shorter term, it would be sensible to make the preservation configurable via init's configuration instead of hard-coding the value. I plan to rework init soon and will keep this suggestion in mind.

I agree. Making the preservation configurable makes the memory reservations more transparent, i.e. everything that requires memory is visible in init's configuration - well, at least if init is used in a static way. Or are there still other memory 'pools' one might not be aware of?

...

Also, it would be nice to have both reserved and maximum quotas for components in init's configuration.

I agree. I will consider this while reworking init.

Great, I like the idea of having both, reserved/soft and a maximum/hard quota.

One last question: how do I calculate the required memory preservation for init on nova_x86_64, based on the number of children?

Thanks, Roman

PS: sorry for screwing up the quoting in my previous mail. Tried another mail client - I'm back to thunderbird now :)

Hi Roman,

How can I distinguish if the condition is handled properly or not? Are there any preceding or following log messages in either case?

I remember the following situations where the messages were somehow useful to diagnose a problem:

- If the scenario got stuck with the this being the last message. This happened in the phase when we introduced the dynamic resource handling originally (around version 13.11). In the meanwhile, we have equipped all places where clients may encounter the out-of- metadata condition with corresponding handlers (i.e., in the form of the 'base/src/include/base/internal/expanding_*_client.h' classes). Therefore, I haven't observed this effect for a long time now.

- If a component aborts right after a quota-exceeded message, this is a strong hint that an exception handler is missing. An example is the issue recently reported by Denis (and fixed with [1]) where core missed to reflect an 'Allocator::Out_of_memory' condition as an 'Rm_session::Out_of_metadata' exception. That said, in this case, the message does not add too much information because the abort message already prints the exception type, which should suffice to diagnose the problem.

- If an unusual amount of quota-exceeded messages appears, this hints at a resource leak or at least an opportunity for optimization. The initial session quotas and the quota upgrades are dimensioned such that the quota updates are needed only sporadically. If the log is flooded by quota-exceeded messages, a superficial number of quota upgrades is performed, which should better be investigated.

The more I think about it, the better I like the idea to remove the message. In the log, it gives a wrong impression of presumed problems where none exists because the quota upgrading is a normal part of life of Genode components. The diagnostic value added by the messages is indeed quite low.

[1] https://github.com/genodelabs/genode/commit/fac69eff48b3d70a9d37ebc5d3ae6746...

...

- Does the size of the metadata allocation for a child depends on
whether I'm using a 32 or 64 bit system? 

Yes. I.e., your scenario produces the message only on 64 bit, not on 32 bit.

Is it worth thinking about calculating the slack memory size based on Genode::addr_t? Or make the value even configurable?

Making the value configurable seems to be the most sensible idea. I will respond to Ben's (Nobody III) posting with an explanation why.

Cheers Norman