Hi Roman,

On 04.10.2016 20:04, Roman Iten wrote:

Hi, I wrote a simple run script 'manytimer', based on 'timer': https://github.com/rite/genode/tree/manytimerWhen the number of components exceeds 19, I get a "Quota exceeded!" warning. It seems that it doesn't matter if I use timer or other components. It also doesn't matter how much RAM quota I configure per timer. - Is this behaviour intended? - Whose quota exceeds (init, core, ...)? - Can I resolve the warning by increasing its quota? - In a scenario with 19 or more components (within one init), is it still possible to "assign all remaining resources to the last child by simply specifying an overly large quantum" as described in "Genode 16.05 Foundations" (see Chapter 6.2.2 "Resource quota saturation"). Or would there be no more slack memory available for init and core respectively?

I encountered the same problem in the Turmvilla scenario. In the presence of the "overly large quantum", init transfers all remaining quota from itself to the corresponding child and preserves just a tiny bit of quota for itself. This preserved quota is needed to accommodate the few metadata allocations that cannot be easily allocated from a specific child (i.e., child-thread meta data). If the number of children becomes too large, this preservation does not suffice. But you can increase the value as done by the following commit:

https://github.com/nfeske/genode/commit/619ce3ff2c81df8e24b10a675ccc78b83ee3...

Cheers Norman

Hi Roman,

• How big is the initial quota of init?

init receives the quota for all the available physical memory from core. The amount depends on the physical memory of the machine. It is printed by core when init is started. Look for a message like this:

24 MiB RAM assigned to init

• Can I change that quota - or only the preserved slack memory?

If you are using Qemu, you may adjust the memory of the virtual machine by putting the following line (for 256 MiB) in your run script:

append qemu_args " -m 256 "

In your run script, the value is 64 MiB, which leaves merely 24 MiB (on 64 bit) for init.

• If not - is there another way to prevent the "Quota exceeded!" warning?

I looked a bit closer at your scenario. Here is what happens:

For each child started by init, init needs to create several capabilities (e.g., the parent capability presented to the child, or the capability for the local ROM session for providing the child's binary as "binary" ROM module). The allocation of those capabilities is performed via the 'Nova_native_pd::alloc_rpc_cap' RPC function. This function naturally consumes session quota of the corresponding PD session (init's PD session). At one point, the initial session quota (that was passed to core when init's PD session was created) is depleted. In this case, core prints the diagnostic message and returns an error to the client (init). Init responds to this error by upgrading the session quota of its PD session using the preserved slack memory. The session upgrading is handled at 'base-nova/src/lib/base/rpc_cap_alloc.cc'.

In your case, the message you see is merely a diagnostic message. The condition is handled properly. In cases where proper error handing of the 'Out_of_metadata' condition is missing, the message used to be quite valuable to spot the problem. So we decided to kept it.

Of course, you may opt to suppress the message by adjusting the code in 'base/include/base/allocator_guard.h'.

• Does the size of the metadata allocation for a child depends on

whether I'm using a 32 or 64 bit system?

Yes. I.e., your scenario produces the message only on 64 bit, not on 32 bit.

• Is a scenario with 19 or more components within one init considered

'large'?

The static part of the current scenarios (like the ones tested by our autopilot) typically consists of fewer components. Personally, I only hit the limit with the Turmvilla scenario.

That said, the limit turned out not to be a problem in your case. The message is a false-positive warning. The default limit becomes a problem not before the PD-session upgrade fails. I can trigger the problem with your run script when configuring Qemu with 64 MiB of memory and starting 76 children. Sorry that my previous email pointed you to the wrong direction.

Cheers Norman

I've had problems with init failing when a component was configured to use all remaining available RAM. I've had to alter some of the standard run scripts to make them work. I don't remember which ones right now, but I think arora was one of them. Maybe init should have an XML option to set its reserved RAM quota, or it could instead automatically calculate the required quota based on the number of components it starts. Also, it would be nice to have both reserved and maximum quotas for components in init's configuration.

On Wed, Oct 5, 2016 at 8:48 AM, Roman Iten <roman.iten@...453...> wrote:

Hi Norman

• How big is the initial quota of init?

init receives the quota for all the available physical memory from core. The amount depends on the physical memory of the machine. It is printed by core when init is started.

Thats what I thought. But I couldn't imagine how init's quota could possibly exceed on a x86-64 machine with several GiB physical memory :)

For each child started by init, init needs to create several capabilities (e.g., the parent capability presented to the child, or the capability for the local ROM session for providing the child's binary as "binary" ROM module). The allocation of those capabilities is performed via the 'Nova_native_pd::alloc_rpc_cap' RPC function. This function naturally consumes session quota of the corresponding PD session (init's PD session). At one point, the initial session quota (that was passed to core when init's PD session was created) is depleted. In this case, core prints the diagnostic message and returns an error to the client (init). Init responds to this error by upgrading the session quota of its PD session using the preserved slack memory. The session upgrading is handled at 'base-nova/src/lib/base/rpc_cap_alloc.cc'.

In your case, the message you see is merely a diagnostic message. The condition is handled properly. In cases where proper error handing of the 'Out_of_metadata' condition is missing, the message used to be quite valuable to spot the problem. So we decided to kept it.

How can I distinguish if the condition is handled properly or not? Are there any preceding or following log messages in either case?

• Does the size of the metadata allocation for a child depends on whether

I'm using a 32 or 64 bit system?

Yes. I.e., your scenario produces the message only on 64 bit, not on 32 bit.

Is it worth thinking about calculating the slack memory size based on Genode::addr_t? Or make the value even configurable?

That said, the limit turned out not to be a problem in your case. The message is a false-positive warning. The default limit becomes a problem not before the PD-session upgrade fails. I can trigger the problem with your run script when configuring Qemu with 64 MiB of memory and starting 76 children. Sorry that my previous email pointed you to the wrong direction.

It didn't. I'm trying to improve my understanding of memory configuration and allocation in Genode. So every hint helps ;)

Thanks, Roman

---

---

Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot _______________________________________________ genode-main mailing list genode-main@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/genode-main

Hi Ben,

thanks for chiming in!

On 05.10.2016 22:22, Nobody III wrote:

Maybe init should have an XML option to set its reserved RAM quota, or it could instead automatically calculate the required quota based on the number of components it starts.

The automatic calculation would work if init was solely used in a static way. But init can actually respond to configuration changes at runtime. Hence, the number of children may vary over the lifetime of init, which makes it impossible to calculate the value up-front.

To address the problem in a fundamental way, init should allocate all child-specific capabilities from the PD session of the respective child. However, this is not a simple change. E.g., it would require us to pass the PD session as argument to 'Entrypoint::manage' and to keep the association of each RPC object with the PD session where its capability was allocated from. Since this is a rare problem (actually only for init at this point), I am not convinced to make the API more complicated for everyone. However, I have no strong opinion yet.

In the shorter term, it would be sensible to make the preservation configurable via init's configuration instead of hard-coding the value. I plan to rework init soon and will keep this suggestion in mind.

Also, it would be nice to have both reserved and maximum quotas for components in init's configuration.

I agree. I will consider this while reworking init.

Cheers Norman

Hi Roman,

I agree. Making the preservation configurable makes the memory reservations more transparent, i.e. everything that requires memory is visible in init's configuration - well, at least if init is used in a static way. Or are there still other memory 'pools' one might not be aware of?

on NOVA, the most important one is the kernel's memory pool, which has a fixed size that is defined in the kernel's linker script. The linker script is located in nova/src/hypervisor.ld (look for '_mempool_f').

Another limited resource is core's capability space, in particular the meta data required to manage the lifetime of capabilities. The details differ from kernel to kernel. On most base platforms, those information are kept in statically allocated arrays, which are dimensioned to accommodate the current scenarios. Core is in a special position because it has to keep track of all capabilities in the system (capabilities are allocated via core's PD service). Since the capability space of core is limited, we should apply Genode's resource-trading concept to capabilities too. In fact, I plan to implement this idea sometime next year. Until then, we have to life with the situation that capability allocations are not properly accounted (which is a potential denial-of-service issue).

One last question: how do I calculate the required memory preservation for init on nova_x86_64, based on the number of children?

I cannot give you a precise formula. My previously reported experiment where I started over 70 children with the default preservation of 128 KiB suggests that 2 KiB per child should suffice. Make it 16 KiB per child and you should be really fine. ;-)

Cheers Norman