Why application request RAM quota from parent even if it is enough?

[Norman Feske]

Hi Alexander,

>> The current mmap(MAP_ANONYMOUS) implementation seems to strive for
>> satisfaction of the application by not probing the allocation but
>> enforcing it. This ends up in the request for more RAM resources as
>> you traced.
> 
> Yes, but why this call (wait for response from parent) just hangs forever instead of return and say that “I can’t”?

even though it might be confusing, this is intentional. Let me try to
give the rationale.

I'm probably over-generalizing but in Genode, child components are
supposed to act according to the parent's wishes. They receive their
configuration and a budget of resources from their parent and act
accordingly. They should not try to be clever, or probe for resources,
or implement fall-backs. That wouldn't be in the interest of the parent.
Now if a child is tasked (by its parent) to do a particular job but it
has not received enough resources to accomplish it, it _must_ escalate
this situation to its parent because the parent is generally the only
one who can resolve this situation (e.g., by upgrading the child's
resources, or changing plans). If a child silently dealt with a ENOMEM
situation without telling the parent, it would deprive the parent from
taking a conscious policy decision.

What do we gain from this rigid approach? Simplicity of the components,
robustness, and deterministic behavior.

*Simplicity* because the components are relieved from handling error
cases, which are - in practice - never thoroughly tested anyway. How
many applications are there in the wild who actually handle ENOMEM? How
many of those few handle it in a reasonable and deterministic way other
than panic? Returning ENOMEM is not a solution because in practice, such
rare conditions are not handled. Worse, in traditional software, you
find a lot of error-handling code that sits there but is almost never
executed or stressed. Such code is the perfect hiding place for
vulnerabilities. By relieving software from the this burden, such
low-quality code can be omitted.

*Robustness* follows from simplicity and the child being transparent
about its problems instead of swallowing them up and failing later with
much less obvious symptoms. Failing early is good.

*Deterministic behavior* is reinforced because the child does not
implement an opaque policy on its own. In the few cases where an
application intents to adapt itself the its resource budgets, Genode
allows a component to request its available budgets
(Env.pd().avail_ram()) but this is a special case.

---

Regarding your practical problem with the Go runtime, the virtual-memory
reservation scheme you encountered is fundamentally at odds with Genode
because anonymous memory is always backed by physical memory, not a
zero-page mapping and a copy-on-write mechanism as in Linux. This is not
a limitation but in line with the philosophy outlined above. A
reservation of 500 GiB of memory is in fact an announcement by the
application that it may potentially _use_ this memory. The Linux kernel
says: fine. So the application proceeds. In the event it actually uses
all this memory, the application or even the entire system will suffer
in indeterministic and complex ways. Genode does not give such promises.
It also does not deny them. Instead, it tells the parent about it so the
parent can in principle resolve it. In your case, the parent simply
prints a message and keeps the child blocking infinitely.

In your position, I'd try to investigate ways around the
reservation-based virtual memory management. We had to overcome similar
problems in the past, e.g., I think for the Java runtime. Unfortunately,
this usually requires one to dig deep into the runtime. If you get
stuck, you may consider making your current working branch publicly
available so that others can have a look and possibly provide guidance.

I hope that I could lift the clouds a bit.

Cheers
Norman

-- 
Dr.-Ing. Norman Feske
Genode Labs

https://www.genode-labs.com · https://genode.org

Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden
Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth