Hi Markus,
I used run/vancouver as starting script, yes. And I noticed that increasing to something around 90 MiB works without problems. Going beyond that (e.g., 100MiB) strange things happen like page faults where there should not be ones, or for example an INT 3 debug instruction which seems to "forget" about the parameter it was passed.
I forgot to mention another point that needs to be considered: The link address of the Vancouver program.
The lower portion of Vancouver's address space corresponds to the guest-physical memory. This one-to-one relationship is imposed by the NOVA hypervisor. For this reason, this particular virtual address range must be kept free from ordinary memory objects (as I outlined in my reply to Julian's posting). This includes dataspaces attached to Vancouver's address space but also Vancouver's text, data, and bss segments. If Vancouver used the default link address used by normal Genode programs, this invariant would be violated. Therefore, the link address is explicitly specified in Vancouver's target.mk file. Currently, it is set to 0x50000000, which might explain your problems. Can you try to increase this value?
Cheers Norman