I am trying to implement not-yet supported (as mentioned by nfeske@ in mailing list) option for mmap() related to anonymous mapping with pre-defined (desired) address. Like libc function with desired address out_addr. void *Libc::Mem_alloc_impl::alloc_at(void *out_addr, size_t size)
I use a couple of useful function in avl allocator. Anyway, I found that straight - forward implementation do not work because of allocation gaps filled together with main allocation.
In particular, if we want to allocate e.g. 0x10000 from address 0x500000 then we typically allocate 0x13000 bytes, including ones for avl metadata/etc stored in the same space, and receive virtual address range used, e.g. 0x500000 - 0x513000:
/* * Calculate block size of needed backing store. The block must hold the * requested 'size' with the requested alignment, a new Dataspace structure * and space for AVL-node slab blocks if the allocation above failed. * Finally, we align the size to a 4K page. */ size_t request_size = size + max((1 << align_log2), 1024) + Allocator_avl::slab_block_size() + sizeof(Dataspace);
Software I used (golang runtime) sometimes try to expand previous allocation, e.g. it can request allocation of 0x8000 bytes before previous range, like 0x4f8000 - 0x500000. In that case I can’t perform the operation - because it will try to allocate 0xa000 bytes from desired address 0x4f8000 - and it obviously fail due to Range_conflict.
So, I split allocation to 2 parts - one for this gap by standard alloc() and random placement and second one by my own alloc_at() with size exactly correspond to the requested one.
Implementation available as a patch to 20.05 at
https://github.com/tor-m6/genode/commit/78fc751bea8acc2b515e04bec9f3a8834615...
Problem I see here is a free operation: it just free the block with requested address, and do not touch others, gap-related blocks. For me it is not obvious that they need to be free, because in the code (e.g. by unmap()) I do not see explicit free of other allocated structures, in particular, dataspace extension or related block metadata.
Question: should I make additional efforts to trace and delete something else - except what standard free does here: /* forward request to our local allocator */ _alloc.free(addr);
Like for dataspace structure or address ranges for second «auxiliary» allocation?
PS Another potential problem is a probability to overlap of first «random address» allocation with desired address. This happens with first attempt to implement allocator - it allocates address from desired range, and immediately put here 0x18 bytes of Dataspace structure, and then fail because requested address is already busy by this structure! Anyway, I consider such a probability relatively small with proposed approach and ignore it (may be I am wrong)
Can anyone help the question below about "additional efforts to trace and delete something else"?
12 авг. 2020 г., в 02:07, Alexander Tormasov via users users@lists.genode.org написал(а):
I am trying to implement not-yet supported (as mentioned by nfeske@ in mailing list) option for mmap() related to anonymous mapping with pre-defined (desired) address. Like libc function with desired address out_addr. void *Libc::Mem_alloc_impl::alloc_at(void *out_addr, size_t size)
I use a couple of useful function in avl allocator. Anyway, I found that straight - forward implementation do not work because of allocation gaps filled together with main allocation.
In particular, if we want to allocate e.g. 0x10000 from address 0x500000 then we typically allocate 0x13000 bytes, including ones for avl metadata/etc stored in the same space, and receive virtual address range used, e.g. 0x500000 - 0x513000:
/* * Calculate block size of needed backing store. The block must hold the * requested 'size' with the requested alignment, a new Dataspace structure * and space for AVL-node slab blocks if the allocation above failed. * Finally, we align the size to a 4K page. */ size_t request_size = size + max((1 << align_log2), 1024) + Allocator_avl::slab_block_size() + sizeof(Dataspace);
Software I used (golang runtime) sometimes try to expand previous allocation, e.g. it can request allocation of 0x8000 bytes before previous range, like 0x4f8000 - 0x500000. In that case I can’t perform the operation - because it will try to allocate 0xa000 bytes from desired address 0x4f8000 - and it obviously fail due to Range_conflict.
So, I split allocation to 2 parts - one for this gap by standard alloc() and random placement and second one by my own alloc_at() with size exactly correspond to the requested one.
Implementation available as a patch to 20.05 at
https://github.com/tor-m6/genode/commit/78fc751bea8acc2b515e04bec9f3a8834615...
Problem I see here is a free operation: it just free the block with requested address, and do not touch others, gap-related blocks. For me it is not obvious that they need to be free, because in the code (e.g. by unmap()) I do not see explicit free of other allocated structures, in particular, dataspace extension or related block metadata.
Question: should I make additional efforts to trace and delete something else - except what standard free does here: /* forward request to our local allocator */ _alloc.free(addr);
Like for dataspace structure or address ranges for second «auxiliary» allocation?
PS Another potential problem is a probability to overlap of first «random address» allocation with desired address. This happens with first attempt to implement allocator - it allocates address from desired range, and immediately put here 0x18 bytes of Dataspace structure, and then fail because requested address is already busy by this structure! Anyway, I consider such a probability relatively small with proposed approach and ignore it (may be I am wrong)
Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users
Hi Alexander,
as I noted in my other posting, I'm afraid that the support of mappings via mmap at fixed addresses can never be robust across kernels because one cannot make assumptions about the virtual address space layout.
To create such a mechanism in a reasonably robust way, I'd go for the use of a managed dataspace. In the libc configuration, one could define the virtual address range where mmap mappings with fixed addresses are supposed to go. Upon start of the program. The libc would try to preserve this area by creating an appropriately sized Region_map (via the RM service) and attaching it at the base of the configured range. If this step fails (this can happen, depending on the kernel or architecture - think of 32bit), it will fail right at this early point.
Once the managed dataspace is attached to the component's virtual address space, the libc can manually manage its layout via 'attach_at' without fearing conflicts. For an example, please have a look at base/src/test/sub_rm/.
This feature would work independently from the 'Libc::Mem_alloc'. I think that the use of 'Libc::Mem_alloc' by 'mmap' is not the right way to go when taking mmap beyond it current scope. In particular, 'Libc::Mem_alloc' will eventually be swapped out by a better allocator (such as jemalloc). Extending its interface (as suggested by your patch) will make this step more complicated.
So, I split allocation to 2 parts - one for this gap by standard alloc() and random placement and second one by my own alloc_at() with size exactly correspond to the requested one.
That's good. The meta data should be separate from the actual payload.
Implementation available as a patch to 20.05 at
https://github.com/tor-m6/genode/commit/78fc751bea8acc2b515e04bec9f3a8834615...
Problem I see here is a free operation: it just free the block with requested address, and do not touch others, gap-related blocks. For me it is not obvious that they need to be free, because in the code (e.g. by unmap()) I do not see explicit free of other allocated structures, in particular, dataspace extension or related block metadata.
That's another reason why the mmap support needs additional infrastructure. I think of a registry of mmap mappings. The registry entries store all the meta data required. They can be allocated on the 'Kernel::_heap'. The actual memory is allocated as RAM dataspaces. The 'Libc::Mem_alloc' is not required.
Question: should I make additional efforts to trace and delete something else - except what standard free does here: /* forward request to our local allocator */ _alloc.free(addr);
Like for dataspace structure or address ranges for second «auxiliary» allocation?
The current implementation of 'munmap' does not suffice for sure. In particular, it would leak the backing store of meta data and the dataspaces added via 'expand_at'.
PS Another potential problem is a probability to overlap of first «random address» allocation with desired address. This happens with first attempt to implement allocator - it allocates address from desired range, and immediately put here 0x18 bytes of Dataspace structure, and then fail because requested address is already busy by this structure! Anyway, I consider such a probability relatively small with proposed approach and ignore it (may be I am wrong)
I think this is a big problem because it can strike at any time at runtime. It can be addressed best with the managed-dataspace approach outlined above. But this is more invasive than your current patch.
Cheers Norman
Hi Norman,
as I noted in my other posting, I'm afraid that the support of mappings via mmap at fixed addresses can never be robust across kernels because one cannot make assumptions about the virtual address space layout.
To create such a mechanism in a reasonably robust way, I'd go for the use of a managed dataspace. In the libc configuration, one could define the virtual address range where mmap mappings with fixed addresses are supposed to go. Upon start of the program. The libc would try to preserve this area by creating an appropriately sized Region_map (via the RM service) and attaching it at the base of the configured range. If this step fails (this can happen, depending on the kernel or architecture - think of 32bit), it will fail right at this early point.
Problem I see here is that current implementation of Libc alloc() do not have notation of "memory allocated, but not committed". In particular, I do not want to allocate during libc init all physical memory which I can use in the future; I want to allocate virtual memory range without backened physical ones, and populate it by demand.
This is a way how golang own runtime memory allocator implemented: they reserve single continuous memory range for so called «arena» where they store go data and some aux structures. They can arrange on different OS up to 544Gb (on some like linux smaller). Later, when necessary, they really fill parts of this range by some real memory. They use mmap with _PROT_NONE, _MAP_ANON|_MAP_PRIVATE to reserve and mmap with _PROT_READ|_PROT_WRITE, _MAP_ANON|_MAP_FIXED|_MAP_PRIVATE to commit. In your implementation of mmap I see that _PROT_NONE is not supported.
as I see, current interface of libc alloc() assume that all memory stored inside is committed (has been called _ds_pool.expand()), so, I can’t reserve memory and populate it by demand naturally, without libc alloc changes?
By the way, I see a couple of references in genode docs to the way to "use managed dataspace" and "populate it by phys memory by demand", while no references to code samples provided? How I can reserve virtual memory range in component without backend storage, in what allocator?
Once the managed dataspace is attached to the component's virtual address space, the libc can manually manage its layout via 'attach_at' without fearing conflicts. For an example, please have a look at base/src/test/sub_rm/.
as I understand, this approach will work only if I alloc whole range of virtual memory _backend by physical memory_? In that case I can use standardised in alloc approach and do not need additional call to expand_at() interface?
by the way, from the recommended example I found the following code:
/* * Create region cap to be used for the actual test */ log("create managed dataspace"); Region_map_client sub_rm(rm.create(SUB_RM_SIZE)); enum { DS_SIZE = 4*4096 }; Ram_dataspace_capability ds = env.ram().alloc(DS_SIZE);
As I understand, we separately alloc DS_SIZE of physical memory and SUB_RM_SIZE of virtual memory? How I can specify here what exact range of virtual addresses I need to preserve?
later I see 2 different way to attach to the same ds (physical memory portion):
log("attach RAM ds to a fixed position at sub rm");
enum { DS_SUB_OFFSET = 4096 }; if ((addr_t)sub_rm.attach_at(ds, DS_SUB_OFFSET, 0, 0) != DS_SUB_OFFSET) fail("attach_at return-value mismatch»); and char *sub_rm_base = env.rm().attach_at(sub_rm.dataspace(), local_attach_addr);
What is a difference between them (except difference in DS_SUB_OFFSET vs local_attach_addr)? More specifically, between returned addresses from sub_rm.attach_at() and env.rm().attach_at()?
This feature would work independently from the 'Libc::Mem_alloc'. I think that the use of 'Libc::Mem_alloc' by 'mmap' is not the right way to go when taking mmap beyond it current scope. In particular,
We need to register our virtual address range where mmap mappings with fixed addresses are supposed to go inside allocator of libc, as I understand. so, it works only if we pre-load Libc::Mem_alloc by our own area backend by real memory? otherwise how this alloc will take memory from it?
'Libc::Mem_alloc' will eventually be swapped out by a better allocator (such as jemalloc). Extending its interface (as suggested by your patch) will make this step more complicated.
Sincerely, Alexander
Hello Alexander,
as I see, current interface of libc alloc() assume that all memory stored inside is committed (has been called _ds_pool.expand()), so, I can’t reserve memory and populate it by demand naturally, without libc alloc changes?
For this reason, I wrote in my previous email:
"This feature would work independently from the 'Libc::Mem_alloc'. I think that the use of 'Libc::Mem_alloc' by 'mmap' is not the right way to go when taking mmap beyond it current scope."
By the way, I see a couple of references in genode docs to the way to "use managed dataspace" and "populate it by phys memory by demand", while no references to code samples provided? How I can reserve virtual memory range in component without backend storage, in what allocator?
A high-level description is provided by the Genode-Foundations book:
https://genode.org/documentation/genode-foundations/20.05/architecture/Core_...
You may find the use of the RM service by the JVM instructive:
https://github.com/genodelabs/genode-world/blob/master/src/app/jdk/lib/jvm/o...
For further use cases, consider grepping the source tree for the corresponding header:
cd genode/repos grep -r "<rm_session/connection" *
What is a difference between them (except difference in DS_SUB_OFFSET vs local_attach_addr)? More specifically, between returned addresses from sub_rm.attach_at() and env.rm().attach_at()?
Whereas 'env.rm().attach_at' refers to the virtual address space of the component, 'sub_rm.attach_at' refers to the layout of the managed dataspace created via the RM service. The address of the former is a virtual address, the address of the latter is an offset relative to the beginning of the managed dataspace.
otherwise how this alloc will take memory from it?
By allocating RAM dataspaces directly via 'env.ram().alloc' and attaching it to an offset within the managed dataspace.
Cheers Norman
Hi Norman,
sorry to bother you
Problem I see here is a free operation: it just free the block with requested address, and do not touch others, gap-related blocks. For me it is not obvious that they need to be free, because in the code (e.g. by unmap()) I do not see explicit free of other allocated structures, in particular, dataspace extension or related block metadata.
That's another reason why the mmap support needs additional infrastructure. I think of a registry of mmap mappings. The registry entries store all the meta data required. They can be allocated on the 'Kernel::_heap'. The actual memory is allocated as RAM dataspaces. The 'Libc::Mem_alloc' is not required.
can you give me an example how I can obtain this Kernel::_heap?
initially I try something like this (from libc)
/* get Kernel::_heap from Libc */ Genode::Allocator &alloc = reinterpret_cast<Libc::Env_implementation &>(env).vfs_env().alloc();
/* and use it as aux data storage */ vm_reg.construct(env, reinterpret_cast<Genode::Heap &>(alloc));
but obvious that this will not work normally because (again) metadata space can’t be taken from this «main» process heap (this is the same address space for component which can clash with main program allocation).
I suspect that if I make inside my component something like Heap _heap { _env.ram(), _env.rm() };
then again I will hit the same problem of clash of local virtual address with this metadata storage? How to «hide» this data from component programs? Or I can’t hide and need artificially pre-alloc area for such data and make separate heap based on this?
Do you have any samples?
Sincerely, Alexander
Hello Alexander,
On 21.09.20 23:33, Alexander Tormasov via users wrote:
can you give me an example how I can obtain this Kernel::_heap?
initially I try something like this (from libc)
/* get Kernel::_heap from Libc */ Genode::Allocator &alloc = reinterpret_cast<Libc::Env_implementation &>(env).vfs_env().alloc();
/* and use it as aux data storage */ vm_reg.construct(env, reinterpret_cast<Genode::Heap &>(alloc));
but obvious that this will not work normally because (again) metadata space can’t be taken from this «main» process heap (this is the same address space for component which can clash with main program allocation).
I suspect that if I make inside my component something like Heap _heap { _env.ram(), _env.rm() };
then again I will hit the same problem of clash of local virtual address with this metadata storage? How to «hide» this data from component programs? Or I can’t hide and need artificially pre-alloc area for such data and make separate heap based on this?
the libc already contains such an internal 'Heap' object. It is created at [1] and passed to the constructor of the 'Libc::Kernel' [2]. The Libc::Kernel keeps a reference to this 'Heap' instance [3]. When I spoke of 'Kernel::_heap', I was referring to this instance.
Just below [1], you can see that the heap reference is passed to the 'Libc::init_fd_alloc' function. This way, the compilation unit fd_alloc.cc [4] gets hold of this 'Kernel::Heap' instance and can thereby use it for the allocation of libc-internal meta data, in this case 'File_descriptor' objects.
The 'file_operations.cc' compilation unit that hosts the 'mmap' implementation already has a 'Libc::init_file_operations' function but the function does not take an 'Allocator &' as argument. In order to allow this compilation unit to use the 'Kernel::_heap', you'll have to extend this function with an 'Allocator &' argument (you may take fd_alloc.cc as blueprint), and adjust the call at [5] accordingly (passing '_heap' as second argument).
[1] https://github.com/genodelabs/genode/blob/master/repos/libports/src/lib/libc... [2] https://github.com/genodelabs/genode/blob/master/repos/libports/src/lib/libc... [3] https://github.com/genodelabs/genode/blob/master/repos/libports/src/lib/libc... [4] https://github.com/genodelabs/genode/blob/master/repos/libports/src/lib/libc... [5] https://github.com/genodelabs/genode/blob/master/repos/libports/src/lib/libc...
Cheers Norman
Hello Norman, I make once again attempt to implement the mmap with MAP_FIXED|MAP_ANON Now I think that it is closer to what I need for golang runtime, at least it works here. Could you take a look?
Due to specific of my tests it appears in 2 patches to genode master (Oct 23 2020): https://github.com/tor-m6/genode/commit/0aa7bb95ce451dc99d2cb76b1a4da09d6478... and 2 files (repos/libports/lib/symbols/libc and repos/ports/src/noux-pkg/go_app/anon_mmap.cc with main algorithm) from here https://github.com/tor-m6/genode/commit/0248ae78851833cfef64068e99764f4fddd2...
(sorry, still have some whitespaces and code style changes in patches)
Question: is it worth to spend more efforts to bring it to the mainstream genode? or continue keeping them in the private repository as now?
Same question for golang support… It can be potentially stored in world repo - is it worth to spend efforts? May be someone interested to takeover this work?
Sincerely, Alexander
Hi Alexander,
On 23.11.20 09:26, Alexander Tormasov via users wrote:
Now I think that it is closer to what I need for golang runtime, at least it works here. Could you take a look?
please bear with me. I'm currently busy with the upcoming Genode release. I'll be happy to give you feedback by mid of next week.
Question: is it worth to spend more efforts to bring it to the mainstream genode? or continue keeping them in the private repository as now?
Same question for golang support… It can be potentially stored in world repo - is it worth to spend efforts? May be someone interested to takeover this work?
I appreciate you bringing up this question.
Yes, I would very much welcome turning your line of work into a community effort. The world repository is certainly the best place.
- It would be consistent with the runtimes for Java and Python, which are already maintained there.
- It would give the project a wider exposure compared to a private repository. So Genode users are more likely to pick it up.
- Since Genode Labs routinely uses the world repository, the repository receives (a certain degree of) maintenance from our side. E.g., we try to keep it up to Genode API changes.
- Since interest in a Go runtime was raised several times in the past, I'm positive that other members of our community might step up for improving it once we have good and visible starting point.
- I think that some of the issued we discussed on the mailing list would be best addressed by changes in Genode's infrastructure such as the libc. Having the use cases of these changes readily available in the world repository certainly increases the motivation to take on such changes.
For these reasons, I'd like to encourage you to contribute your work to the world repository. As the first step, please open a corresponding issue at [1]. We will guide you from there.
[1] https://github.com/genodelabs/genode-world/issues
Cheers Norman
Hi Norman, issue submitted: https://github.com/genodelabs/genode-world/issues/244
while it has at least 4 separate issues - golang runtime port itself, and some libraries/features support from genode side… May be it worth to have them as separate issues?
25 нояб. 2020 г., в 13:25, Norman Feske norman.feske@genode-labs.com написал(а):
For these reasons, I'd like to encourage you to contribute your work to the world repository. As the first step, please open a corresponding issue at [1]. We will guide you from there.
Now I think that it is closer to what I need for golang runtime, at least it works here. Could you take a look?
please bear with me. I'm currently busy with the upcoming Genode release. I'll be happy to give you feedback by mid of next week.
I just ported current set of patches to 20.11. please, take a look at https://github.com/tor-m6/genode.git branch 20.11 main implementation of mmap part temporary in 1 file and 1 patch anon_mmap.cchttp://anon_mmap.cc/ from https://github.com/tor-m6/genode/commit/cf79a564e1b004fef5660e84d01d6c290674... and genode files minor modifications https://github.com/tor-m6/genode/commit/41b57fd06516c9b33b4b06e1f91cb134a3da...
Hallo Alexander,
Now I think that it is closer to what I need for golang runtime, at least it works here. Could you take a look?
I just ported current set of patches to 20.11. please, take a look at https://github.com/tor-m6/genode.git branch 20.11 main implementation of mmap part temporary in 1 file and 1 patch anon_mmap.cchttp://anon_mmap.cc/ from https://github.com/tor-m6/genode/commit/cf79a564e1b004fef5660e84d01d6c290674... and genode files minor modifications https://github.com/tor-m6/genode/commit/41b57fd06516c9b33b4b06e1f91cb134a3da...
I'm going to take a look at your changes and probably will get back to you with some Golang related questions in general (it might take one or two days).
Regards Josef
Hello Alexander,
as predicted I am back with some questions.
First, thanks for your efforts of bringing Golang to Genode. I took a look at the commits on your topic branch and your README was very helpful in getting started; especially the current limitations section reflects questions I am going to ask. Please bear with me if some questions sound obvious but I lack the understanding of the inner workings of Golang's run-time and I did not follow all your previous E-Mails on the ML.
So, I checked out your 20.11 branch and tried to follow your instructions. Unfortunately, I could not compile libgo out-of-the-box because libtool was complaining about “unrecognized options” and I stopped at this point.
This brings me to my first question
* Is there a technical reason for executing the auto-tools build system rather than porting the libraries, i.e., create the .mk files for Genode's build system?
The instructions in the README lack the motivation behind that decision.
and furthermore
* Is it strictly necessary to solely produce static libraries or could we get away with a dynamic shared-object, libgo.lib.so, that incorporates everything and the target application is linked against or is it possible to have multiple shared-objects. After all, there is already a libffi port - ffi - that is built as .lib.so.
While stepping through the libgo commit I noticed the patches:
* You choose to add a new run-time target, “inno”, that is based on NetBSD rather than using the existing FreeBSD target. For all intents and purposes, Genode appears to be FreeBSD for ported software due to using its libc¹. Is there a reason why that is not possible for Golang's run-time?
IIRC, on some platforms, e.g. Solaris(?), the normal Golang run-time solely uses the libc to interface with the OS instead of wrapping system-calls directly. Is it the same with gccgo's run-time?
With regard to the first questions, if we remove auto-tools from the equation we could shadow specific run-time parts where the upstream implementation does not work on Genode.
¹) Granted, our libc port is not feature-complete and might lack expected functionality, e.g. kqueue/sysctl or as you already noticed mmap handling, but we could take this as an opportunity to extended our libc port.
* I noticed that you use internal headers for implementing the stack handling. Is in this context the available Genode::Thread API not enough?
My somewhat naive point of view would be to use our POSIX layer wherever possible and only diverge where necessary - that only boils down to go-routine and stack handling or does it not?
Finally, the most important question:
* From what I can tell the port itself still somewhat feels like a proof-of-concept that as you mentioned in the README is the side result of a research project. It would require further clean-up before we could add it to Genode.
(This includes the current short-comings like missing context support for go-routines, reliance on -fno-omit-framepointer, etc. The support code that is currently part of the component should be moved to a central location like you already suggested.)
Would you be willing to do that clean-up work or is it from your point of view enough to just keep the port as an example in case somebody else wants to do that? Your reaction so far, creating the github issue, suggest that you do have some motivation.
I ask because in the past we got a welcome one-time contribution for Rust support on Genode. However, since we do not use Rust, we merely dragged it along and eventually removed it². As support for Golang has no immediate priority for us, we hesitate to spend our limited time on this at the moment.
That being said, I wrote our CI-tools web front end in Golang and running it on Genode certainly would be nice and could be a reason to nonetheless set some time aside to get that up and running :-). Judging by the reaction on your issue, we could very well make a community effort out of supporting Golang.
²) To not make the same mistake I imagine adding support to our tooling for creating Golang based components is necessary. Maybe it is worthwhile to extend 'goa'[sic] in that regard so that a Golang-developer can use 'go' to initially develop his or her application and later one use 'goa' to create a binary for Genode, while 'goa' does the heavy-lifting behind the scenes. I think you briefly touched the “go build → Makefile” topic in one of your mails. After all, rather than writing specific Genode components in Golang I guess people want to deploy their existing applications first and would be fine with interacting with Genode via the POSIX/VFS layer.
Regards Josef
So, I checked out your 20.11 branch and tried to follow your instructions. Unfortunately, I could not compile libgo out-of-the-box because libtool was complaining about “unrecognized options” and I stopped at this point.
sorry, forget to remove it - it does not matter, everything configured correctly even with them. To avoid misunderstanding I fix this and coding style in additional patch for the same 20.11 branch, please, take a look
This brings me to my first question
- Is there a technical reason for executing the auto-tools build system
rather than porting the libraries, i.e., create the .mk files for Genode's build system?
configuration of the system is VERY complex, result of 10+ years legacy code development as a part of gcc, and IMHO better to get rid of manual handling of it. In particular, list of files to be compiled for golang runtime sources generated «on the fly» and complied using libtool script with complex 2-3 times in a row generation process (from .m4 to Makefile and inside for every go package in relation to choosed subsystem)
Also, I assume that using «native» build system from gcc could simplify future port of newer version of golang runtime and related gcc libraries/parts
The instructions in the README lack the motivation behind that decision.
my porting experience is the main motivator - closer to original build system, easier to support it later.
and furthermore
- Is it strictly necessary to solely produce static libraries or could
we get away with a dynamic shared-object, libgo.lib.so, that incorporates everything and the target application is linked against or is it possible to have multiple shared-objects. After all, there is already a libffi port
- ffi - that is built as .lib.so.
I do not check this carefully, but standalone libffi and libffi from inside gcc distribution (which I port smoothly) -IMHO different beasts. At least, gcc build system assume part of it (e.g. generated file ffi.h) to be placed inside specific place of gcc build environment; and configuration options could be different (I do not check this in details). May be I am wrong, and we can use standard libffi (which is already ported)
about shared vs .a libraries … frankly, I do not remember exactly a reason, probably the same - default build system do generate .a file which is good for golang application. Do not investigated yet, may be it will be easy.
While stepping through the libgo commit I noticed the patches:
- You choose to add a new run-time target, “inno”, that is based on NetBSD
rather than using the existing FreeBSD target. For all intents and purposes, Genode appears to be FreeBSD for ported software due to using its libc¹. Is there a reason why that is not possible for Golang's run-time?
Golang runtime deeply integrated with OS services and sys calls, use a lot of them. Genode implementation of FreeBSD ones has significant implementation gaps. NetBSD is not that advanced, has smaller set of features and, therefore, size of gaps is also smaller. So, from plain porting perspective in the very beginning, volume of work for NetBSD version port seems significantly smaller than ones based on FreeBSD (which Genode port does not have full feature sets available in original OS). This was for me a reason. Again, may be on this stage we can add some features from freebsd version as well.
IIRC, on some platforms, e.g. Solaris(?), the normal Golang run-time solely uses the libc to interface with the OS instead of wrapping system-calls directly. Is it the same with gccgo's run-time?
There are some semantical differences between libc, sys calls, POSIX and even glibc. I can’t say that there are single simple answer - I see some random files related to subsystems whch partially belongs to any BSD-based implementation, some just directly call sys calls via generated golang or even c wrappers. This is a result of 11 years golang ports evolution and support of 9 very different subsystems. Runtime developers try to mimic/mask differences in features from main code, not always correctly IMHO.
With regard to the first questions, if we remove auto-tools from the equation we could shadow specific run-time parts where the upstream implementation does not work on Genode.
This is not that straightforward and easy task. For example, Golang use concept of packages, and packages do have initializers (like constructors) which called before main.main(). So, even if you do not use some of them, they can potentially call low-level functions (e.g. system calls) and, if they are not implemented, or we put dummy stubs - logic can fail and we will not able to start the program.
This refer to more common question about porting strategy. What we can do is 1. Modify or not modify golang runtime. 2. Modify or not modify Genode OS services (especially on per-OS basis) 3. Do create universal dummy/proxy functions to mimic problems from runtime. I think that 2 is most complicated. I use partially approach 1 from above (by introducing additional subsystem with minimization of changes in runtime), and file the rest by 3. Unfortunately, due to historical reasons, I have all 3 approaches combined - some dirty hacks on golang go runtime to avoid call of unimplemented functions, Provide some dummy implementation of small set of system calls, and small modification of Genode related to ANON mmap and set/getcontext support.
¹) Granted, our libc port is not feature-complete and might lack expected functionality, e.g. kqueue/sysctl or as you already noticed mmap handling, but we could take this as an opportunity to extended our libc port.
In that case it is relatively easy to return support of some features what I manually cut from go code (they are relatively isolated), eg take them from freeBSD golang runtime.
- I noticed that you use internal headers for implementing the stack
handling. Is in this context the available Genode::Thread API not enough?
My somewhat naive point of view would be to use our POSIX layer wherever possible and only diverge where necessary - that only boils down to go-routine and stack handling or does it not?
No, because to implement context switching I need to create a support for set/getcontext family of functions. They are from one point of view, similar to set jump/longjump, but they also has internal call to set/restore signals mask using sys call API which is absent in Genode. And, moreover, it has hidden problem because Genode threads should keep their stack in some pre-defined area which is tracked in myself() function. goroutines behaves like user-space fibers, they should be switched manually using manually crafted context. Makecontex() should obtain stack from mentioned area - this is the main reason why I use internal function to create my own stack. Posix threads do not have such problem, so, I can’t use just them.
Finally, the most important question:
- From what I can tell the port itself still somewhat feels like a
proof-of-concept that as you mentioned in the README is the side result of a research project. It would require further clean-up before we could add it to Genode.
(This includes the current short-comings like missing context support for go-routines, reliance on -fno-omit-framepointer, etc. The support code that is currently part of the component should be moved to a central location like you already suggested.)
Would you be willing to do that clean-up work or is it from your point of view enough to just keep the port as an example in case somebody else wants to do that? Your reaction so far, creating the github issue, suggest that you do have some motivation.
I am limited in time which I can spend around this project. Anyway, I think that work done can be good basis for future development and in general I want to bring it to the level which will be suitable for people interested in utilization and continuation. For better understanding, I had a project to port docker container support to some microkernel os, which started (as a experimental testbed) from Genode. Initially I do not assume to deal with go at all, but later found that it probably will be the easiest step (taking into account the availability of gccgo in Genode). So, I make initial dirty port of runtime as proof-of-concept and try to compile docker itself. I found a lot of interesting things about direct golang connection to os services/syscalls on different platforms, and have finished minimal port of it using makefiles (generated ones), not «go build». So, golang runtime port became «side project» of it.
In general I assume that I can do some work (at least this year) toward bringing the project to more usable state, together with potential testbed and relation of improvement of my knowledge about Genode in context of another potential project related to port of part of OS (related to persistency model).
Even docker port can be considered later (having in mind my experience of its port attempt to Genode)
I ask because in the past we got a welcome one-time contribution for Rust support on Genode. However, since we do not use Rust, we merely dragged it along and eventually removed it². As support for Golang has no immediate priority for us, we hesitate to spend our limited time on this at the moment.
That being said, I wrote our CI-tools web front end in Golang and running it on Genode certainly would be nice and could be a reason to nonetheless set some time aside to get that up and running :-). Judging by the reaction on your issue, we could very well make a community effort out of supporting Golang.
²) To not make the same mistake I imagine adding support to our tooling for creating Golang based components is necessary. Maybe it is worthwhile to extend 'goa'[sic] in that regard so that a Golang-developer can use 'go' to initially develop his or her application and later one use 'goa' to create a binary for Genode, while 'goa' does the heavy-lifting behind the scenes. I think you briefly touched the “go build → Makefile” topic in one of your mails. After all, rather than writing specific Genode components in Golang I guess people want to deploy their existing applications first and would be fine with interacting with Genode via the POSIX/VFS layer.
In general I think that it could be a good idea to list a set of sub task to make golang port and related infrastructure more feasible
In particular, I think that it is necessary to create in genode mainstream the following 1. setcontext family of calls (I am not happy with current one) 2. Anon mmap extended support (better than current one, potentially mine can be intermediate version) 3. Go as language support in genode build infrastructure 4. Some of aux libraries could be compiled as a part of tools (libatomic/libbacktrace, may be even libffi)
Everything else (including libgo and test apps) could be started as a part of “world” repo
Probably we need to specify a list of things to cleanup/rebuild current port
What do you think?
Hello Alexander,
[…]
thanks for answering my questions so thoroughly, much appreciated.
Unfortunately, I am currently occupied with other tasks and only skimmed over your reposonse so far. From what I gathered things are not as easy as I thought them too be, which is whay I am going to spend some time over christmas to look into Golang's internals (I guess on that account TamaGo is probably also worth a look) to gain a bit more insight and will get back to you after that.
Regards Josef
Hello,
Unfortunately, I am currently occupied with other tasks and only skimmed over your reposonse so far. From what I gathered things are not as easy as I thought them too be, which is whay I am going to spend some time over christmas to look into Golang's internals (I guess on that account TamaGo is probably also worth a look) to gain a bit more insight and will get back to you after that.
golang runtime is around 800k source lines with deep OS sys calls integration, I spent 1-2 month to dig. So, feel free to ask about, this could speedup things
Hello all,
We at gapfruit are also interested in golang runtime support and would like to invest some time into it. @Alexander, thank you for paving the way for golang support on Genode. Now might be a good time to help to solve some of the remaining issues, so we can keep up the momentum. Will it be ok if we join for a while? We will need some time to dive into the work you have done already and evaluate how much time we can invest. We will be glad to discuss the issues you think we can help with.
Sincerely, Sergey
--- Sergey Platonov gapfruit AG Baarerstrasse 135 6300 Zug +41 762 444 560 sergey.platonov@gapfruit.com
Alexander Tormasov via users писал 23.11.2020 09:26:
Hello Norman, I make once again attempt to implement the mmap with MAP_FIXED|MAP_ANON Now I think that it is closer to what I need for golang runtime, at least it works here. Could you take a look?
Due to specific of my tests it appears in 2 patches to genode master (Oct 23 2020): https://github.com/tor-m6/genode/commit/0aa7bb95ce451dc99d2cb76b1a4da09d6478... and 2 files (repos/libports/lib/symbols/libc and repos/ports/src/noux-pkg/go_app/anon_mmap.cc with main algorithm) from here https://github.com/tor-m6/genode/commit/0248ae78851833cfef64068e99764f4fddd2...
(sorry, still have some whitespaces and code style changes in patches)
Question: is it worth to spend more efforts to bring it to the mainstream genode? or continue keeping them in the private repository as now?
Same question for golang support… It can be potentially stored in world repo - is it worth to spend efforts? May be someone interested to takeover this work?
Sincerely, Alexander _______________________________________________ Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users
Will be happy to help! Please take a look to readme and to my reply to Josef Söntgen about problems mentioned.
I think that we need first to make a plan of fixes to bring current state of the project to more «industrial» state - both in genode and in golang runtime code. Some points listened in my reply … I think that first and most important is to make «portable» (ARM/x86 32/64 bit, and signal mask processing) implementation of setcontext/getcontex/makecontext - this is crucial for golang goroutines support...
7 дек. 2020 г., в 18:30, Sergey Platonov <sergey.platonov@gapfruit.commailto:sergey.platonov@gapfruit.com> написал(а):
Hello all,
We at gapfruit are also interested in golang runtime support and would like to invest some time into it. @Alexander, thank you for paving the way for golang support on Genode. Now might be a good time to help to solve some of the remaining issues, so we can keep up the momentum. Will it be ok if we join for a while? We will need some time to dive into the work you have done already and evaluate how much time we can invest. We will be glad to discuss the issues you think we can help with.
Sincerely, Sergey
--- Sergey Platonov gapfruit AG Baarerstrasse 135 6300 Zug +41 762 444 560 sergey.platonov@gapfruit.commailto:sergey.platonov@gapfruit.com
Alexander Tormasov via users писал 23.11.2020 09:26:
Hello Norman, I make once again attempt to implement the mmap with MAP_FIXED|MAP_ANON Now I think that it is closer to what I need for golang runtime, at least it works here. Could you take a look?
Due to specific of my tests it appears in 2 patches to genode master (Oct 23 2020): https://github.com/tor-m6/genode/commit/0aa7bb95ce451dc99d2cb76b1a4da09d6478... and 2 files (repos/libports/lib/symbols/libc and repos/ports/src/noux-pkg/go_app/anon_mmap.cchttp://anon_mmap.cc with main algorithm) from here https://github.com/tor-m6/genode/commit/0248ae78851833cfef64068e99764f4fddd2...
(sorry, still have some whitespaces and code style changes in patches)
Question: is it worth to spend more efforts to bring it to the mainstream genode? or continue keeping them in the private repository as now?
Same question for golang support… It can be potentially stored in world repo - is it worth to spend efforts? May be someone interested to takeover this work?
Sincerely, Alexander _______________________________________________ Genode users mailing list users@lists.genode.orgmailto:users@lists.genode.org https://lists.genode.org/listinfo/users
forget to mention issue with some proposals related to golang runtime https://github.com/genodelabs/genode-world/issues/244
8 дек. 2020 г., в 16:34, Alexander Tormasov via users users@lists.genode.org написал(а):
Will be happy to help! Please take a look to readme and to my reply to Josef Söntgen about problems mentioned.
I think that we need first to make a plan of fixes to bring current state of the project to more «industrial» state - both in genode and in golang runtime code. Some points listened in my reply … I think that first and most important is to make «portable» (ARM/x86 32/64 bit, and signal mask processing) implementation of setcontext/getcontex/makecontext - this is crucial for golang goroutines support...
7 дек. 2020 г., в 18:30, Sergey Platonov <sergey.platonov@gapfruit.commailto:sergey.platonov@gapfruit.com> написал(а):
Hello all,
We at gapfruit are also interested in golang runtime support and would like to invest some time into it. @Alexander, thank you for paving the way for golang support on Genode. Now might be a good time to help to solve some of the remaining issues, so we can keep up the momentum. Will it be ok if we join for a while? We will need some time to dive into the work you have done already and evaluate how much time we can invest. We will be glad to discuss the issues you think we can help with.
Sincerely, Sergey
--- Sergey Platonov gapfruit AG Baarerstrasse 135 6300 Zug +41 762 444 560 sergey.platonov@gapfruit.commailto:sergey.platonov@gapfruit.com
Alexander Tormasov via users писал 23.11.2020 09:26:
Hello Norman, I make once again attempt to implement the mmap with MAP_FIXED|MAP_ANON Now I think that it is closer to what I need for golang runtime, at least it works here. Could you take a look?
Due to specific of my tests it appears in 2 patches to genode master (Oct 23 2020): https://github.com/tor-m6/genode/commit/0aa7bb95ce451dc99d2cb76b1a4da09d6478... and 2 files (repos/libports/lib/symbols/libc and repos/ports/src/noux-pkg/go_app/anon_mmap.cchttp://anon_mmap.cc/ with main algorithm) from here https://github.com/tor-m6/genode/commit/0248ae78851833cfef64068e99764f4fddd2...
(sorry, still have some whitespaces and code style changes in patches)
Question: is it worth to spend more efforts to bring it to the mainstream genode? or continue keeping them in the private repository as now?
Same question for golang support… It can be potentially stored in world repo - is it worth to spend efforts? May be someone interested to takeover this work?
Sincerely, Alexander _______________________________________________ Genode users mailing list users@lists.genode.orgmailto:users@lists.genode.org https://lists.genode.org/listinfo/users
_______________________________________________ Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users
Hi all
@Alex, sorry for not replying for a while: we were diving into your work and prioritizing issues. So, far we have decided that some restructure needs to be done: we will move most of it to genode-world. Also, we will take a look at the context-related functions. Do you think porting some existing libraries, like libpcl or boost.context will do the trick?
@Jozef, should we create issues in the genode-world repo?
Sincerely, Sergey
--- gapfruit AG Baarerstrasse 135 6300 Zug +41 762 444 560 sergey.platonov@gapfruit.com
Alexander Tormasov via users писал 08.12.2020 17:07:
forget to mention issue with some proposals related to golang runtime https://github.com/genodelabs/genode-world/issues/244
8 дек. 2020 г., в 16:34, Alexander Tormasov via users users@lists.genode.org написал(а):
Will be happy to help! Please take a look to readme and to my reply to Josef Söntgen about problems mentioned.
I think that we need first to make a plan of fixes to bring current state of the project to more «industrial» state - both in genode and in golang runtime code. Some points listened in my reply ... I think that first and most important is to make «portable» (ARM/x86 32/64 bit, and signal mask processing) implementation of setcontext/getcontex/makecontext - this is crucial for golang goroutines support...
7 дек. 2020 г., в 18:30, Sergey Platonov sergey.platonov@gapfruit.com написал(а):
Hello all,
We at gapfruit are also interested in golang runtime support and would like to invest some time into it. @Alexander, thank you for paving the way for golang support on Genode. Now might be a good time to help to solve some of the remaining issues, so we can keep up the momentum. Will it be ok if we join for a while? We will need some time to dive into the work you have done already and evaluate how much time we can invest. We will be glad to discuss the issues you think we can help with.
Sincerely, Sergey
Sergey Platonov gapfruit AG Baarerstrasse 135 6300 Zug +41 762 444 560 sergey.platonov@gapfruit.com
Alexander Tormasov via users писал 23.11.2020 09:26: Hello Norman, I make once again attempt to implement the mmap with MAP_FIXED|MAP_ANON Now I think that it is closer to what I need for golang runtime, at least it works here. Could you take a look?
Due to specific of my tests it appears in 2 patches to genode master (Oct 23 2020): https://github.com/tor-m6/genode/commit/0aa7bb95ce451dc99d2cb76b1a4da09d6478... and 2 files (repos/libports/lib/symbols/libc and repos/ports/src/noux-pkg/go_app/anon_mmap.cc with main algorithm) from here https://github.com/tor-m6/genode/commit/0248ae78851833cfef64068e99764f4fddd2...
(sorry, still have some whitespaces and code style changes in patches)
Question: is it worth to spend more efforts to bring it to the mainstream genode? or continue keeping them in the private repository as now?
Same question for golang support... It can be potentially stored in world repo - is it worth to spend efforts? May be someone interested to takeover this work?
Sincerely, Alexander _______________________________________________ Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users
_______________________________________________ Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users _______________________________________________ Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users
So, far we have decided that some restructure needs to be done: we will move most of it to genode-world. Also, we will take a look at the context-related functions. Do you think porting
tor> ok. while question - how to call «configure» from it? the main reason why I make it inside «port/noux» - it allow to call configure from target.mk. I am against approach to extract list of files and compile them. This operation need to be done every new version of soft to be port release… IMHO this should be as seamless as possible, best of all involve original build environment.
some existing libraries, like libpcl or boost.context will do the trick?
tor> Sorry for not knowing - but seems that libpcl is a kind of image processing library using boost?
about boost.context. In general, I think that it will be good to have it ported. but for particular go runtime I see here 2 problems
1. go runtime directly call getcontext/setcontext/makecontext (not call swap context, as I remember), so their semantic need to be emulated exactly. At least, related to «kernel» signals (get/setsigmask). And, important, need to create stacks for threads to be switched from dedicated area of Golang (not as it implemented in boost.context via standard heap)
2. boost do require C++ specific initialisation which also need to be called during construction before C main() which is called from golang runtime. It could have some limitations/tricks. E.g. too many unnecessary C++ internal functions/code will be linked to the executable, and/or tricky sequence. Typically such «rich» library hard to split and port partially, I doubt that boost porting is a separate complex task tightly bounded to the compiler version/llvm/etc.
@Jozef, should we create issues in the genode-world repo?
I created one single issue about golang port - may be we need separate ones for components to be kept in main repo (outside of world) may be it worth to move this discussion here?
Hi all
tor> ok. while question - how to call «configure» from it? the main reason why I make it inside «port/noux» - it allow to call configure from target.mk. I am against approach to extract list of files and compile them. This operation need to be done every new version of soft to be port release… IMHO this should be as seamless as possible, best of all involve original build environment.
Noux has been declared as retired recently [1].
I am against approach to extract list of files and compile them. This operation need to be done every new version of soft to be port release… IMHO this should be as seamless as possible, best of all involve original build environment.
You right, that would be wonderful, but I am not sure if it is possible. We will try to take a look at the `goa` project. @Norman @Jozef, do you have an opinion here?
tor> Sorry for not knowing - but seems that libpcl is a kind of image processing library using boost?
I've meant this one [2], sorry for the confusion:
- go runtime directly call getcontext/setcontext/makecontext
(not call swap context, as I remember), so their semantic need to be emulated exactly. At least, related to «kernel» signals (get/setsigmask). And, important, need to create stacks for threads to be switched from dedicated area of Golang (not as it implemented in boost.context via standard heap)
Yes, as far as I know, any coroutines library has an implementation of getcontext/setcontext/makecontext and my idea to use the existing implementation.
- boost do require C++ specific initialisation which also need
to be called during construction before C main() which is called from golang runtime. It could have some limitations/tricks. E.g. too many unnecessary C++ internal functions/code will be linked to the executable, and/or tricky sequence. Typically such «rich» library hard to split and port partially, I doubt that boost porting is a separate complex task tightly bounded to the compiler version/llvm/etc.
I agree boost is hard to split. Hopefully, they will decide to fix it in the upcoming releases [3]
I created one single issue about golang port - may be we need separate ones for components to be kept in main repo (outside of world) may be it worth to move this discussion here?
Sorry, I have failed to find it.
[1] https://genode.org/documentation/release-notes/20.05#Retired_Noux_runtime_en... [2] http://www.xmailserver.org/libpcl.html [3] https://lists.boost.org/Archives/boost/2020/11/250354.php
Have a nice weekend! Sergey -- gapfruit AG Baarerstrasse 135 6300 Zug +41 762 444 560 sergey.platonov@gapfruit.com
On Dienstag, 15. Dezember 2020 16:48:16 CET, Alexander Tormasov wrote:
So, far we have decided that some restructure needs to be done: we will move most of it to genode-world. Also, we will take a look at the context-related functions. Do you think porting
tor> ok. while question - how to call «configure» from it? the main reason why I make it inside «port/noux» - it allow to call configure from target.mk. I am against approach to extract list of files and compile them. This operation need to be done every new version of soft to be port release… IMHO this should be as seamless as possible, best of all involve original build environment.
some existing libraries, like libpcl or boost.context will do the trick?
tor> Sorry for not knowing - but seems that libpcl is a kind of image processing library using boost?
about boost.context. In general, I think that it will be good to have it ported. but for particular go runtime I see here 2 problems
- go runtime directly call getcontext/setcontext/makecontext
(not call swap context, as I remember), so their semantic need to be emulated exactly. At least, related to «kernel» signals (get/setsigmask). And, important, need to create stacks for threads to be switched from dedicated area of Golang (not as it implemented in boost.context via standard heap)
- boost do require C++ specific initialisation which also need
to be called during construction before C main() which is called from golang runtime. It could have some limitations/tricks. E.g. too many unnecessary C++ internal functions/code will be linked to the executable, and/or tricky sequence. Typically such «rich» library hard to split and port partially, I doubt that boost porting is a separate complex task tightly bounded to the compiler version/llvm/etc.
@Jozef, should we create issues in the genode-world repo?
I created one single issue about golang port - may be we need separate ones for components to be kept in main repo (outside of world) may be it worth to move this discussion here?
tor> ok. while question - how to call «configure» from it? the main reason why I make it inside «port/noux» - it allow to call configure from target.mk. I am against approach to extract list of files and compile them. This operation need to be done every new version of soft to be port release… IMHO this should be as seamless as possible, best of all involve original build environment.
Noux has been declared as retired recently [1].
I am not talking about usage of noux, I am talking about build system. e.g. standard libraries can’t be build using «configure» call - while targets from repos/ports can. I am not sure that exactly noux dir is necessary, probably it could be the same in repos/world. need to check...
I am against approach to extract list of files and compile them. This operation need to be done every new version of soft to be port release… IMHO this should be as seamless as possible, best of all involve original build environment.
You right, that would be wonderful, but I am not sure if it is possible. We
sure that can - we need to update a bit sources and try to run standard environment using configure.
will try to take a look at the `goa` project. @Norman @Jozef, do you have an opinion here?
I suspect that it run only ontop of linux
tor> Sorry for not knowing - but seems that libpcl is a kind of image processing library using boost?
I've meant this one [2], sorry for the confusion:
- go runtime directly call getcontext/setcontext/makecontext
(not call swap context, as I remember), so their semantic need to be emulated exactly. At least, related to «kernel» signals (get/setsigmask). And, important, need to create stacks for threads to be switched from dedicated area of Golang (not as it implemented in boost.context via standard heap)
Yes, as far as I know, any coroutines library has an implementation of getcontext/setcontext/makecontext and my idea to use the existing implementation.
As I understand, there are no such files (set/get context) in sources of libpcl, as mentioned in the docs, it can use them as interface.
Moreover, original port of libc from FreeBSD used for Genode do contain some code related to implementations of them - but it was not ported yet (as I understand because it call signal-related syscalls from asm, @norman can correct me if I am wrong. May be variable args could also be a problem).
I created one single issue about golang port - may be we need
separate ones for components to be kept in main repo (outside of world) may be it worth to move this discussion here?
Sorry, I have failed to find it.
if was mentioned in my previous post: https://github.com/genodelabs/genode-world/issues/244
Hi Alexander,
I am not talking about usage of noux, I am talking about build system. e.g. standard libraries can’t be build using «configure» call - while targets from repos/ports can. I am not sure that exactly noux dir is necessary, probably it could be the same in repos/world. need to check...
the name is just historic. When I removed Noux, I actually wanted to rename it but had not found a suitable name yet. Maybe we should rename the directory to autoconf-pkgs?
will try to take a look at the `goa` project. @Norman @Jozef, do you have an opinion here?
I suspect that it run only ontop of linux
The testing workflow of Goa targets Linux currently. But the resulting binaries and depot archives work across all kernels.
As I understand, there are no such files (set/get context) in sources of libpcl, as mentioned in the docs, it can use them as interface.
Moreover, original port of libc from FreeBSD used for Genode do contain some code related to implementations of them - but it was not ported yet (as I understand because it call signal-related syscalls from asm, @norman can correct me if I am wrong. May be variable args could also be a problem).
I'm admittedly not well versed in this corner of the libc.
Regarding cooperative threading models in current use, Genode's Thread API provides the method 'alloc_secondary_stack'. We use that in combination with setjmp/longjmp at several places (dde_linux, dde_bsd, libc internal) to implement cooperative control flows.
Cheers Norman
-
Alexander Tormasov -
Josef Söntgen -
Norman Feske -
Sergey Platonov