[set,get,make,swap]context from glibc for amd64 implementation and question for Norman

[Alexander Tormasov]

Hello Josef,


the problem that this family together with headers and ucontext structure
is a part of general libc.
how I can mask its usage, e.g. in configure scripts?

That is indeed a problem and one of the reasons why I already mentioned that
using configure for building *libraries* for Genode is not the way to go.
For the time being the manual effort of writing library .mk files to bring
a library to Genode is the preferred way.

In doing so the library is better integrated into Genode's build-system (amongst
others, this also eases cross compilation), you gain more insight into the target
you are currently porting as you are required to check the compilation units as
well as the used compiler flags. Generally speaking you are more in control of
the compilation process and do not rely on the original “black-box”.

In the case of GCC's libgo the overhead of this manual approach should not be
so high because I would imagine it would be updated together with the tool-chain,
which is around every 2 years (I do not how intertwined the library is with
the compiler).

The problem is that current confgure from gcc do generate a list of files on the fly, depending of the subsystem. It is not easy to extract them even from listing: in .go files you can use
pseudo comment like
// +build aix linux darwin dragonfly freebsd openbsd netbsd solaris
where the fact of compilation is defined on the correspondence of subsystem, file name (e.g. file_bsd.go included into any psd-related systems, memmove_linux_amd64_test.go will be included only for linux) and arch (cipherhw_amd64.go).

Moreover, some files are generated «on the fly» during configure run and then preprocessed couple times to generate some other files .c or .go files, and they should be placed to inside build directory (not inside source tree) for compilation.

in theory, I can copy all files and add them into all arch/etc, but this is error-prone process (especially repeating after 2 years when you make it last time ;-).
IMHO easier to run original configure, as I do now - I see all compilation options in the listing and know what’s happens. What do you think?

Also we have generated file «macros.h» from your patch with offsets of current ucontext.h structures which should be manually regenerated from attached to mcontext.c test after any changes in these structures (not sure how to force error in case of structures update).


That being said, with better tooling in place it is quite possible to reuse
the existing build tools in the future.

see above, this is definitely not that straight forward



Due to specific of Genode we have another potential problem - we can’t use
standard malloc() for stack allocation because stack should be from specific
memory area to allow context switch, so, external packages compatibility will
be limited anyhow.

I would argue that if go-packages operate on a level where they need to know
where or how the stack is allocated you properly want a Genode-specific backend
anyway. Or is that an implementation detail that leaks through the standard
library?


yes, definitely so - the only place where stack for new threads is generated is hidden inside library in proc.c (it done for every goroutine which could be plenty for application)


And, last but not least - setcontex/getcontext should store and restore signal
context (including signal handlers) which is not that clear for me in Genode.
Currently is is limited, and where it should go later? how we can store/restore
state of signals for current thread?

That is a good question where I do not have an answer. In general the
signal handling in our libc is minimal and we rely on user-code to call
an preemtion point (“system-call”, I/O) to deliver signals, i.e., there
is no preemption through signals.


hm, anyway, may be it worth to call a setprocmask()/__libc_sigprocmask() from libc,
just to be sure that if it will be supported then we can run it (and call preemption,
not sure that it will work)


Everything else for golang support I will have in world, this is the
only part of core (may be together with extended anonymous mmap
support).

Please see my branch [1] where I have changed your code in a way that the
whole mcontext implementation is now part of a static library
“mcontext-support” that is linked against your mcontext test component.
Hopefully I did not break anything in the process; the test seems to
be fine. It also contains the 'allocate_seconday_stack' implementation.

 [1] https://github.com/cnuke/genode/commits/mcontext-2021-03-23

good, thank, will use it as a part of implementation.



(I removed the available makecontext implementation because it seems
to be merely a left-over that is not required. Stubbing it as well
feels more in line. I moved the implementation into the support library
which is why it depends on libc to get access to the header files
because I omitted making the compilation unit standalone.)


Anyway, I use makecontext.c from original implementation, so, I think that I need to return it back (it fit to original ucontext.h from genode). While it requires to setup specific «size" field inside it to be suitable for makecontext. This is not supported inside proc.c and I need to patch it (it was a reason for error in the proc.c code you try to debug in last letter).


So in case of go, the libgo library should be linked against such an
support library, that - as it uses the Genode API - needs to be linked
against the Genode base library. If that is not possible, presumeably
because adding additional libraries to libgo's build-system is cumbersum,
linking your go component against the library should do the trick.

libgo.a is not a shared library, so, it just can be linked to application directly together with mcontext.a library, should not be a problem


It goes without saying that this approach only works if you are fine
with linking against the base library but I think for the time being

probably fine - except that I still have a problem in moving anon_mmap support from libc to separate library.  Problem that I  need kernel heap reference from Libc::Kernel instance which is somehow private. Seems that I can’t just call Kernel heap allocator (need for metadata store inside anon_mmap), and I should patch file_operations.cc<http://file_operations.cc> to expose reference to _heap via some function.


that is workable solution. It removes extending the libc from your
critical path and gives us time to decide how to deal with it properly.

ok, I make some patch for moving parts into world (see my git forks for genode and world), now I will change it to proposed scheme, and will test again golang test.
last time I have a problem - version without debugger works ok, with debugger fail in try to obtain some more caps (4) in extend_cpu_session for unclear reason (I have 2800 caps for gdb
and 1300 caps for test-go).
Do debugger quota for caps include caps for internal process to be debug, or they are separate?

And I still have a problem when try to run code with 2 CPU in SMP mode, some nested mutex call appears (I know that this is potentially because of internal call from unwind stack, seems that jmp_slot code invoked for strlen() function inside, compilation/linking error I think).


Please let me know if you share the assessment or if I am still missing
something obvious.

potentially it works.
my plan:
I need to fix a problem with anon_mmap initialisation (need kernel heap reference).
Need to fix a problem with 2+CPU.
Later , thanks to S. Platonov help, I want to add a network support.
For network I need samples of run script configuration to provide tcp capabilities for executable.


Regards
Josef

--
Josef Söntgen
Genode Labs

http://www.genode-labs.com/ · http://genode.org/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.genode.org/pipermail/users/attachments/20210324/20ee908b/attachment-0001.html>