Aw: Re: How to switch thread stack between threads?

[Uwe]

> Gesendet: Freitag, 16. Juli 2021 um 17:40 Uhr
> Von: "Alexander Tormasov via users" <users at lists.genode.org>
> An: "Genode users mailing list" <users at lists.genode.org>
> Cc: "Alexander Tormasov" <a.tormasov at innopolis.ru>
> Betreff: Re: How to switch thread stack between threads?
>
> 
> >> 
> >> probably I give a wrong picture of operations
> > I did understand you the first time around. But I have to disappoint you.
> > What you want is IMPOSSIBLE! At least in genode. Because you can't create such a context.
> 
> I do not plan to create it, I plan to update saved by taking some data from running thread. 
> I am also considering re-mapping of a part of stack area. We do know that all aux (os-relates) structures from current and saved context has the same size (in running instance) and mapping offset from start of area, even could be aligned to page bound.
For that you woud need support from the os. It could be that the UTCB contains capabilities (at least
in some constellations) which will be protected (of course). That in turn means you won't get that support.
And if in some constellations that support willbe there (because no capabilities have to be protected)
that will be seen as a mistake and rescinded as fast as possible.
> so, I can potentially save ucontext, take last stack pointer (e.g RSP register) and re-map OS-related areas to currently running thread… while technically it is similar to just copying it.
> this is a kind of hack, still not sure that it will work reliably (while it could be definitely limited to combination of utcb and native_thread structure states for some platform, they could be ported on face-by-face) 
It  will be at least destroying the mapping from numbers to capabilities.
> 
> >> 1. I run arbitrary function with stack associated with first thread
> > Every stack implies (is bound to) an os level thread.
> >> 2. I copy current context using getcontext and store it somewhere
> > You must store a pointer to an os level object that holds the os level thread (mutex is fine)
> > with the context to be able to later resume in the correct context.
> 
> as I see, in this moment Stack object do contains 2 data structures - native_thread and utcb handled by native OS (mean updated) as a way to store native os thread data (as well as 3-d "Thread object" reference - this is genode object).
> how they will be related to proposed mutex?
It works like the go thread communication with system calls. The model of user level threads is very near
the goroutines. The only difference I see so far is that user level threads are pinned to the os level
thread that created them. And this has to do with capabilities. The os level thread contains a registry
which maps numbers to capabilities. Because every number in an user level thread can potentially be a valid
capability (in which case it must be changed if the os level thread and therefore the registry is changed)
or really be a number (in which case it mustn't be changed) it is impossible to decide and therefore the
registry has to be locked to the user level thread. But that is in turn only possible at the os level.
Therefore user level threads have to be pinned to an os level thread. The only exception I could think of
is to shed all capabilities. Even the one neded to access the stack. Which would defeat the purpose.
> 
> >> 3. I stop doing function from 1, by switching to another function/stack associated with thread 1
> > That is possible.
> >> 4. after some time I create a new os thread and run some code inside it
> >> 5. then inside 2 thread I take old context from 2 above and perform setcontext from inside 2 thread to replace current function with state in the thread to the first one
> > You can not use context from 2 in another thread. Alternatively you can reconstruct the call chain from the first thread in the second thread with Duffs Device (https://en.wikipedia.org/wiki/Duff's_device#See_also)
> > And then you can construct a mirror context, which is the first context but in the second thread.
> 
> The problem that model of context is already implemented inside golang runtime (size of ~1m lines of code) using set/get/make context calls.
> I do not understand how I can emulate them using Duff device co-routines without significant modification (mostly rewriting) of this not-mine code?
> Even including go compiler: golang use stack variables, and do generate code which handle them (compiler).
> Duff device approach require different model as I know, nothing should be stored in the stack...
> Also see below
> 
> >> 
> >> So, I don’t need mutexes and wait for something - I have a time gap between suspend of function in os 1 and it continuation on thread 2.
> > The mutexes have another purpose. They regulate the os level threads when user level threads yield.
> 
> golang already does this.
> as a part of language and runtime, it have a wrappers around sys calls and user-space preemtion points. 
And around {g,s}etcontext() too? If no, you would have to write it in. If yes these mutexes have to become
(pointer is enough)part of the context. 
> It aware about existence of OS threads and plurality if goroutines, and remap goroutines in preemption points from one OS thread to another one.
> so, user lever threads never call kernel directly to yield/etc, this is done on the language/runtime level. see (1)
> in short, it periodically check possibility of preemption, and always do it during sys call (before and after), and handle blocked threads (AKA M structures) by itself having different queues for global/local instances, idle/blocked state/etc, and manipulate native OS threads (create/block/delete/etc) via pthread or similar libraries.
> E.g., if you have a blocking sys call from inside goroutine G running on M OS thread, then, before actual call, it «park» M and appropriate G in such a way that it already utilise OS object to wait (e.g. futex on linux) and correctly return/resurrect blocked M after return from syscall, again via user-level scheduler and preemption point (it just activate M and find appropriate goroutine for it to run inside context, may be the same as make syscall and block it, may be another - via user-level scheduler).
> This again assume mobility to goroutines with related stack between OS threads.
> 
> IN the last version of runtime golang developers even implement own simplified «setcontext» in asm without kernel sys calls (typically made for signals processing), - this is a core of all runtime.
> So, if we want to have Golang running inside genode - we need to find a way to support their model of context switches by emulation of make/set/getcontext semantics.
In principle they are close enough. But migrating goroutines to another os level thread has to stop.
Or the goroutines are interpreted (at least at preemption points) and the user level threads all(!) run the
same VM that does the interpreting.
> And, it assume that any address could be used as a stack for thread (but genode allow only pre-defined and «chunked» stacks now).
> This significantly limit the number of potential goroutines co-existing (in real heavy load programms it could be 10-th K of them…).
Not reallly if you use the interpreting model. That would implement a third level. Many goroutines per user
level thread. Many user level threads per os level thread. Many os level threads per process.
For this you only have to write an API that redirects all os level thread manipulation that the go runtime
needs to the level of user level threads. And implement trampoline user level threads to allow migration
of goroutines between user level threads.
> 
> Thats why I still think that may be it worth to have a bit different way to obtain from genode thread reference to the Thread object by replacing Thread::myself() function (this is just an idea, not sure is it possible to implement it only for particular application).
> 
> 
> 1 https://medium.com/swlh/different-threading-models-why-i-feel-goroutine-is-better-though-with-some-limitations-b73863ba4dae
> 
> _______________________________________________
> Genode users mailing list
> users at lists.genode.org
> https://lists.genode.org/listinfo/users