Aw: Re: How to switch thread stack between threads?

[Uwe]

> Gesendet: Donnerstag, 15. Juli 2021 um 16:49 Uhr
> Von: "Alexander Tormasov via users" <users at lists.genode.org>
> An: "Genode users mailing list" <users at lists.genode.org>
> Cc: "Alexander Tormasov" <a.tormasov at innopolis.ru>
> Betreff: Re: How to switch thread stack between threads?
>
> 	Hello, Uwe
> 
> >> *
> >> * A thread may be associated with more than one stack. Additional secondary
> >> * stacks can be associated with a thread, and used for user level scheduling.
> > Did you see this ^ ? {g,s}etcontext() count as user level scheduling!
> 
> yes, - but probaby we have a different understanding of this term.
> If we have 2 os threads, running in the user space, - I assume that I can run code in 1 thread and then switch this running code to another thread (second in this example ) and continue execution? like call makecontxt/getcontex from first OS thread and call setcontext with taken data in second OS thread?
You can run code in whatever thread you want. However, the result may not be what you want, but that
depends on your code. The model you should think about is the following. A process has many os level threads.
Every os level thread has at least one stack. Every stack equals a user level thread. Any user level thread
belongs to one and only one os level thread. This is a strict two level tree.
> I do not pretend for anything related to OS/etc, including invocation of them - I just want to be able to continue execution in already started os.
> 
> Current golang code support this model in goroutines, and the only problem that it use small number of thread local storage (TLS) variables for operations.
> I found that if I switch code in the same way as I does in, e.g. Linux, - then in genode it does not work because linux define «myself» using sys calls to obtain threadid (this operation is stack agnostic), while genode hardwire stack to backbend OS thread and do not allow simple setcontext() like switch to another OS (non-user!) thread with own stack, and, therefore, TLS variables became wrong.
That depends on the implementation of setcontext(). As I said below, there must be a mutex that should be
part of the context (and denotes the os level thread the context belongs to). And setcontext() must run
two different algorithms, depending on the mutexes that are part of the contexts. If not you will run a
stack on the wrong os level thread. With bad consequences.
> 
> >> 
> >> So, to fix it I need during switch of context to non-local thread (setcontext() or even longjump() functions) I should update these data to current running stack.
> > user level switching is only valid within the same thread. The only way to do this is to do a local
> > user level switch to a user level thread that immediately blocks the os level thread and wakes the os level
> > thread, that is blocked in the same procedure and corresponds to the target user level thread. At wakeup that
> > user level thread, which was blocked at the os level, reads the target user level thread and makes a local
> > user level switch to it.
> > The Mutex on which the user level threads blocks (at least its address) needs to be part of the context.
> 
> thank you for proposed solution. 
> I have a question related to it:  you assume to start the same function with stack instance or different one?
Is same or different. Important is os level thread matches user level thread. If same trivially true. If not
special sync needed. As described.
>  if the same - it will contain correct user state, but incorrect (old) os thread related data (as I have now);
> if not the same - I need to read the content of old stack/etc, parse it and copy to new stack on new OS thread? 
> 
> later example is incorrect: if we have a local reference stored inside stack, then we doomed… 
> 
> this is example of code which will not works:
> 
> f(int * p)
> {
> 	*p = 2;
> 	getcontext()
> 	… here we can appear in old or new threads
> 	*p = 3; // here we will point to variable in local stack - should be sure that it is not outside
> }
> 
> g()
> {
> 	int a=0;
> 	f(&a);
> 	print(a);
> }
> 
> if I call g() and switch inside f() to new thread - then local stack of g() and f() will contain reference &a to variable inside the stack. So, if I just copy stack, run new code and free old stack - it will contain a reference to old stack and "3" will be written not to a variable but to somewhere else.
> 
> In golang code typically we first save context in arbitrary os thread using getcontext(), then we will run code which just read saved context and set it for current OS thread.
In genode you can't migrate user level thread (=context) to other os thread. User level thread implies os
level thread. The equivalent you can do is using Duffs Device to record continuations in one os level thread, and
replay them in the target thread. Although I would recommend against it.
> if this is the same OS thread - everything works ok. IF this is another OS thread - we already appears in it and try to just setup RSP register in x86 to point to old stack which attributed with the old OS thread (reference to Thread object and UTCB at least in genode)…
> 
> I suppose that the only reasonable straightforward  solution here is to copy os thread data (Thread and utcb objects) from new thread where I appears to stack to be setup (taken typically from getcontext/makecontext call). May be by introducing of re_construction function (or method?) to be applied to Stack instance. it definitely contains reference to itself (eg _thread and _utcb pointers) and simple memcpy will not works… main question here is that this approach do require confidence that Thread and UTCB objects do not contains references to fields inside - and for UTCB this is definitely not low level OS agnostic...
> 
> More generic solution: could be  implementation of kind of registry for associations between OS level thread and genode Thread without stack instances, potentially kind of virtualisation of low lever OS thread id with 1 to 1 translation to genode id. 
> IMHO in general genode good to have virtualisation  like namespaces/cgroups in linux or windows. this also simplify checkpoint/restore, migration, fast restart of drivers and core and other related cross-instances operations.
> 
> Alexander
> 
> 
> 
> _______________________________________________
> Genode users mailing list
> users at lists.genode.org
> https://lists.genode.org/listinfo/users