Aw: Re: How to switch thread stack between threads?

[Uwe]

> Gesendet: Sonntag, 18. Juli 2021 um 00:13 Uhr
> Von: "Alexander Tormasov via users" <users at lists.genode.org>
> An: "Genode users mailing list" <users at lists.genode.org>
> Cc: "Alexander Tormasov" <a.tormasov at innopolis.ru>
> Betreff: Re: How to switch thread stack between threads?
>
> >> 
> >>>> 1. I run arbitrary function with stack associated with first thread
> >>> Every stack implies (is bound to) an os level thread.
> >>>> 2. I copy current context using getcontext and store it somewhere
> >>> You must store a pointer to an os level object that holds the os level thread (mutex is fine)
> >>> with the context to be able to later resume in the correct context.
> >> 
> >> as I see, in this moment Stack object do contains 2 data structures - native_thread and utcb handled by native OS (mean updated) as a way to store native os thread data (as well as 3-d "Thread object" reference - this is genode object).
> >> how they will be related to proposed mutex?
> > It works like the go thread communication with system calls. The model of user level threads is very near
> > the goroutines. The only difference I see so far is that user level threads are pinned to the os level
> > thread that created them. And this has to do with capabilities. The os level thread contains a registry
> > which maps numbers to capabilities. Because every number in an user level thread can potentially be a valid
> > capability (in which case it must be changed if the os level thread and therefore the registry is changed)
> > or really be a number (in which case it mustn't be changed) it is impossible to decide and therefore the
> > registry has to be locked to the user level thread. But that is in turn only possible at the os level.
> > Therefore user level threads have to be pinned to an os level thread. The only exception I could think of
> > is to shed all capabilities. Even the one neded to access the stack. Which would defeat the purpose.
> 
> Pthread model below golang runtime which is based on genode port of libc has a bit different model (at least emulated it).
> It assumes common space for created threads (which as mapped, correct me if I am wrong, to single OS thread in 1<->1 mode), and, at east, for common memory space and some subset of capabilities related to os resources, shared between threads. E.g. I assume that file, opened in one thread, I can use in another thread - therefore, related capability is common between them - it created from single session.
I think that works with TLS. At the user level. Another indirection on top of the native numbers. I think
in the POSIX Library.
> So, numerical translation of number <-> capability , as you mention above, do not broken.
> in general, I think that and of analogy between capability and pointer to memory and handles/descriptors in traditional OS is correct.
> Golang do not know about capabilities, everything hidden inside translation mechanism from kernel to user-space and back in the same way as it is in standard OS I do not know kernel address of related to fd structure.
> 
> Talking about golang model:
> they use common space for memory and descriptors and utilise as minimum os thread as possible. 
> typically number of os threads equal number of processor cores, and it growth only in situation when thread blocked by syscall. 
> in such case it stopped and new thread take from idle list or created from the scratch).
I did read your link. And if you disable the global queue it may be possible that go never executes code that
is invalid on genode although it is included. And therefore not reliable. And that could be an issue with the
philosophy of genode.
> Reason for goroutine migration between os threads is a performance, we need just to keep core busy by something.
> e.g. migration of code between different process in linux/windows require virtualisation of descriptors/handles because the same value of fd mean different opened files in different processes/domains.
> 
> Talking about optimal model in genode, I think, if we can group a set of threads in the same way as we does with pthread model (making kind of «domain of threads with shared resources» as derivative from main one), then we can migrate code inside this «domain».
This «domain» is the group of user level threads pinned to one os level thread.
> This could be analog of «process with a set of os threads sharing same objects/memory space» like in linux/windows/other OS.
> This approach can give us kind of «naturally limited» capability/allowance to move code only between involved OS threads, not to any other ones where real value of numbers could be the same.
> 
> 
> >> 
> >> golang already does this.
> >> as a part of language and runtime, it have a wrappers around sys calls and user-space preemtion points. 
> > And around {g,s}etcontext() too? If no, you would have to write it in. If yes these mutexes have to become
> > (pointer is enough)part of the context. 
> 
> yes, places where they called limited and clear, in C part of runtime
> I can keep pointers to anything, e.g. Thread object/utcb/native thread or even create mutex - inside context if need .
> the only question is how to use it exactly?
> 
> they are taken in some context (one os thread) and when code run in another thread, should I just lock this mutex from old context? I need this, not old is thread to run…
> so, as I understand,
> 1. I run os thread and inside create a mutex
> 2. inside 1 thread I rub getcontext and also store inside mutex reference
> 3. I do create os thread 2 
> 4. inside code run in os thread 2 I do read saved in 2 context and wait for mutex using reference (?)
> 5. after return from mutex wait inside 2 os thread context I call setcontext and continue execution of goroutine in os thread 2 
> 
> in 4 above I should not switch to original os thread 1 - it is busy for some other goroutines...
No, you *can* switch to the original os level thread *but* not to the original user level thread in the
os level thread or rather you first message the running original thread to do a user level switch to a
dedicated user level thread which is there to wait on the mutex.
> 
> >> 
> >> IN the last version of runtime golang developers even implement own simplified «setcontext» in asm without kernel sys calls (typically made for signals processing), - this is a core of all runtime.
> >> So, if we want to have Golang running inside genode - we need to find a way to support their model of context switches by emulation of make/set/getcontext semantics.
> > In principle they are close enough. But migrating goroutines to another os level thread has to stop.
> > Or the goroutines are interpreted (at least at preemption points) and the user level threads all(!) run the
> > same VM that does the interpreting.
> 
> imho this is too heavyweight solution, and, moreover, golang do not have language VM and interpreter...
Not really. *But only if* you think the VM inside-out. All compiled code between preemption points is
a *primitive* in this inside-out VM. And the instructions for the VM describe only the pattern for thread
switches.
> 
> >> And, it assume that any address could be used as a stack for thread (but genode allow only pre-defined and «chunked» stacks now).
> >> This significantly limit the number of potential goroutines co-existing (in real heavy load programms it could be 10-th K of them…).
> > Not reallly if you use the interpreting model. That would implement a third level. Many goroutines per user
> > level thread. Many user level threads per os level thread. Many os level threads per process.
> > For this you only have to write an API that redirects all os level thread manipulation that the go runtime
> > needs to the level of user level threads. And implement trampoline user level threads to allow migration
> > of goroutines between user level threads.
> 
> in such approach seems that we will utilise the only thread - while the main reason for existence of multiply threads is to run single thread per cpu, with switchable goroutines till any of them will be blocked.
> 
No, the idea is to create 2 os level threads per CPU core. From the start of the program. Independent of code.
Half of them run goroutines. The other half run the system calls of these goroutines. The needed task switches
should be obvious.
> 
> _______________________________________________
> Genode users mailing list
> users at lists.genode.org
> https://lists.genode.org/listinfo/users