Gesendet: Sonntag, 04. April 2021 um 15:18 Uhr Von: "Alexander Tormasov via users" users@lists.genode.org An: "Genode users mailing list" users@lists.genode.org Cc: "Alexander Tormasov" a.tormasov@innopolis.ru Betreff: how to open elf file image from inside genode? libbacktrace question

During port of libbacktrace code from gcc compiler I found that I need to be able to read a.out or similar file in runtime. This library try to read original executable file and parse it for functions addresses/etc, as a part of garbage collection for golang (it parse call stack in some specific way using gcc-specific calls).

I have to port to genode the following code from src/lib/gcc/libbacktrace/fileline.c:

for (pass = 0; pass < 5; ++pass) { int does_not_exist;

  switch (pass)

{ case 0: filename = state->filename; break; case 1: filename = getexecname (); break; case 2: filename = "/proc/self/exe"; break; case 3: filename = "/proc/curproc/file"; break; case 4: snprintf (buf, sizeof (buf), "/proc/%ld/object/a.out", (long) getpid ()); filename = buf; break; default: abort (); }

best idea is to implement library call getexecname(), e.g. as a part of libc.

Your parent needs a ROMFile capability to run your program. Ask it for that capability to read your own file. You can call that RPC getexecname or whatever strikes your fancy. Of course you would have to extend the official interface to the parent else your program could be started from a component that doesn't implement that function. Or you can write a wrapper that guarantees to implement that function and start your program exclusively with that wrapper. You can call your wrapper "go run" or whatever strikes your fancy.

Anyway, this is complex question, because we need to be able to open it and read a content later. How to configure this in run file?

PS for better understanding why I need this - call stack where it is called, from gdb:

#0 fileline_initialize (state=0x29000, error_callback=0x1198543 <error_callback>, data=0x405fe770) at /var/services/homes/admin/gen/21.02/contrib/libbacktrace-cabcd83bc903137f607eda3a65eb09feaa5f507e/src/lib/gcc/libbacktrace/fileline.c:102 #1 0x000000000119a9a5 in backtrace_pcinfo (state=0x29000, pc=0x1198610, callback=0x119808e <callback>, error_callback=0x1198543 <error_callback>, data=0x405fe770) at /var/services/homes/admin/gen/21.02/contrib/libbacktrace-cabcd83bc903137f607eda3a65eb09feaa5f507e/src/lib/gcc/libbacktrace/fileline.c:177 #2 0x000000000119accc in unwind (context=0x405fe420, vdata=0x405fe700) at /var/services/homes/admin/gen/21.02/contrib/libbacktrace-cabcd83bc903137f607eda3a65eb09feaa5f507e/src/lib/gcc/libbacktrace/backtrace.c:91 #3 0x00000000011a87e9 in _Unwind_Backtrace (trace=0x119abfc <unwind>, trace_argument=0x405fe700) at /genodelabs-genode-2491eee/contrib/gcc-3cade18cf9defeefa714aa91de3b157fbad4aa18/src/noux-pkg/gcc/libgcc/unwind.inc:307 #4 0x000000000119ada2 in backtrace_full (state=0x29000, skip=0x0, callback=0x119808e <callback>, error_callback=0x1198543 <error_callback>, data=0x405fe770) at /var/services/homes/admin/gen/21.02/contrib/libbacktrace-cabcd83bc903137f607eda3a65eb09feaa5f507e/src/lib/gcc/libbacktrace/backtrace.c:127 #5 0x0000000001198611 in runtime_callers (skip=0x2, locbuf=0xc42000e8d8, m=0x20, keep_thunks=0x0) at /var/services/homes/admin/gen/21.02/contrib/libgo-281260d9bdc27fefb62c00310025c54e1e629a2e/src/lib/gcc/libgo/runtime/go-callers.c:207 #6 0x00000000010fdf28 in runtime.callers (skip=0x1, locbuf=...) at /var/services/homes/admin/gen/21.02/contrib/libgo-281260d9bdc27fefb62c00310025c54e1e629a2e/src/lib/gcc/libgo/go/runtime/traceback_gccgo.go:56 #7 0x00000000010d8f7d in runtime.mcommoninit (mp=0xc42000e800) at /var/services/homes/admin/gen/21.02/contrib/libgo-281260d9bdc27fefb62c00310025c54e1e629a2e/src/lib/gcc/libgo/go/runtime/proc.go:546 #8 0x00000000010db6c5 in runtime.allocm (_p_=0xc420008000, fn=0x121e298 <runtime.mspinning..f>, allocatestack=0x0) at /var/services/homes/admin/gen/21.02/contrib/libgo-281260d9bdc27fefb62c00310025c54e1e629a2e/src/lib/gcc/libgo/go/runtime/proc.go:1466 #9 0x00000000010dbdb2 in runtime.newm (fn=0x121e298 <runtime.mspinning..f>, _p_=0xc420008000) at /var/services/homes/admin/gen/21.02/contrib/libgo-281260d9bdc27fefb62c00310025c54e1e629a2e/src/lib/gcc/libgo/go/runtime/proc.go:1769 #10 0x00000000010dc360 in runtime.startm (_p_=0xc420008000, spinning=0x1) at /var/services/homes/admin/gen/21.02/contrib/libgo-281260d9bdc27fefb62c00310025c54e1e629a2e/src/lib/gcc/libgo/go/runtime/proc.go:1925 #11 0x00000000010dc702 in runtime.wakep () at /var/services/homes/admin/gen/21.02/contrib/libgo-281260d9bdc27fefb62c00310025c54e1e629a2e/src/lib/gcc/libgo/go/runtime/proc.go:2006 #12 0x00000000010dfabf in __go_go (fn=0x10d72f2, arg=0x0) at /var/services/homes/admin/gen/21.02/contrib/libgo-281260d9bdc27fefb62c00310025c54e1e629a2e/src/lib/gcc/libgo/go/runtime/proc.go:3144

---

Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users

Gesendet: Montag, 05. April 2021 um 00:59 Uhr Von: "Alexander Tormasov via users" users@lists.genode.org An: "Genode users mailing list" users@lists.genode.org Cc: "Alexander Tormasov" a.tormasov@innopolis.ru Betreff: Re: how to open elf file image from inside genode? libbacktrace question

Thanks for suggestion, Uwe

...

...

Your parent needs a ROMFile capability to run your program. Ask it for that capability to read your own file. You can call that RPC getexecname

Oops, that is possibly the source of a misunderstanding. I should have proposed getexechandle.

...

or whatever strikes your fancy.

the question is what path for open() call I should use for ROMfile?

You don't open() the session, that does the parent, because it needs to do it anyway. You only ask for the already opened file handle. I don't have the exact startup sequence In memory, but it could be that the handle is already communicated as part of the startup sequence to build the ds session. Then you only need to extend the startup sequence with a copy of that session.

Sincerely, Alexander

---

Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users

Returning to my question, could I use posix open(«/test-go») to open my binary from which I run executable, if I have the following code in run file:

<start name="test-go" caps="250"> <resource name="RAM" quantum="20M"/> <config verbose="yes" ld_verbose="yes" > <vfs> <dir name="dev"> <log/> </dir> </vfs> <libc stdout="/dev/log" stderr="/dev/log"/> </config> </start>

or I need to modify it and add another «double» option, something like

<start name="test-go" caps="250"> <resource name="RAM" quantum="20M"/> <config verbose="yes" ld_verbose="yes" > <vfs> <dir name="dev"> <log/> </dir> <rom name="test-go"/> </vfs> <libc stdout="/dev/log" stderr="/dev/log"/> </config> </start>

5 апр. 2021 г., в 14:36, ttcoder@netcourrier.com написал(а):

...

...

Your parent needs a ROMFile capability to run your program. Ask it for that capability to read your own file. You can call that RPC getexecname or whatever strikes your fancy.

the question is what path for open() call I should use for ROMfile?

Not sure if that's relevant to the discussion as I'm jumping in mid-stream, but in my Genode run scenarios I use the following, in order to access data files (not exe files) :

<start name="my_application"> <config> <vfs> <rom name="background.jpeg"/> ..etc

Sincerely, Alexander

...

or I need to modify it and add another «double» option, something like

You need this configuration. But why do you want to use a way that is dependent upon config, when you need it for a language detail. That is why make something compiled in depend on something configured. And why has it to be POSIX open ()? Alternatively you can store that capability on startup in a static var in open () where it is returned from upon encountering a special parameter combination for instance an empty string as filename.

of course, I would prefer to be able to use posix open (or even just obtain a dup() of handle for my own executable file) to use it for read()/write() operations later. This is not mine code (libbacktrace is a part of original gcc), and I just want to be able to run it maximally smoothly inside genode. In this moment it try to call posix open() for some names like /proc/self/exe or similar, and later returned handle used deeply inside code. I will be happy to find ANY smooth way to have a handle for myself…

and, I am not so sure that loaded into ram current executable is suitable for such operations like read of elf header… probably I need to open it explicitly, but to do so I need to find it inside genode file system mounted name space (/vfs?)

Gesendet: Montag, 05. April 2021 um 22:05 Uhr Von: "Alexander Tormasov via users" users@lists.genode.org An: "Genode users mailing list" users@lists.genode.org Cc: "Alexander Tormasov" a.tormasov@innopolis.ru Betreff: Re: how to open elf file image from inside genode? libbacktrace question

First let me tell sorry for the spellchecker ran amok in my last post. It smuggled the ist into two instances of static to create statistic. Please ignore the contraband.

Now that I was able to look into the library source I can give you a recipe. 1) Define a number that is used as descriptor next 2) On startup dup2() the capability (possibly after casting to a POSIX Descriptor) to the defined number 3) In fileline.c introduce a last option of using the empty filename "" 4) In posix.c in function backtrace_open () check for empty filename 5) If so, blindly dup() the defined number. If successful return the created (By dup()) handle. If error you can either declare failure or try first to really open () the (empty) filename.

...

...

or I need to modify it and add another «double» option, something like

You need this configuration. But why do you want to use a way that is dependent upon config, when you need it for a language detail. That is why make something compiled in depend on something configured. And why has it to be POSIX open ()? Alternatively you can store that capability on startup in a static var in open () where it is returned from upon encountering a special parameter combination for instance an empty string as filename.

of course, I would prefer to be able to use posix open (or even just obtain a dup() of handle for my own executable file) to use it for read()/write() operations later. This is not mine code (libbacktrace is a part of original gcc), and I just want to be able to run it maximally smoothly inside genode. In this moment it try to call posix open() for some names like /proc/self/exe or similar, and later returned handle used deeply inside code. I will be happy to find ANY smooth way to have a handle for myself…

and, I am not so sure that loaded into ram current executable is suitable for such operations like read of elf header… probably I need to open it explicitly, but to do so I need to find it inside genode file system mounted name space (/vfs?)

---

Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users

Gesendet: Donnerstag, 08. April 2021 um 15:23 Uhr Von: "Josef Söntgen" josef.soentgen@genode-labs.com An: "Alexander Tormasov via users" users@lists.genode.org Betreff: Re: how to open elf file image from inside genode? libbacktrace question

Hallo Alexander,

...

Returning to my question, could I use posix open(«/test-go») to open my binary from which I run executable, if I have the following code in run file: […]

Yes, using 'open()' is the way to go. As libbacktrace hardcodes a few

Is that really a good idea? Depending on vfs for something as bas(e)ic as stack tracing seems to me as overly broad. Because the needed capability is already a (private) member of the environment as _binary- >cap(). I only need to find a way to make it accessible by POSIX later. Or make it directly accessible by the stack tracing part.

options you could piggy-back on one of them. I picked the '/proc/self/exe' option:

! <start name="test-go" caps="250"> ! <resource name="RAM" quantum="12M"/> ! <config verbose="yes" ld_verbose="yes"> ! <vfs> ! <dir name="dev"> <log/> </dir> ! ! <dir name="proc"> ! <dir name="self"> ! <rom name="exe" label="test-go"/> ! </dir> ! </dir> ! </vfs> ! <libc stdout="/dev/log" stderr="/dev/log"/> ! </config> ! </start>

With this VFS configuration in place libbacktrace opens the binary successfully. However, you will be greeted with the by now infamous deadlock ahead error message when using '-smp 2':

! [init -> test-go] Error: deadlock ahead, mutex=0x164040, return ip=0xf3383

The mutex in question is the 'Linker::mutex()' object. Again this is due to the fact that as side-effect of 'dl_iterate_phdr' a jump-slot relocation for 'mknod' takes place (*). Why that is I have not checked.

Now we are not entirely sure if taking the mutex in 'dl_iterate_phdr' is strictly necessary - that is something we have to investigate. Removing the mutex allows the 'go_app' to run through. So you are free to do the same. But to be clear, we do not know if this leads to other regressions. We are going to look into that.

(*) You can instrument that by applying the patch I attached that makes the linker verbose and shows you the object and index for the relocation:

! […]
! [init -> test-go] LD: SLOT libc.lib.so 0x20c
! [init -> test-go] LD: SLOT binary 0xe
! [init -> test-go] Error: deadlock ahead, mutex=0x164040, return ip=0xf3383

With this information you can check to which symbol the index
belongs to:

$ readelf --dyn-syms test-go|grep '\<14:'
    14: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND mknod

Regards Josef

-- Josef Söntgen Genode Labs

http://www.genode-labs.com/ · http://genode.org/ _______________________________________________ Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users

Hello Alexander,

On Thu, Apr 08, 2021 at 18:18:01 CEST, Alexander Tormasov via users wrote:

page fault, pd='init' thread='ep' cpu=0 ip=0xb1d73 address=0x401eeffc stack pointer=0x401eeff0 qualifiers=0x6 irUWp reason=1

The fault address is in the stack area and looks suspicious: 0x401eeff0 is 4100 bytes below 64K of stack. Could it be your stack size is 64K which does not suffice your use case?

Greets

Alexander,

On Fri, Apr 09, 2021 at 13:07:37 CEST, Alexander Tormasov via users wrote:

...

On Thu, Apr 08, 2021 at 18:18:01 CEST, Alexander Tormasov via users wrote:

...

page fault, pd='init' thread='ep' cpu=0 ip=0xb1d73 address=0x401eeffc stack pointer=0x401eeff0 qualifiers=0x6 irUWp reason=1

The fault address is in the stack area and looks suspicious: 0x401eeff0 is 4100 bytes below 64K of stack. Could it be your stack size is 64K which does not suffice your use case?

it is really hard to say something about, but in the past 64k was definitely enough. I suspect nested "panic in panic" during processing of c++ exceptions…

while it is hard to say something because of strange debugger work (request 4 more caps from init and hang), I try to find any methods to understand/debug situation…

Sorry but I completely ignored the fact that the failing component is the root init instance. So, it might help to just remove ld_verbose="yes" from its config while debugging with Josef's instrumentation. The change seems to trigger unexpected stack usage in init which you don't seem to debug currently.

Regards

Hello Alexander,

[…] So, this is inside dl_iterate_phdr under mutex, as usual, and double mutex call related to search of errno. I suspect other potential code path where we can hit «seach for address of some functions from shared libraries»...

Finally I will remove, as suggested , mutex guard from dl_iterate_phdr completely - while not so sure that this will not lead to error.

Please replace your mutex-remove commit with the following commit [1], which replaces the linker mutex with the shared-object mutex. This behaviour is in line with other dynamic-linkers.

[1] https://github.com/genodelabs/genode/commit/c45d964bc23f4f

(I moved the quotation down a bit to get concise block of text.)

In short, it try to open ld.lib.so (from inside genode), open() fail, and it try to obtain libc errno (via jmp_slot/etc from shared library). So, this is inside dl_iterate_phdr under mutex, as usual, and double mutex call related to search of errno. I suspect other potential code path where we can hit «seach for address of some functions from shared libraries»... Also I add the following to the run file to allow read of ld.lib.so from inside (not sure that this is correct - may be need something else? like direct reference to rom session?) <dir name=«ld.lib.so»> </dir>

In case libbacktrace wants to access other shared-objects directly, besides the binary itself, I would deal with it in the same way, e.g:

! […] ! <vfs> ! <dir name="dev"> <log/> </dir> ! ! <rom name="ld.lib.so"/> ! <rom name="xyz.lib.so"/> ! ! <dir name="proc"> ! <dir name="self"> ! <rom name="exe" label="test-go"/> ! </dir> ! </dir> ! </vfs> ! […]

This config operates under the assumption that libbacktrace wants to access the shared-objects via 'open("/ld.lib.so")' - I have not checked it that's true.

Seems that after that operation my SMP small test with even 4 CPU works somehow (need more extensive tests to be run for golang and goroutunes).

Question: how to verify that removed mutex call does not break something important?

Please just use the commit I referenced above. As far as our automated tests go, it did not introduce any regressions (of course that is no guarantee that all potential corner-cases are covered).

Regards Josef