Hi! While I am new on genode/sel4, I encounter traditional problem with app debug.
I finally make iso image on my Golang test application. I use noux_fork.run as prototype, replacing fork for my test-go: <start name="noux" caps="500"> <resource name="RAM" quantum="1G"/> <config verbose="yes" stdin="/null" stdout="/log" stderr="/log"> <fstab> <null/> <log/> <rom name="test-go" /> </fstab> <start name="test-go"> </start> </config> </start>
As usual, it is failed on first start accessing 0 address: …. virtual address layout of core: overall [0000000000002000,0000000200000000) core image [0000000002000000,0000000003805000) ipc buffer [0000000003805000,0000000003806000) boot_info [0000000003806000,0000000003808000) stack area [0000000040000000,0000000050000000) ….
[init -> noux] arg(0): "test-go" ^M[init -> noux] --- noux started --- ^MWarning: flush page table entries - mapping cache full - PD: init -> noux -> test-go ^MWarning: flush page table entries - mapping cache full - PD: init -> noux -> test-go ^Mno RM attachment (READ pf_addr=0x0 pf_ip=0xa95b1 from pager_object: pd='init -> noux -> test-go' thread='ep') ^MWarning: invalid signal-context capability ^MWarning: page-fault, pager_object: pd='init -> noux -> test-go' thread='ep' ip=0xa95b1 pf-addr=0x0
Where I can find a map of load of my application? And symbols? I found only something like test-go file, which I can read using readelf -s… How to correspond pf_ip=0xa95b1 dress with code?
good idea is ti be able to print a calls stack at some point (may be with abort) - how I can do this?
Hi Alexander,
^Mno RM attachment (READ pf_addr=0x0 pf_ip=0xa95b1 from pager_object: pd='init -> noux -> test-go' thread='ep') ^MWarning: invalid signal-context capability ^MWarning: page-fault, pager_object: pd='init -> noux -> test-go' thread='ep' ip=0xa95b1 pf-addr=0x0
Where I can find a map of load of my application? And symbols? I found only something like test-go file, which I can read using readelf -s… How to correspond pf_ip=0xa95b1 dress with code?
I'm relatively new to Genode too, and try to collect responses to questions like those; mostly for my sake, but might be useful to you too; see Section 2 "Mapping the address to source code and function names" here:
http://chiselapp.com/user/ttcoder/repository/genode-book/wiki?name=Book:Tips
Will keep an eye out for responses to your question in this thread, in case I can amend/improve that page some more.
Cedric
Thank you for information!
Dump of my file show Program Header: PHDR off 0x0000000000000040 vaddr 0x0000000000200040 paddr 0x0000000000000000 align 2**3 filesz 0x0000000000000150 memsz 0x0000000000000150 flags r-- INTERP off 0x000000000012bee2 vaddr 0x000000000112aee2 paddr 0x000000000112aee2 align 2**0 filesz 0x000000000000000a memsz 0x000000000000000a flags r-- LOAD off 0x0000000000001000 vaddr 0x0000000001000000 paddr 0x0000000001000000 align 2**12 filesz 0x000000000015c9fc memsz 0x000000000015c9fc flags r-x LOAD off 0x000000000015e000 vaddr 0x000000000115d000 paddr 0x000000000115d000 align 2**12 filesz 0x0000000000081e9c memsz 0x00000000001112f4 flags rw- DYNAMIC off 0x00000000001c0c18 vaddr 0x00000000011bfc18 paddr 0x00000000011bfc18 align 2**5 filesz 0x000000000001f284 memsz 0x000000000001f26c flags rw- EH_FRAME off 0x0000000000156030 vaddr 0x0000000001155030 paddr 0x0000000001155030 align 2**2 filesz 0x00000000000079cc memsz 0x00000000000079cc flags r-- Does it mean that real offset in code is
ip=0xa95b1- 0x1000
like 0xa85b1 Disassembly shows the following: 10a85b1: 8b 15 69 85 11 00 mov 0x118569(%rip),%edx # 11c0b20 <genode_argv+0x260>
Seems that argv is damaged… While I still try to find exactly which function fail here
23 авг. 2019 г., в 22:26, ttcoder@netcourrier.com написал(а):
Hi Alexander,
^Mno RM attachment (READ pf_addr=0x0 pf_ip=0xa95b1 from pager_object: pd='init -> noux -> test-go' thread='ep') ^MWarning: invalid signal-context capability ^MWarning: page-fault, pager_object: pd='init -> noux -> test-go' thread='ep' ip=0xa95b1 pf-addr=0x0
Where I can find a map of load of my application? And symbols? I found only something like test-go file, which I can read using readelf -s… How to correspond pf_ip=0xa95b1 dress with code?
I'm relatively new to Genode too, and try to collect responses to questions like those; mostly for my sake, but might be useful to you too; see Section 2 "Mapping the address to source code and function names" here:
http://chiselapp.com/user/ttcoder/repository/genode-book/wiki?name=Book:Tips
On 23.08.2019 23:07, Alexander Tormasov via users wrote:
Thank you for information!
Dump of my file show Program Header: PHDR off 0x0000000000000040 vaddr 0x0000000000200040 paddr 0x0000000000000000 align 2**3 filesz 0x0000000000000150 memsz 0x0000000000000150 flags r-- INTERP off 0x000000000012bee2 vaddr 0x000000000112aee2 paddr 0x000000000112aee2 align 2**0 filesz 0x000000000000000a memsz 0x000000000000000a flags r-- LOAD off 0x0000000000001000 vaddr 0x0000000001000000 paddr 0x0000000001000000 align 2**12 filesz 0x000000000015c9fc memsz 0x000000000015c9fc flags r-x LOAD off 0x000000000015e000 vaddr 0x000000000115d000 paddr 0x000000000115d000 align 2**12 filesz 0x0000000000081e9c memsz 0x00000000001112f4 flags rw- DYNAMIC off 0x00000000001c0c18 vaddr 0x00000000011bfc18 paddr 0x00000000011bfc18 align 2**5 filesz 0x000000000001f284 memsz 0x000000000001f26c flags rw- EH_FRAME off 0x0000000000156030 vaddr 0x0000000001155030 paddr 0x0000000001155030 align 2**2 filesz 0x00000000000079cc memsz 0x00000000000079cc flags r-- Does it mean that real offset in code is
ip=0xa95b1- 0x1000
like 0xa85b1 Disassembly shows the following: 10a85b1: 8b 15 69 85 11 00 mov 0x118569(%rip),%edx # 11c0b20 <genode_argv+0x260>
Seems that argv is damaged… While I still try to find exactly which function fail here
You also can try "objdump -S" on your binary, and search for the trap address in the output. This
command should show a disassembly, intermixed with a source code (if it's C/C++, not sure
about Golang). So, you can determine, which function exactly is trapped. And yes, you need to
compile with full debug symbols.
Significant problem here is that you need to know where it will be loaded in memory Disasm start at 100000 while probably basic load address is 1000 as mentioned below But I still not sure that this 1000 is for my code test-go, may be it is for noux? Or to something else? Where to find a detailed map of memory load? Log from VM seems to be not enough...
Отправлено с iPhone
24 авг. 2019 г., в 11:01, Valery V. Sedletski via users users@lists.genode.org написал(а):
On 23.08.2019 23:07, Alexander Tormasov via users wrote: Thank you for information!
Dump of my file show Program Header: PHDR off 0x0000000000000040 vaddr 0x0000000000200040 paddr 0x0000000000000000 align 2**3 filesz 0x0000000000000150 memsz 0x0000000000000150 flags r-- INTERP off 0x000000000012bee2 vaddr 0x000000000112aee2 paddr 0x000000000112aee2 align 2**0 filesz 0x000000000000000a memsz 0x000000000000000a flags r-- LOAD off 0x0000000000001000 vaddr 0x0000000001000000 paddr 0x0000000001000000 align 2**12 filesz 0x000000000015c9fc memsz 0x000000000015c9fc flags r-x LOAD off 0x000000000015e000 vaddr 0x000000000115d000 paddr 0x000000000115d000 align 2**12 filesz 0x0000000000081e9c memsz 0x00000000001112f4 flags rw- DYNAMIC off 0x00000000001c0c18 vaddr 0x00000000011bfc18 paddr 0x00000000011bfc18 align 2**5 filesz 0x000000000001f284 memsz 0x000000000001f26c flags rw- EH_FRAME off 0x0000000000156030 vaddr 0x0000000001155030 paddr 0x0000000001155030 align 2**2 filesz 0x00000000000079cc memsz 0x00000000000079cc flags r-- Does it mean that real offset in code is
ip=0xa95b1- 0x1000
like 0xa85b1 Disassembly shows the following: 10a85b1: 8b 15 69 85 11 00 mov 0x118569(%rip),%edx # 11c0b20 <genode_argv+0x260>
Seems that argv is damaged… While I still try to find exactly which function fail here
You also can try "objdump -S" on your binary, and search for the trap address in the output. This
command should show a disassembly, intermixed with a source code (if it's C/C++, not sure
about Golang). So, you can determine, which function exactly is trapped. And yes, you need to
compile with full debug symbols.
Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users
Hi Alexander,
the virtual memory address of your test application is 0x1000000, so the page fault IP address 0xa95b1 is not in the application itself. You can get the memory addresses of shared libraries printed with
<config ld_verbose="yes"...>
For example, with 'run/log' the output would look like:
[init -> test-log] 0x40000000 .. 0x4fffffff: stack area [init -> test-log] 0x30000 .. 0x197fff: ld.lib.so
IP addresses below 0x1000000 usually belong to ld.lib.so, which is loaded first, whereas additional shared libraries usually reside above the application.
Now you can find the source code location of the page fault with
$ genode-x86-objdump -dSCl debug/ld-sel4.lib.so | less
With ld.lib.so and the application image, you can look up the location with the page fault address directly, but with other shared libraries you would need to subtract the load address first.
I wonder if your test application really needs to be started with Noux though. If it doesn't need fork() or execve(), it would probably be easier not to use Noux.
Christian
Thank you for information! Anyway, it is sill not clear for me
IP addresses below 0x1000000 usually belong to ld.lib.so, which is loaded first, whereas additional shared libraries usually reside above the application.
Now you can find the source code location of the page fault with
$ genode-x86-objdump -dSCl debug/ld-sel4.lib.so | less
With ld.lib.so and the application image, you can look up the location with the page fault address directly, but with other shared libraries you would need to subtract the load address first.
I do not find any information about other libs used by my application. It is not printed in log, only init, init->timer and init->log_terminal: virtual address layout of core: overall [0000000000002000,0000000200000000) core image [0000000002000000,000000000362a000) ipc buffer [000000000362a000,000000000362b000) boot_info [000000000362b000,000000000362d000) stack area [0000000040000000,0000000050000000) ... [init -> timer] 0x40000000 .. 0x4fffffff: stack area [init -> timer] 0x30000 .. 0x197fff: ld.lib.so [init -> log_terminal] 0x40000000 .. 0x4fffffff: stack area [init -> log_terminal] 0x30000 .. 0x197fff: ld.lib.so [init] child "timer" announces service "Timer" [init] child "log_terminal" announces service "Terminal" Warning: void Genode::Rpc_cap_factory::free(Genode::Native_capability) not implemented - resources leaked: 0x10 no RM attachment (READ pf_addr=0x0 pf_ip=0xa95b1 from pager_object: pd='init -> test-go' thread='ep') Warning: invalid signal-context capability Warning: page-fault, pager_object: pd='init -> test-go' thread='ep' ip=0xa95b1 pf-addr=0x0
This is all what I have in log (by the way, same address for stack and ld.lib.so mean that this is different process and virtual address spaces?).
And, for fault address inside debug/ld-sel4.lib.so page fault IP address 0xa95b1 Objdump show hat first address inside is from 49440 - is it real address which will be in memory? /genode/build/x86_64/debug/ld-sel4.lib.so: file format elf64-x86-64
Disassembly of section .init:
0000000000049440 <_init>: _init(): /genode/repos/base/src/lib/ldso/startup/startup.cc:36http://startup.cc:36 void _init(void) __attribute__((used,section(".init")));
void _init(void) { /* call static constructors */ for(ld_hook *func = _lctors_end; func > _lctors_start + 1; (*--func)()); 49440: 48 8d 05 f9 ff ff ff lea -0x7(%rip),%rax # 49440 <_init> Does it mean that it exactly correspond (page fault IP address 0xa95b1) place below, or I should subtract something from it (not clear still for me exact list of modules in memory)? ... /genode/repos/base/src/lib/base/thread.cc:51http://thread.cc:51 Ram_allocator * const ram = env_stack_area_ram_allocator; a959f: 48 b8 88 07 00 00 00 movabs $0x788,%rax a95a6: 00 00 00 a95a9: 49 8b 44 05 00 mov 0x0(%r13,%rax,1),%rax a95ae: 48 8b 30 mov (%rax),%rsi /genode/repos/base/src/lib/base/thread.cc:52http://thread.cc:52 Ram_dataspace_capability const ds_cap = ram->alloc(ds_size); a95b1: 48 8b 06 mov (%rsi),%rax a95b4: 4c 8b 40 10 mov 0x10(%rax),%r8 a95b8: 4e 3b 04 29 cmp (%rcx,%r13,1),%r8 a95bc: 0f 85 5e 02 00 00 jne a9820 <Genode::Stack::size(unsigned long)+0x330> _ZN6Genode25Constrained_ram_allocator5allocEmNS_15Cache_attributeE(): /genode/repos/base/include/base/ram_allocator.h:89 Ram_quota_guard::Reservation ram (_ram_guard, Ram_quota{page_aligned_size}); a95c2: 48 8b 5e 10 mov 0x10(%rsi),%rbx
I wonder if your test application really needs to be started with Noux though. If it doesn't need fork() or execve(), it would probably be easier not to use Noux.
I suppose that some of emulation of generic libc implemented inside noux (like used inside libgo getpid/pthread/etc) - and this is the main reason why I try to link it. May be I am wrong ?
Alexander
Hi Alexander,
On 26.08.19 11:18, Alexander Tormasov via users wrote:
I do not find any information about other libs used by my application. It is not printed in log, only init, init->timer and init->log_terminal: virtual address layout of core: overall [0000000000002000,0000000200000000) core image [0000000002000000,000000000362a000) ipc buffer [000000000362a000,000000000362b000) boot_info [000000000362b000,000000000362d000) stack area [0000000040000000,0000000050000000) ... [init -> timer] 0x40000000 .. 0x4fffffff: stack area [init -> timer] 0x30000 .. 0x197fff: ld.lib.so [init -> log_terminal] 0x40000000 .. 0x4fffffff: stack area [init -> log_terminal] 0x30000 .. 0x197fff: ld.lib.so [init] child "timer" announces service "Timer" [init] child "log_terminal" announces service "Terminal" Warning: void Genode::Rpc_cap_factory::free(Genode::Native_capability) not implemented - resources leaked: 0x10 no RM attachment (READ pf_addr=0x0 pf_ip=0xa95b1 from pager_object: pd='init -> test-go' thread='ep') Warning: invalid signal-context capability Warning: page-fault, pager_object: pd='init -> test-go' thread='ep' ip=0xa95b1 pf-addr=0x0
This is all what I have in log (by the way, same address for stack and ld.lib.so mean that this is different process and virtual address spaces?).
core, init, timer, log_terminal, test-go, ... are different processes with their individual address spaces. So it looks like the page fault happened already before ld.lib.so could print the memory layout for '[init -> test-go]'.
And, for fault address inside debug/ld-sel4.lib.so page fault IP address 0xa95b1 Objdump show hat first address inside is from 49440 - is it real address which will be in memory? /genode/build/x86_64/debug/ld-sel4.lib.so: file format elf64-x86-64
Disassembly of section .init:
0000000000049440 <_init>: _init(): /genode/repos/base/src/lib/ldso/startup/startup.cc:36http://startup.cc:36 void _init(void) __attribute__((used,section(".init")));
void _init(void) { /* call static constructors */ for(ld_hook *func = _lctors_end; func > _lctors_start + 1; (*--func)());49440: 48 8d 05 f9 ff ff ff lea -0x7(%rip),%rax # 49440 <_init> Does it mean that it exactly correspond (page fault IP address 0xa95b1) place below, or I should subtract something from it (not clear still for me exact list of modules in memory)? ... /genode/repos/base/src/lib/base/thread.cc:51http://thread.cc:51 Ram_allocator * const ram = env_stack_area_ram_allocator; a959f: 48 b8 88 07 00 00 00 movabs $0x788,%rax a95a6: 00 00 00 a95a9: 49 8b 44 05 00 mov 0x0(%r13,%rax,1),%rax a95ae: 48 8b 30 mov (%rax),%rsi /genode/repos/base/src/lib/base/thread.cc:52http://thread.cc:52 Ram_dataspace_capability const ds_cap = ram->alloc(ds_size); a95b1: 48 8b 06 mov (%rsi),%rax a95b4: 4c 8b 40 10 mov 0x10(%rax),%r8 a95b8: 4e 3b 04 29 cmp (%rcx,%r13,1),%r8 a95bc: 0f 85 5e 02 00 00 jne a9820 <Genode::Stack::size(unsigned long)+0x330> _ZN6Genode25Constrained_ram_allocator5allocEmNS_15Cache_attributeE(): /genode/repos/base/include/base/ram_allocator.h:89 Ram_quota_guard::Reservation ram (_ram_guard, Ram_quota{page_aligned_size}); a95c2: 48 8b 5e 10 mov 0x10(%rsi),%rbx
For ld.lib.so and test-go, the address corresponds exactly. So, the page fault appears to occur at
Ram_dataspace_capability const ds_cap = ram->alloc(ds_size);
Maybe 'env_stack_area_ram_allocator' (thread.cc:51) has not been initialized?
I wonder if your test application really needs to be started with Noux though. If it doesn't need fork() or execve(), it would probably be easier not to use Noux.
I suppose that some of emulation of generic libc implemented inside noux (like used inside libgo getpid/pthread/etc) - and this is the main reason why I try to link it. May be I am wrong ?
I would try a simple "hello world" application without Noux first and hope that libgo does not need fork, execve, wait or getpid for that. pthreads should also work without Noux.
Christian
For ld.lib.so and test-go, the address corresponds exactly. So, the page fault appears to occur at
Ram_dataspace_capability const ds_cap = ram->alloc(ds_size);
Maybe 'env_stack_area_ram_allocator' (thread.cc:51) has not been initialized?
Yes, this is the core of the problem. Seems that function Genode::Core_env bool _init_stack_area() { init_stack_area(); return true; } Does no called. From core_env.h
How this could happened? I have for go own runtime function main() in normal C style: Like ... /* The main function. */
int main (int argc, char **argv) { runtime_isarchive = false; ...
And it is called from genode _main where, I assume, Genode::Core_env is initialized? genode/repos/base/src/lib/startup/_main.cc: void Component::construct(Genode::Env &env) __attribute__((weak)); void Component::construct(Genode::Env &env) { /* call real main function */ exit_status = main(genode_argc, genode_argv, genode_envp);
May be I need to call something else before my main?
I wonder if your test application really needs to be started with Noux though. If it doesn't need fork() or execve(), it would probably be easier not to use Noux.
I suppose that some of emulation of generic libc implemented inside noux (like used inside libgo getpid/pthread/etc) - and this is the main reason why I try to link it. May be I am wrong ?
I would try a simple "hello world" application without Noux first and hope that libgo does not need fork, execve, wait or getpid for that. pthreads should also work without Noux.
This is exactly what I am doing. I found that I can’t link test_go app without libc_noux.lib.so … I add it to the list, and now fight with non-initialized variables.
-
Alexander Tormasov -
Christian Prochaska -
ttcoder＠netcourrier.com -
Valery V. Sedletski