Hello Genodians
While extending my tests for the feature to build libraries with Goa, I encountered an error when building a library, that works well on x86_64, for arm_v8a.
The error is as follows:
/usr/local/genode/tool/21.05/bin/../lib/gcc/aarch64-none-elf/10.3.0/../../../../aarch64-none-elf/bin/ld: ../../lib/libc++.lib.so: The first section in the PT_DYNAMIC segment is not the .dynamic section /usr/local/genode/tool/21.05/bin/../lib/gcc/aarch64-none-elf/10.3.0/../../../../aarch64-none-elf/bin/ld: final link failed: bad value
I guess, this is due to some remaining bug in my changes. Other libraries built from the same source tree (using cmake) can be built without error.
Searching for the problem, I got [1] and [2].
The first blames binutils, the second is more specific by identifying objcompy. AFAIK objcopy is part of binutils.
Has somebody an idea what could cause this error in the Goa case? Could it be, that the wrong version of objcopy is used?
The verbose output is as follows (shortened and line breaked for better readability): /usr/local/genode/tool/21.05/bin/genode-aarch64-g++ -fPIC -fPIC -O2 -D__GENODE__ -nostdinc -D__BSD_VISIBLE -D__FreeBSD__=8 -O3 -I/data/.../api/base/2022-09-17/include/spec/arm_64 ... -Wl,-gc-sections -Wl,-z -Wl,max-page-size=0x1000 -Wl,-Ttext=0x01000000 -Wl,--eh-frame-hdr -Wl,-rpath-link=. -Wl,--whole-archive -nostartfiles -nodefaultlibs -static-libgcc -L/data/gapfruit/goa/llvm/var/abi/arm_v8a -lgcc -l:ldso_so_support.lib.a -fPIC -T /home/pirmin/.local/goa/share/goa/ld/genode_rel.ld -shared -o ../../lib/libc++.lib.so CMakeFiles/cxx_shared.dir/algorithm.cpp.obj ...
-nodefaultlibs
I have already pushed some new commits that fix small things that I discovered during extended testing.
Regard, Pirmin
[1] https://bugzilla.redhat.com/show_bug.cgi?id=679435 [2] https://sourceware.org/bugzilla/show_bug.cgi?id=12516
Hello Pirmin,
On 1/31/23 13:17, Duss Pirmin wrote:
Hello Genodians
While extending my tests for the feature to build libraries with Goa, I encountered an error when building a library, that works well on x86_64, for arm_v8a.
The error is as follows:
/usr/local/genode/tool/21.05/bin/../lib/gcc/aarch64-none-elf/10.3.0/../../../../aarch64-none-elf/bin/ld: ../../lib/libc++.lib.so: The first section in the PT_DYNAMIC segment is not the .dynamic section /usr/local/genode/tool/21.05/bin/../lib/gcc/aarch64-none-elf/10.3.0/../../../../aarch64-none-elf/bin/ld: final link failed: bad value
I guess, this is due to some remaining bug in my changes. Other libraries built from the same source tree (using cmake) can be built without error.
Searching for the problem, I got [1] and [2].
These are some very old issues (2011). Could you let me know how to reproduce this?
Thanks,
Sebastian
Hello Sebastian
On 31.01.23 19:39, Sebastian Sumpf wrote:
These are some very old issues (2011). Could you let me know how to reproduce this?
The Goa project can be found as follows: - repository: git@github.com:trimpim/wasmedge-genode.git - branch: llvm-arm_v8a_failure
For Goa to build libraries the following version is required: - repository: git@gitlab.com:/gapfruit-community/goa.git - branch: library_support
In the llvm directory execute the following commands: ``` goa build-dir goa build ```
In theory `goa build` should suffice, but for some reason this fails with some compiler errors. As it works by issuing the commands separate, I did not investigate further.
Regards, Pirmin
Hello Pirmin,
On 2/1/23 08:49, Duss Pirmin wrote:
On 31.01.23 19:39, Sebastian Sumpf wrote:
These are some very old issues (2011). Could you let me know how to reproduce this?
The Goa project can be found as follows: - repository: git@github.com:trimpim/wasmedge-genode.git - branch: llvm-arm_v8a_failure
For Goa to build libraries the following version is required: - repository: git@gitlab.com:/gapfruit-community/goa.git - branch: library_support
This, I could not find and used the one from GitHub.
In the llvm directory execute the following commands:
goa build-dir goa buildIn theory `goa build` should suffice, but for some reason this fails with some compiler errors. As it works by issuing the commands separate, I did not investigate further.
The attached patch [1] resolves the linking error for me, somehow the .got ended up first in the DYNAMIC segment. Can you please check if it works for you? However, I see a lot of undefined references to cxx.lib.a and pthread stuff afterwards. Looks like "-Lllvm/var/abi/arm_v8a" is missing in the linking command of libc++.lib.so. Additionally, "-lgcc" seems a little dangerous.
Regards,
Sebastian
[1] dynamic.patch
Regards, Pirmin
Genode users mailing list users@lists.genode.org https://lists.genode.org/listinfo/users
Hello Sebastian
Thanks for the quick patch.
On 01.02.23 13:26, Sebastian Sumpf wrote:
For Goa to build libraries the following version is required: - repository: git@gitlab.com:/gapfruit-community/goa.git - branch: library_support
This, I could not find and used the one from GitHub.
This was okay. Sorry for putting the wrong repository there.
The attached patch [1] resolves the linking error for me, somehow the .got ended up first in the DYNAMIC segment. Can you please check if it works for you? However, I see a lot of undefined references to cxx.lib.a and pthread stuff afterwards. Looks like "-Lllvm/var/abi/arm_v8a" is missing in the linking command of libc++.lib.so. Additionally, "-lgcc" seems a little dangerous.
The patch indeed fixes the problem. I get the same linker errors that you described now.
I think, these I can fix, by tweaking the Goa project.
Regards, Pirmin
Hello again,
On 2/1/23 14:18, Duss Pirmin wrote:
Hello Sebastian
Thanks for the quick patch.
On 01.02.23 13:26, Sebastian Sumpf wrote:
For Goa to build libraries the following version is required: - repository: git@gitlab.com:/gapfruit-community/goa.git - branch: library_support
This, I could not find and used the one from GitHub.
This was okay. Sorry for putting the wrong repository there.
The attached patch [1] resolves the linking error for me, somehow the .got ended up first in the DYNAMIC segment. Can you please check if it works for you? However, I see a lot of undefined references to cxx.lib.a and pthread stuff afterwards. Looks like "-Lllvm/var/abi/arm_v8a" is missing in the linking command of libc++.lib.so. Additionally, "-lgcc" seems a little dangerous.
The patch indeed fixes the problem. I get the same linker errors that you described now.
I think, these I can fix, by tweaking the Goa project.
Great. Two notes from my side:
1. libgcc is in ld.lib.so on Genode, so you should link against ld.lib.so or its api and get rid of -lgcc and -static-libgcc. The pthread things are in libc.lib.so
2. Because of the error you encountered it might very well be that the resulting shared object will not contain a .dynamic section. This would make it impossible for the dynamic linker to load the library at runtime. So, please make sure with
! genode-aarch64-objdump -ph libc++.lib.so
that the library has a PT_DYNAMIC segment as well as a .dynamic section with data in it once you got everything linked together. You can also send the output here and I will have a look.
Good luck,
Sebastian
Hello again too,
On 2/1/23 17:23, Sebastian Sumpf wrote:
- libgcc is in ld.lib.so on Genode, so you should link against
ld.lib.so or its api and get rid of -lgcc and -static-libgcc. The pthread things are in libc.lib.so
Would this be only for libraries or for binaries too? I'm asking, as I did not introduce this, but only modified code that was there before.
- Because of the error you encountered it might very well be that the
resulting shared object will not contain a .dynamic section. This would make it impossible for the dynamic linker to load the library at runtime. So, please make sure with
At least on x86_64 the WasmEdge binary was able to use the resulting libraries.
! genode-aarch64-objdump -ph libc++.lib.so
that the library has a PT_DYNAMIC segment as well as a .dynamic section with data in it once you got everything linked together. You can also send the output here and I will have a look.
I will check that and send you the output, if necessary.
Regards, Pirmin
Good morning Sebastian
On 01.02.23 17:23, Sebastian Sumpf wrote:
! genode-aarch64-objdump -ph libc++.lib.so
I have done this for the following combinations:
library x86_64 arm_v8a ------------------+-------------+----------- libunwind.lib.so | X | X libc++abi.lib.so | X | X libc++.lib.so | X |
I'm not yet able to build libc++ for arm_v8a. Still analyzing what libraries contain the missing symbols. Some seem to be from libunwind. Currently trying to find out why these symbols are missing on arm_v8a but not on x86_64.
I have attached the output for all available combinations in the list above. One big difference between the platforms stands out. On x86_64 the Dynamic section is present, on arm_v8a it is missing.
When I replace `-lgcc` with `-l:ld.lib.so` I observe the following: - on x86_64 I'm still able to build the libraries. The output of objdump is as expected a bit differnet (see libunwind-x86_64_2.txt). - on x86_64 building the WasmEdge binary fails with missing symbols (__umodti3 and __udivti3). - on arm_v8a the output of objdump is exactly the same as without the change.
Should I also apply your fixes from #4750 to the linker scripts of Goa?
Regards, Pirmin
Hello Pirmin,
On 2/2/23 09:58, Duss Pirmin wrote:
Good morning Sebastian
On 01.02.23 17:23, Sebastian Sumpf wrote:
! genode-aarch64-objdump -ph libc++.lib.so
I have done this for the following combinations:
library x86_64 arm_v8a ------------------+-------------+----------- libunwind.lib.so | X | X libc++abi.lib.so | X | X libc++.lib.so | X |
I'm not yet able to build libc++ for arm_v8a. Still analyzing what libraries contain the missing symbols. Some seem to be from libunwind. Currently trying to find out why these symbols are missing on arm_v8a but not on x86_64.
I have attached the output for all available combinations in the list above. One big difference between the platforms stands out. On x86_64 the Dynamic section is present, on arm_v8a it is missing.
When I replace `-lgcc` with `-l:ld.lib.so` I observe the following: - on x86_64 I'm still able to build the libraries. The output of objdump is as expected a bit differnet (see libunwind-x86_64_2.txt). - on x86_64 building the WasmEdge binary fails with missing symbols (__umodti3 and __udivti3). - on arm_v8a the output of objdump is exactly the same as without the change.
Okay, I confused libgcc with the unwind things (I think libgcc_eh). I built the libc package for ARMv8 and libgcc is clearly linked there:
/usr/local/genode/tool/21.05/bin/../lib/gcc/aarch64-none-elf/10.3.0/libgcc.a
So please keep it.
The unwind code for C++ exceptions is in ld.lib.so. But for shared libraries we do not link ld.lib.so, because the missing unwind symbols should end up as undefined symbols in the library. Only when the dynamic binary is linked ld.lib.so should be required to resolve these undefined symbols. The questions is why the linking of libc++.lib.so complains about these undefined symbols in the first place?
Should I also apply your fixes from #4750 to the linker scripts of Goa?
You can.
Sebastian
Hello Sebastian,
On 2/2/23 10:09, Sebastian Sumpf wrote:
The unwind code for C++ exceptions is in ld.lib.so. But for shared libraries we do not link ld.lib.so, because the missing unwind symbols should end up as undefined symbols in the library. Only when the dynamic binary is linked ld.lib.so should be required to resolve these undefined symbols. The questions is why the linking of libc++.lib.so complains about these undefined symbols in the first place?
Using Genode aarch64 objdump on the generated libraries shows, that only the `_init` symbol ends up in there.
Using the same command for x86_64 shows many more things in the libraries for this platform.
Now I wonder what is the difference in linking libraries for aarch64 compared to x86_64?
The commands to link the library for both platforms are attached.
Regards, Pirmin
Hello Sebastian and all
After a lot of trail and error, I'm finally able to build shared libraries for the arm_v8a platform. My test component (wasmedge) runs successful on my iMX8 device :-).
It turns out, that when using gcc to link libraries the argument `-shared` passed by cmake seems to behave differently between x86_64 and arm_v8a.
In the arm_v8a case this needs to be prefixed with `-Wl,`.
I have changed share/goa/lib/build/cmake.tcl to add this to the linker flags for libraries on all platforms. This is somewhat redundant, and there might be a better solution, that I currently can't see. I'll be happy to improve during the code review.
Regards, Pirmin
Hi Pirmin,
On 07.02.23 15:14, Duss Pirmin wrote:
Hello Sebastian and all
After a lot of trail and error, I'm finally able to build shared libraries for the arm_v8a platform. My test component (wasmedge) runs successful on my iMX8 device :-).
Congrats! I find it really cool that you put all this effort in library support for goa! Looking forward to get my hands on it.
Cheers, Martin
Hello Pirmin,
On 2/7/23 15:14, Duss Pirmin wrote:
Hello Sebastian and all
After a lot of trail and error, I'm finally able to build shared libraries for the arm_v8a platform. My test component (wasmedge) runs successful on my iMX8 device :-).
It turns out, that when using gcc to link libraries the argument `-shared` passed by cmake seems to behave differently between x86_64 and arm_v8a.
In the arm_v8a case this needs to be prefixed with `-Wl,`.
Awesome! It's sometimes these small details that make the difference and that can be hard to find.
Congrats,
Sebastian
-
Duss Pirmin -
Martin Stein -
Pirmin Duss
-
Sebastian Sumpf