Hi Genodians,
I've got this little project for a web service[1]. It works as a run-script and even boots as disk image on my intel box. *)
It's fast. It serves a file at 40MBytes/second and only 5 ms latency between connecting to it and the first bytes on a Gbps LAN. Although that file was served from memory, half the speed of the LAN without any tuning is promising!
But I get stuck with making it into a depot-package. I lack the clarity of what the next steps are.
My subgoals: 1. make it a depot-package; 2. make it configurable by user; 3. replace ragel parser[2] with hammer library[3] as a dependency.
How could I proceed with these steps? Is there an example run-script to package conversion I could follow?
Cheers, Guido.
1: github.com/gwitmond/feather 2: https://www.colm.net/open-source/ragel/ 3a: https://github.com/UpstandingHackers/hammer 3b: http://www.langsec.org/
*) Sculpt runs fine on AMD's Ryzen too, except for SATA, virtualisation.
On 06/25/2018 10:44 PM, Guido Witmond wrote:
But I get stuck with making it into a depot-package. I lack the clarity of what the next steps are.
I forgot to mention that I read the book and the articles on the site and I think I know the abstract of it but I lack the concrete steps.
What I think I know know: - I have to make the directory into a SRC-package - From there I do a build to get BIN package - That I copy to a webroot and point my testbox to it.
Hope this clarifies where I'm stuck.
Cheers, Guido.
Hi Guido,
I forgot to mention that I read the book and the articles on the site and I think I know the abstract of it but I lack the concrete steps.
What I think I know know:
- I have to make the directory into a SRC-package
our regular work flow is to extract src packages from the Genode repository. If you want to use this work flow, you have to create a so-called recipe. All recipes are stored under <genode>/<repo>/recipes/. You can find more information about the use of recipes in Section 5.5.4. "Automated extraction of archives from the source tree" of the Genode Foundations book [1]. Should the documentation be too scarce, I'd like to know what additional information would help you to connect the dots.
[1] https://genode.org/documentation/genode-foundations-18-05.pdf
That said, you can assemble your src archive by hand just as well. It is basically a Genode repository supplemented with a tiny bit of meta data (the 'used_apis' and optional 'api' file in the top level directory). Please have a look at an existing archive like [2] as reference.
[2] <genode>/depot/genodelabs/src/init/2018-06-26
- From there I do a build to get BIN package
Yes. This is done with the tool/depot/create tool. A typical invocation of the tool looks like this (please refer to the book for the description of the arguments):
cd genode ./tool/depot/create guido/bin/x86_64/init \ UPDATE_VERSIONS=1 FORCE=1 REBUILD=
- That I copy to a webroot and point my testbox to it.
You won't copy the content of the binary archive directly to the web server. First, there is a so-called 'publish' step, which creates tar archives for the to-be-published archive and its dependencies, and signs those tar archives with your private OpenPGP key. This step populates the <genode>/public/ directory. The latter can then be rsync'd to your web server.
I think that all this information should be covered in the book's Section 5.5. If I was telling anything new that is not covered there, I'd appreciate a hint from you. ;-)
Cheers Norman
On 06/27/2018 03:36 PM, Norman Feske wrote:
Hi Norman,
Here's my experiences with packaging my app.
1. As I'm in the role of system integrator (not genode developer), I originally placed my feather-package in genode/projects/feather and added that path to REPOSITORIES in genode/build/x86_64/etc/build.conf. While that works for the make run/feather mechanism, it does not work for the packaging. That requires the source code in genode/repos/feather. It's in the manual but was my first stumbling block.
2. I needed to add the content.mk as well as the hash file to my recipes directory to make the depot/create work. Without either I got the error: "incomplete or missing recipe (guido/src/feather). It's probably in the book but the error message is quite vague. A message "missing file<genodedir>/repo/<repo>/recipes/src/<project>/hash" would be more helpful. I'll add it after I get the hang of the packaging system (to prevent adding stuff that's not needed).
3. The part where I found the documentation lacking was the description of the content.mk file. The documentation did not lead me to a working version. Ultimately, I copied the one from gems/.../depot_deploy and adapted it.
After that depot/create worked. It gave me a depot/guido/src/feather directory with the same structure as my repo/feather.
Question: do you foresee that third party developers place their git source in repos/<developer> like I did or in depot/<developer> and bypassing that recipes/hash/content.mk part.
4. I did a depot/build guido/bin/x86_64/feather/2018-06-30 KEEP_BUILD_DIR=1 That gave me this error: Makefile: 131: *** "g++ version 6.3.0 is required, we got /usr/bin/g++".
I added that last part to builddir/build.mk to diagnose what compiler ran. It seems that this code in depot/guido/bin/...<version>.build/etc/tools.conf expects SPECS to be just x86_64 while specs.conf contains: SPECS += genode x86_64 ifeq ($(filter-out $(SPECS),x86_64),) CROSS_DEV_PREFIX ?= /usr/local/genode-gcc/bin/genode-x86- endif It left CROSS_DEV_PREFIX unset, falling back to the compiler in $PATH.
Is this a bug or did I miss something?
5. Setting export CROSS_DEV_PREFIX=/usr/local/.../genode-x86- by hand made the build get further. I got these errors. "Library-description file libc.mk is missing." (And so for libc_lwip.mk, libc_lwip_nic_dhcp.mk and posix.mk). Here the error message is correct but lacks the hint to add the dependencies to used_apis in the previous step.
6. After setting the dependencies in used_apis, and running create again, I get this error at build: checking library dependencies... Skip target feather because it requires libc_support_for_your_target_platform created guido/bin/x84_64/feather/2018-07-01-b
The libc_support... 'target' suggests that it can't determine the platform. Is that the SPECS += genode x86_64 bug again?
Yet it did create a that directory it specfied but it's empty and further invocations of build happily tell me that there is everything is up to date. I expect it would not make that directory or retry builing until it has more deliverables.
Hope my feedback helps you to improve Genode.
Cheers, Guido.
On 07/01/2018 11:15 PM, Guido Witmond wrote:
It seems that this code in depot/guido/bin/...<version>.build/etc/tools.conf expects SPECS to be just x86_64 while specs.conf contains: SPECS += genode x86_64 ifeq ($(filter-out $(SPECS),x86_64),) CROSS_DEV_PREFIX ?= /usr/local/genode-gcc/bin/genode-x86- endif It left CROSS_DEV_PREFIX unset, falling back to the compiler in $PATH.
I've grepped the code and I found the idiom: 'ifeq(filter-out $SPECS, X),<empty>)' a few more times. If I'm not mistaken, it is true if $SPECS contains only X and false if SPECS contains two or more items even if including an X.
But it goes against the manual of make [1] where the first param is the pattern X and the second is the string 'text' to look up the string X.
I wonder why the code is not: 'ifeq (filter (X,$(SPECS)),X)' as that would signal a that an X is found in SPECS, irrespective of the rest in SPECS.
Cheers, Guido.
1: https://www.gnu.org/software/make/manual/html_node/Text-Functions.html#Text-...
Hello Guido,
$(filter-out ...)
...
But it goes against the manual of make [1] where the first param is the pattern X and the second is the string 'text' to look up the string X.
the relevant part of the documentation is:
"Returns all whitespace-separated words in text that do not match any of the pattern words, removing the words that do match one or more."
Note that both arguments may be lists of words.
The use of 'filter-out' has the advantage that the pattern doesn't need to be repeated, which would otherwise produce possible inconsistencies during later maintenance steps (e.g., when changing the pattern).
I've grepped the code and I found the idiom: 'ifeq (filter-out $SPECS, X),)'
Here, 'X' appears once.
I wonder why the code is not: 'ifeq (filter (X,$(SPECS)),X)'
Here, 'X' appears twice.
Cheers Norman
Hi Guido,
- As I'm in the role of system integrator (not genode developer), I
originally placed my feather-package in genode/projects/feather and added that path to REPOSITORIES in genode/build/x86_64/etc/build.conf. While that works for the make run/feather mechanism, it does not work for the packaging. That requires the source code in genode/repos/feather. It's in the manual but was my first stumbling block.
you may keep your code anywhere you like by creating a symlink '<genode>/repos/feather' pointing to the location of your code.
- The part where I found the documentation lacking was the description
of the content.mk file. The documentation did not lead me to a working version. Ultimately, I copied the one from gems/.../depot_deploy and adapted it.
It's certainly the best to take an existing recipe as reference. As far as documenting the process of creating a 'content.mk' file goes, it is not easy to find a useful level of detail.
After that depot/create worked. It gave me a depot/guido/src/feather directory with the same structure as my repo/feather.
Question: do you foresee that third party developers place their git source in repos/<developer> like I did or in depot/<developer> and bypassing that recipes/hash/content.mk part.
Similar to hosting a symbolic link '<genode>/repos/feather', you may in principle host a symbolic link in '<genode>/depot/guido/src/feather', pointing to your source tree. I admittedly haven't tried this but you may give it a try. Note, however, that the publish step would archive the entire content of the directory. So it might be preferable to have a tool (makefile) in place to assemble the content.
- I did a depot/build guido/bin/x86_64/feather/2018-06-30 KEEP_BUILD_DIR=1
That gave me this error: Makefile: 131: *** "g++ version 6.3.0 is required, we got /usr/bin/g++".
This is strange. Have you successfully built one of Genode's binary archives, e.g., guido/bin/x86_64/report_rom? If this works, what is the difference between your src archive and report_rom's? By any chance, do you happen to have an 'etc/' or 'mk/' directory in your archive? This shouldn't be.
I added that last part to builddir/build.mk to diagnose what compiler ran. It seems that this code in depot/guido/biI think, given that there are plentiful examples to learn from, n/...<version>.build/etc/tools.conf expects SPECS to be just x86_64 while specs.conf contains: SPECS += genode x86_64 ifeq ($(filter-out $(SPECS),x86_64),) CROSS_DEV_PREFIX ?= /usr/local/genode-gcc/bin/genode-x86- endif It left CROSS_DEV_PREFIX unset, falling back to the compiler in $PATH.
Is this a bug or did I miss something?
This is a follow-up problem of the one above. I recommend experimenting with Genode's existing packages first, and then comparing your's with those.
Cheers Norman
On 07/03/2018 02:55 PM, Norman Feske wrote:
As far as documenting the process of creating a 'content.mk' file goes, it is not easy to find a useful level of detail.
I know. Writing good documents is hard. I hope my experiences help you find a base level of detail. Perhaps Genode Labs could curate a set of example files, rated from simple to complex and referred to in the documentation. That would certainly have helped me.
Question: do you foresee that third party developers place their git source in repos/<developer> like I did or in depot/<developer> and bypassing that recipes/hash/content.mk part.
Similar to hosting a symbolic link '<genode>/repos/feather', you may in principle host a symbolic link in '<genode>/depot/guido/src/feather', pointing to your source tree. I admittedly haven't tried this but you may give it a try. Note, however, that the publish step would archive the entire content of the directory. So it might be preferable to have a tool (makefile) in place to assemble the content.
Although helpful, that was not the answer I was expecting. I wanted your vision on whether third parties should create their repositories like genode/repos/<mystuff> complete with a recipe to transform it into a depot/<me>/... structure or to skip the genode/repos part entirely and use a depot/<me>/../<package> structure as base of development? The latter would save a lot of documenting :-)
- I did a depot/build guido/bin/x86_64/feather/2018-06-30 KEEP_BUILD_DIR=1
That gave me this error: Makefile: 131: *** "g++ version 6.3.0 is required, we got /usr/bin/g++".
This is strange. Have you successfully built one of Genode's binary archives, e.g., guido/bin/x86_64/report_rom? If this works, what is the difference between your src archive and report_rom's? By any chance, do you happen to have an 'etc/' or 'mk/' directory in your archive? This shouldn't be.
I found the source of this problem. I fat-fingered the x86_64 as x84_64. That gave the error. When I transcribed the command line from my VM (I could not copy-paste), I 'corrected' it in my previous mail to you. Sorry about that confusion.
Cheers, Guido.
-
Guido Witmond -
Norman Feske