Dear Genodeians,
I started working with the depot tools in order to create and deploy my own archives. Unfortunately, I'm a bit puzzled about the hashing/versioning. Let me clarify my confusion on a minimal example:
$> git clone https://github.com/genodelabs/genode.git $> cd genode $> ./tool/depot/extract local/api/base Error: /home/johanness/repos/genode/repos/base/recipes/api/base/hash is out of date
This error indicates that the archived source code has changed, which should be reflected by incrementing the archive version and updating the hash of the recipe. You may update the recipe hash via the following command:
echo 2018-06-25 1cece72259a129c0906c20d32da078ee0638e545 > /home/johanness/repos/genode/repos/base/recipes/api/base/hash [...]
The latest commit on the current master branch updated the recipe hashes, which is why I assumed the extract tool should calculate the same hashes but it doesn't.
Next, I executed the same command with `VERBOSE=` to get the command that calculates the hash. I tried to run the command manually as follows:
$> cd depot/local/api/base/incomplete $> find . -type f | sort | xargs -d '\n' cat | sha1sum | sed "s/ .*// 731162f0b76bebabc0741366600973b9bbe23651
As you can see, I get a different hash as the extract tool suggested before. Both hashes are different from the recipe hash, by the way. I might be lacking some sleep today and am probably misconceiving the process. Anyway, I'd appreciate if anyone could help and clarify this point.
PS: I came across this issue when I tried to create my first recipe. As this uses the base API, it complained that the hash is out of date. This seems counterintuitive as the base API should be pretty stable and deploying my own archives should not require me to provide my own base API as well. Please correct me if I'm wrong.
Cheers Johannes
Hi Johannes,
$> git clone https://github.com/genodelabs/genode.git $> cd genode $> ./tool/depot/extract local/api/base
for reference, these commands work on my machine without error. It's puzzling.
$> cd depot/local/api/base/incomplete $> find . -type f | sort | xargs -d '\n' cat | sha1sum | sed "s/ .*// 731162f0b76bebabc0741366600973b9bbe23651
As you can see, I get a different hash as the extract tool suggested before. Both hashes are different from the recipe hash, by the way. I might be lacking some sleep today and am probably misconceiving the process. Anyway, I'd appreciate if anyone could help and clarify this point.
I think the third hash can be explained: The 'sort' order depends on your locale settings. Within the depot tools, we use
export LC_COLLATE=C
You certainly use a different setting on your interactive shell. May you try your command with this setting?
Still, the fact that extract tool creates a hash different from the one in repos/base/recipes/api/base/hash is strange. Could you compare the content of the archive created by you with the content downloaded from [1]?
[1] https://depot.genode.org/genodelabs/api/base/2018-06-12.tar.xz
Cheers Norman
Hi Norman,
thanks for your reply. I was a bit inexact with my commands. I actually performed a `git checkout genodelabs/master` rather than a git clone.
As it turned out, I had an unversioned file in base/include which was responsible for the different hash as the base/api archive copies all files from base/include/.
I'm afraid having unversioned files in the working tree that pollute the hash calculation could be a common pitfall (such as *.orig files from a merge conflict or similar). What do you think about using `git ls-files` when copying the content of archives?
Cheers Johannes
On Wed, 27 Jun 2018 11:28:19 +0200 Norman Feske norman.feske@genode-labs.com wrote:
Hi Johannes,
$> git clone https://github.com/genodelabs/genode.git $> cd genode $> ./tool/depot/extract local/api/base
for reference, these commands work on my machine without error. It's puzzling.
$> cd depot/local/api/base/incomplete $> find . -type f | sort | xargs -d '\n' cat | sha1sum | sed "s/ .*// 731162f0b76bebabc0741366600973b9bbe23651
As you can see, I get a different hash as the extract tool suggested before. Both hashes are different from the recipe hash, by the way. I might be lacking some sleep today and am probably misconceiving the process. Anyway, I'd appreciate if anyone could help and clarify this point.
I think the third hash can be explained: The 'sort' order depends on your locale settings. Within the depot tools, we use
export LC_COLLATE=C
You certainly use a different setting on your interactive shell. May you try your command with this setting?
Still, the fact that extract tool creates a hash different from the one in repos/base/recipes/api/base/hash is strange. Could you compare the content of the archive created by you with the content downloaded from [1]?
[1] https://depot.genode.org/genodelabs/api/base/2018-06-12.tar.xz
Cheers Norman
-
Johannes Schlatow -
Norman Feske