depot package version requirements are too strict

[Norman Feske]

Hello,

On 07.06.2018 03:19, Nobody III wrote:
> Currently, if I modify a library or server to fix a bug or add a feature, I
> can't get other packages to use the updated version without modifying them
> also. This leads to two problems:
> 1. unnecessary installation of multiple versions of the same package
> (unnecessary downloading and disk usage)
> 2. awkwardness with applying modifications globally

the depot is immutable by design, which is a deviation from traditional
package managers. It is probably easier to wrap your head around it when
drawing an analogy to how Git works. An archive stored in the depot is
like a Git data blob. Once an archive enters the depot, it should not be
modified. A modification results in a new data blob with a new identity
(in Git's case the commit hash, in the depot's case the versioned path).

A pkg archive is like a Git tag. It is cheap because it merely
references data blobs but doesn't contain any heavy-weight data. If you
look at pkg archives, like for example vbox5-nova-sculpt [1], you will
notice that they are tiny - not even 1 KiB in size.

[1] https://depot.genode.org/genodelabs/pkg/vbox5-nova-sculpt/

When updating the libc, all affected libc-using pkgs (like
vbox5-nova-sculpt) must be updated. But that does not mean downloading a
new vbox executable. Only the new pkg archive that then refers to the
new libc must be updated.

Ideally, the updating of the pkgs should be done by the provider of the
pkg. But in urgent cases, the change can be even made locally on your
machine. It is as easy as copying the corresponding pkg to a new
location (let's say 'local/') within the depot and adjusting the
reference to the libc (the 'archives' list). Then, instead of deploying

  genodelabs/pkg/vbox5-nova-sculpt/2018-06-12

you would deploy

  local/pkg/vbox5-nova-sculpt/quickfix

Granted, it would probably be nice to support such a workflow with some
convenient tooling but that's just sugar on the cake. To stress the
analogy to Git, the "porcelain". The depot that you see today represents
the "plumbing".

Just to complete the picture, the deploy configuration already allows
you to route individual ROM sessions manually. So should you want to
enforce the use of a certain libc version globally, you can achieve that
by adding the following route to the '<common_routes>' of the your
'config/deploy' file:

  <service name="ROM" label_last="libc.lib.so">
    <child name="depot_rom"
           label="blarson/bin/x86_64/libc/2018-06-14/libc.lib.so"/>
  </service>

This is a sledgehammer method I wouldn't really recommend. I just wanted
to illustrate the degrees of freedom you already have with Sculpt.

> A potential piece of a solution would be to allow packages to specify the
> oldest version they are backward-compatible with, e.g. let libc/2018-06-05
> specify that it is compatible with libc >= 2018-05-28.

That's a can of worms that I prefer to keep closed.

> For more flexibility, we could also allow packages to specify multiple
> dependency options, e.g. let noux-system depend on bash or bash-minimal.

As outlined above, pkg archives are cheap (like Git tags). Just create
new pkg archives in these cases.

> Also, we need some mechanism to allow cross-repo substitutions, e.g. let
> blarson/src/libc replace genodelabs/src/libc, or vice versa. For security
> reasons, there should be restrictions on which repos are allowed for
> substitutions, so untrusted/src/libc cannot replace genodelabs/src/libc.

If you want to replace the libc in a genodelabs pkg, you have to host
and publish your modified pkg under 'blarson'. This is exactly as
intended, and secure.

Norman

-- 
Dr.-Ing. Norman Feske
Genode Labs

https://www.genode-labs.com · https://genode.org

Genode Labs GmbH · Amtsgericht Dresden · HRB 28424 · Sitz Dresden
Geschäftsführer: Dr.-Ing. Norman Feske, Christian Helmuth