allow multiple slices with the same content paths to co-exist within the same Chisel release

[cjdcordeiro]

Diffs between dotnet-host-7.0 and dotnet-host SDFs

package: dotnet-host-7.0                                      | package: dotnet-host
slices:                                                         slices:
  bins:                                                           bins:
    essential:                                                      essential:
      - libc6_libs                                                    - libc6_libs
      - libgcc-s1_libs                                                - libgcc-s1_libs
      - libstdc++6_libs                                               - libstdc++6_libs
    contents:                                                       contents:
      /usr/lib/dotnet/dotnet:                                         /usr/lib/dotnet/dotnet:

Problem

By design, Chisel is currently looking at the entire Chisel release to catch any conflicts [1]. The reason why it was designed this way was to cope with people doing /usr/lib/* in their slices, thus avoiding the produced outputs from growing unbounded as fat packages change.

In practice, this means that even if we’re just installing .NET7, Chisel will still complain about conflicts with .NET6, even though it is not targeted. And that’s because dotnet-host-7.0 and dotnet-host both have the same content path.

Reproduce

Copy the two SDFs from above, into an existing chisel-release (tested with kinetic), as slices/dotnet-host-7.0.yaml and slices/dotnet-host.yaml, respectively.

Then run

$ chisel cut --release ./ --root rootfs base-files_base

And you’ll get

2023/02/03 12:36:33 Processing ./ release...
error: slices dotnet-host_bins and dotnet-host-7.0_bins conflict on /usr/lib/dotnet/dotnet

Chisel version: https://github.com/canonical/chisel/tree/bd27f8700cd7d2a6b4e0df6b10c3761c83a70485

Refs

[1] https://github.com/canonical/chisel/blob/main/internal/setup/setup.go#L154

[kholia]

I was running into a similar problem when generating slices for python3 stuff.

I came up with the following dirty hack to workaround this issue:

diff --git a/internal/setup/setup.go b/internal/setup/setup.go
index c47c050..a1a00ff 100644
--- a/internal/setup/setup.go
+++ b/internal/setup/setup.go
@@ -179,7 +179,8 @@ func (r *Release) validate() error {
                                if old.Package > new.Package || old.Package == new.Package && old.Name > new.Name {
                                        old, oldPath, new, newPath = new, newPath, old, oldPath
                                }
-                               return fmt.Errorf("slices %s and %s conflict on %s and %s", old, new, oldPath, newPath)
+                               // return fmt.Errorf("slices %s and %s conflict on %s and %s", old, new, oldPath, newPath)
+                               return nil
                        }
                }
                paths[newPath] = new

[cjdcordeiro]

what were the conflicting paths in your case @kholia ?

[kholia]

The conflicting paths (centered around /usr/lib/python3.11/ path) involved the following slices:

• https://github.com/kholia/chisel-releases/blob/base_image_python/slices/libpython3.11-minimal.yaml
• https://github.com/kholia/chisel-releases/blob/base_image_python/slices/libpython3.11-stdlib.yaml
• https://github.com/kholia/chisel-releases/blob/base_image_python/slices/libpython3.11.yaml

Here is the full patch I used to workaround this issue:

kholia@3ef4f1a

[kholia]

My Python slices are quite hacky at the moment - I just wanted to do a quick PoC in the shortest amount of time.

With these hacky slices, and the patched chisel tool, I was able to get a python3 slice going just fine.

[cjdcordeiro]

Cool. Glad you were able to work around it @kholia .

Btw, we'll probably have, in a couple of days, python3.8 for Focal -> canonical/chisel-releases#44

[endersonmaia]

Just for the record, I just hit this issue with busybox and busybox-static.

[cjdcordeiro]

Noted @endersonmaia . Thanks for the heads up. This is in our roadmap and we'll definitely tackle it soon.