add use case-specific doc for libicu #21
Comments
|
This conversation came up today. We should address it. You'll see that I added commentary about ICU in my announcement. |
|
This example Dockerfile demonstrates how one would include ICU to a downstream image: FROM golang:1.18 as chisel
RUN git clone --depth 1 -b main https://github.com/canonical/chisel /opt/chisel
WORKDIR /opt/chisel
RUN go build ./cmd/chisel
FROM mcr.microsoft.com/dotnet/sdk:6.0-jammy AS build
COPY --from=chisel /opt/chisel/chisel /usr/bin/
RUN mkdir /rootfs \
&& chisel cut --release "ubuntu-22.04" --root /rootfs \
libicu70_libs
WORKDIR /source
# copy csproj and restore as distinct layers
COPY *.csproj .
RUN dotnet restore
# copy and publish app and libraries
COPY . .
RUN dotnet publish -c release -o /app --no-restore
# final stage/image
FROM mcr.microsoft.com/dotnet/nightly/runtime:6.0-jammy-chiseled
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false
COPY --from=build /rootfs /
WORKDIR /app
COPY --from=build /app .
ENTRYPOINT ["dotnet", "dotnetapp.dll"]I've tested it out with our sample app in dotnet-docker and it works as expected. Note that due to canonical/chisel#10, it will have result in inefficiencies due to the dependencies that libicu70 has. Essentially, the resulting image ends up with two copies of libc6, libgcc-s1, and libstdc++6. Therefore, this scenario stresses the importance of having a fix for canonical/chisel#10. UPDATE: See my next post which describes a workaround for canonical/chisel#10 that allows you to achieve a clean image without duplicate data. |
I have a workaround for this issue. It's a bit verbose in the Dockerfile, but gives you exactly what you need with 0 bytes of duplicate data in the image. FROM golang:1.18 as chisel
RUN git clone --depth 1 -b main https://github.com/canonical/chisel /opt/chisel
WORKDIR /opt/chisel
RUN go build ./cmd/chisel
FROM mcr.microsoft.com/dotnet/sdk:6.0-jammy AS build
RUN apt-get update \
&& apt-get install -y fdupes \
&& rm -rf /var/lib/apt/lists/*
COPY --from=chisel /opt/chisel/chisel /usr/bin/
COPY --from=mcr.microsoft.com/dotnet/nightly/runtime:6.0-jammy-chiseled / /runtime-ref
RUN mkdir /rootfs \
&& chisel cut --release "ubuntu-22.04" --root /rootfs \
libicu70_libs \
\
# Remove duplicates from rootfs that exist in runtime-ref
&& fdupes /runtime-ref /rootfs -rdpN \
\
# Delete duplicate symlinks
# Function to find and format symlinks w/o including root dir (format: /path/to/symlink /path/to/target)
&& getsymlinks() { find $1 -type l -printf '%p %l\n' | sed -n "s/^\\$1\\(.*\\)/\\1/p"; } \
# Combine set of symlinks between rootfs and runtime-ref
&& (getsymlinks "/rootfs"; getsymlinks "/runtime-ref") \
# Sort them
| sort \
# Find the duplicates
| uniq -d \
# Extract just the path to the symlink
| cut -d' ' -f1 \
# Prepend the rootfs directory to the paths
| sed -e 's/^/\/rootfs/' \
# Delete the files
| xargs rm \
\
# Delete empty directories
&& find /rootfs -type d -empty -delete
WORKDIR /source
# copy csproj and restore as distinct layers
COPY *.csproj .
RUN dotnet restore
# copy and publish app and libraries
COPY . .
RUN dotnet publish -c release -o /app --no-restore
# final stage/image
FROM mcr.microsoft.com/dotnet/nightly/runtime:6.0-jammy-chiseled
ENV DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=false
COPY --from=build /rootfs /
WORKDIR /app
COPY --from=build /app .
ENTRYPOINT ["dotnet", "dotnetapp.dll"]The key differences from my previous post are in the build stage with these sets of lines: RUN apt-get update \
&& apt-get install -y fdupes \
&& rm -rf /var/lib/apt/lists/*COPY --from=mcr.microsoft.com/dotnet/nightly/runtime:6.0-jammy-chiseled / /runtime-ref # Remove duplicates from rootfs that exist in runtime-ref
&& fdupes /runtime-ref /rootfs -rdpN \
\
# Delete duplicate symlinks
# Function to find and format symlinks w/o including root dir (format: /path/to/symlink /path/to/target)
&& getsymlinks() { find $1 -type l -printf '%p %l\n' | sed -n "s/^\\$1\\(.*\\)/\\1/p"; } \
# Combine set of symlinks between rootfs and runtime-ref
&& (getsymlinks "/rootfs"; getsymlinks "/runtime-ref") \
# Sort them
| sort \
# Find the duplicates
| uniq -d \
# Extract just the path to the symlink
| cut -d' ' -f1 \
# Prepend the rootfs directory to the paths
| sed -e 's/^/\/rootfs/' \
# Delete the duplicate files
| xargs rm \
\
# Delete empty directories
&& find /rootfs -type d -empty -deleteThis cleans up the rootfs in preparation to have it copied into the final stage. It removes all the duplicate files that already exist in the target image. This leaves you with a very clean final image with no wasted space as can be seen from this output from the dive tool: Compare this to the output of dive if you don't clean up the rootfs directory: But all of this is just a workaround for the chisel tool not being aware of the dependencies that already exist in the target image (canonical/chisel#10). If it was aware of that, the output of chisel would produce a /rootfs directory that is equivalent to what is achieved by this workaround. |
|
cool workaround @mthalman . thanks. Indeed, canonical/chisel#10 is asking for a broader solution. We already have someone (@rebornplusplus) who's going to start working on it soon. |
|
Is there any progress on official guidance how to include ICU package in Chiseled images? |
|
@kamilzzz the proposed solution from #21 (comment) is the latest guide on this. As you can see from #21 (comment), this is not the ideal solution though, and the latter is a bit too verbose to be included as the official way to achieve this. We're expecting a new feature in Chisel, in the near future, which migth make this process much cleaner. But until then, #21 (comment) is the way to go. |
|
Today I tried to just rerun the same build pipeline I had for my application with this hacky hack script So mine Dockerfile was like that I received such results When I changed Dockerfile to |
|
hi @Bafff this can indeed happen since this recipe is building the Chisel binary from its git HEAD. We had to recently change our Dockerfiles in a similar manner -> 0957b39 To avoid this, you can:
|
|
Note: We're planning on producing Microsoft images with icu built-in shortly. That said, making chisel easier to use to add ICU or other libraries remains important. |
|
@cjdcordeiro, this issue has been open for quite some time and there's an increasing interest in Chiseled images now. While we have Chiseled images with ICU built-in, we're still interested in guidance for how to install additional packages on top of an existing chiseled Ubuntu installation. Are there any plans for that? |
|
hi @lbussell . This is a generic chiselling topic, and we'll be working on creating such documentation over the next few months. For now, I have added some .net-specific guides on how to do that, in https://github.com/ubuntu-rocks/dotnet/#building-the-net-application-image. I hope that helps ;) |
Thanks! Will this work with chisel-wrapper in order to generate a merged dpkg status file? We'd like to retain the ability to accurately scan images with tools like Trivy, etc. |
|
I'm afraid not. The wrapper generates a dpkg/status file only for the given slices in the corresponding command. Until the Chisel DB is merged, I'm afraid you'd have to merge the contents. Something like (using the example from https://github.com/ubuntu-rocks/dotnet/#building-the-net-application-image): ...
COPY --from=mcr.microsoft.com/dotnet/runtime-deps:8.0.0-jammy-chiseled-amd64 /var/lib/dpkg/status /original-dpkg-status
RUN chisel-wrapper --generate-dpkg-status /additional-dpkg-status ... && \
cat /additional-dpkg-status /original-dpkg-status > /merged-dpkg-status
...
# FINAL IMAGE
...
COPY --from=base /merged-dpkg-status /var/lib/dpkg/statusIt isn't super clean, but our efforts are currently going to the Chisel DB as we believe that's the right path. |
|
Our containers depend on the geoipupdate utility to keep the MaxMind database fresh on the container. How can we install the deb package in our image that is based on chiseled image? When we have the |
|
Hi @johnwc . Mixing debs and slices on the same image can be tricky because the installation of debs relies on package managers and other system dependencies, which is one of the things chisel removes. My suggestion is: propose the slice definitions for that package in https://github.com/canonical/chisel-releases/, and then install those pkg slices alongside the other ones. If you really need the deb, then you may try to run |
|
I created a request for a guide in the chisel repo. canonical/chisel#118 |
libicu might be needed for some use cases, but we're not including it in these chiseled images.
we should then document the process (with an example) of adding libicu to the existing runtime* ROCKs
The text was updated successfully, but these errors were encountered: