Add tar index mode to erofs snapshotter#1
Merged
Conversation
aadhar-agarwal
changed the title
aadhar-agarwal
commented
May 23, 2025
aadhar-agarwal
commented
May 23, 2025
aadhar-agarwal
commented
May 23, 2025
hsiangkao
reviewed
May 24, 2025
|
Hi @aadhar-agarwal, I think we could just review the draft as a containerd project draft PR instead? |
jiria
reviewed
May 27, 2025
| return erofs.NewErofsDiffer(cs, config.MkfsOptions), nil | ||
| var opts []erofs.DifferOpt | ||
|
|
||
| if len(config.MkfsOptions) > 0 { |
There was a problem hiding this comment.
What config is this? Containerd toml config? I think passing the deterministic parameters through this would feel pretty unwieldy.
There was a problem hiding this comment.
It's used for erofs extra mount options, because many users have their requirement for -U and -T or they just don't care about reproducible build for non-CoCo use cases.
There was a problem hiding this comment.
for tardev use cases, I think just config the containerd toml as below:
enable_tar_index = true
mkfs_options = "-Uc1b9d5a2-f162-11cf-9ece-0020afc76f16 -T0 --mkfs-time"
Another reason I tend to leave mkfs_option is --mkfs-time is only added since erofs-utils 1.8.2, but erofs-utils 1.7.x also supports --tar=f/i modes.
We could document the recommanded config in the containerd erofs snapshotter doc.
There was a problem hiding this comment.
The mid term plan is to introduce go-erofs library for containerd to avoid erofs-utils dependency, so it doesn't matter too, but needs more development resource for this.
hsiangkao
reviewed
May 27, 2025
aadhar-agarwal
force-pushed
the
aadagarwal/add_tar_index_mode
branch
from
May 29, 2025 00:14
3db16f0 to
61d4b97
Compare
hsiangkao
reviewed
May 29, 2025
hsiangkao
reviewed
May 29, 2025
|
@aadhar-agarwal other than |
hsiangkao
reviewed
May 30, 2025
aadhar-agarwal
force-pushed
the
aadagarwal/add_tar_index_mode
branch
2 times, most recently
from
May 30, 2025 18:11
ea222b5 to
b7ecd1d
Compare
Owner
Author
Created a draft PR in containerd - containerd#11919
|
aadhar-agarwal
force-pushed
the
aadagarwal/add_tar_index_mode
branch
from
May 30, 2025 18:23
b7ecd1d to
b6cb58f
Compare
aadhar-agarwal
force-pushed
the
aadagarwal/add_tar_index_mode
branch
2 times, most recently
from
June 12, 2025 20:53
59e13d3 to
b936a8d
Compare
aadhar-agarwal
force-pushed
the
aadagarwal/add_tar_index_mode
branch
2 times, most recently
from
June 18, 2025 00:52
bac7024 to
78e5e38
Compare
aadhar-agarwal
force-pushed
the
aadagarwal/add_tar_index_mode
branch
from
July 8, 2025 21:38
78e5e38 to
8956352
Compare
Signed-off-by: Aadhar Agarwal <aadagarwal@microsoft.com> Minor style updates to erofs.md and differ_linux.go Signed-off-by: Aadhar Agarwal <aadagarwal@microsoft.com> Add use case for tar index in erofs.md Signed-off-by: Aadhar Agarwal <aadagarwal@microsoft.com>
aadhar-agarwal
force-pushed
the
aadagarwal/add_tar_index_mode
branch
from
July 8, 2025 21:44
8956352 to
b641933
Compare
aadhar-agarwal
pushed a commit
that referenced
this pull request
Oct 28, 2025
Silence a govulncheck noise: ``` Vulnerability #1: GO-2025-3595 Incorrect Neutralization of Input During Web Page Generation in x/net in golang.org/x/net More info: https://pkg.go.dev/vuln/GO-2025-3595 Module: golang.org/x/net Found in: golang.org/x/net@v0.37.0 Fixed in: golang.org/x/net@v0.38.0 Your code is affected by 0 vulnerabilities. This scan also found 0 vulnerabilities in packages you import and 1 vulnerability in modules you require, but your code doesn't appear to call these vulnerabilities. ``` Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
aadhar-agarwal
pushed a commit
that referenced
this pull request
Nov 5, 2025
[Carry #1] fix goroutine leak of container Attach
aadhar-agarwal
pushed a commit
that referenced
this pull request
Feb 14, 2026
This change sets the AppArmor policy used by containerd to indicate it is `abi/3.0`. This was chosen based on some code archeology which indicated that containerd 1.7 came out in March 2023, before the AppArmor 4.0 ABI. The AppArmor policies themselves date to much older; the last apparmor version-checks were removed in 4baa187 and c990e3f, and both were looking for AppArmor 2.8.96 or older, pointing to abi/3.0 being the "correct" one to pick. Nothing is preventing containerd from migrating to a newer AppArmor ABI; note, however, that anything newer than `abi/4.0` will need modifications to preserve UNIX domain sockets. This was tested by building a custom k3s v1.35.0+k3s3, with the following modification: ``` diff --git a/go.mod b/go.mod index 4e7bacd204..0fcaf76b8f 100644 --- a/go.mod +++ b/go.mod @@ -8,7 +8,7 @@ replace ( github.com/cilium/ebpf => github.com/cilium/ebpf v0.12.3 github.com/cloudnativelabs/kube-router/v2 => github.com/k3s-io/kube-router/v2 v2.6.3-k3s1 github.com/containerd/containerd/api => github.com/containerd/containerd/api v1.9.0 - github.com/containerd/containerd/v2 => github.com/k3s-io/containerd/v2 v2.1.5-k3s1 + github.com/containerd/containerd/v2 => github.com/achernya/containerd/v2 v2.0.0-20260206214308-5e0dce89c422 github.com/containerd/imgcrypt => github.com/containerd/imgcrypt v1.1.11 github.com/containerd/stargz-snapshotter => github.com/k3s-io/stargz-snapshotter v0.17.0-k3s1 github.com/docker/distribution => github.com/docker/distribution v2.8.3+incompatible ``` to use a precursor to this commit. Once built, the resulting k3s was tested on a brand-new Proxmox installation: ``` root@containerd-test:~# uname -a Linux containerd-test 6.17.2-1-pve #1 SMP PREEMPT_DYNAMIC PMX 6.17.2-1 (2025-10-21T11:55Z) x86_64 GNU/Linux root@containerd-test:~# pveversion pve-manager/9.1.1/42db4a6cf33dac83 (running kernel: 6.17.2-1-pve) ``` Files were copied over: ``` achernya@achernya-dev:~/src/k3s$ scp -r dist/artifacts/ root@containerd-test: ``` and installed ``` root@containerd-test:~# mkdir -p /var/lib/rancher/k3s/agent/images/ /usr/local/bin root@containerd-test:~# cp artifacts/k3s /usr/local/bin/ root@containerd-test:~# cp artifacts/k3s-airgap-images-amd64.tar.zst /var/lib/rancher/k3s/agent/images/ ``` then finally started with `k3s server`. Argo CD was then installed: ``` root@containerd-test:~# k3s kubectl create namespace argocd namespace/argocd created root@containerd-test:~# k3s kubectl apply -n argocd --server-side --force-conflicts -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml [elided] root@containerd-test:~# k3s kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE argocd argocd-application-controller-0 1/1 Running 0 31s argocd argocd-applicationset-controller-77475dfcf-6b4cb 1/1 Running 0 32s argocd argocd-dex-server-6485c5ddf5-ckp5s 1/1 Running 0 32s argocd argocd-notifications-controller-758f795776-djx69 1/1 Running 0 32s argocd argocd-redis-6cc4bb5db5-lt9fh 1/1 Running 0 32s argocd argocd-repo-server-c76cf57cd-mr4mc 1/1 Running 0 32s argocd argocd-server-6f85b59c87-w6cns 0/1 Running 0 32s kube-system coredns-6b4688786f-pnds2 1/1 Running 0 4m1s kube-system helm-install-traefik-crd-cn28g 0/1 Completed 0 4m1s kube-system helm-install-traefik-hc9gp 0/1 Completed 2 4m1s kube-system local-path-provisioner-6bc6568469-7wglx 1/1 Running 0 4m1s kube-system metrics-server-77dbbf84b-nqzsc 1/1 Running 0 4m1s kube-system svclb-traefik-fe6d3a0b-z7jsp 2/2 Running 0 3m14s kube-system traefik-5fdc878c8d-cjhx5 1/1 Running 0 3m15s ``` Fixes: containerd#12726 Signed-off-by: Alex Chernyakhovsky <alex@achernya.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add EROFS Tar Index Mode Support
This PR introduces a new "tar index" mode for the EROFS snapshotter and differ, providing a more efficient way to handle OCI image layers by enabling direct, indexed access to tar content without full extraction. The changes include:
Key Highlights
Tar Index Mode Implementation
Configuration Options
enable_tar_indexconfiguration flag for both the snapshotter and differ, available in the respective TOML configs.Code Changes
internal/erofsutils/mount_linux.go: Added logic for tar index generation and combination.plugins/diff/erofs/differ_linux.go: New differ options and tar index application logic.plugins/diff/erofs/plugin/plugin_linux.go: Plugin config and initialization updated to support tar index mode.docs/snapshotters/erofs.md: Documentation updated with usage, configuration, and mode explanation.Usage
Enable tar index mode in your containerd configuration:
The system will automatically verify
mkfs.erofscapabilities and enable the feature if supported.