uid-nri-plugin

A containerd NRI (Node Resource Interface) plugin that injects deterministic uid/gid mappings into every container's OCI spec, enabling arbitrary Kubernetes workload images to run within a limited HPC subuid allocation.

Why?

We are generating this because we need to run Userspace-Kubernetes on a multi-tenant HPC system, where each user has a much smaller range of UIDs. Rootless containers need to handle any uid in 0-65535 since arbitrary images from the wild can use any uid. Standard uid mapping can't cover 65536 values with only 2048 slots.

Solution

At container creation time, inject OCI UIDMappings / GIDMappings entries that wrap the full 0-65535 range into the pool via modulo:

host_uid = base_uid + (container_uid % pool_size)

This is expressed as repeated contiguous blocks in the OCI spec:

container 0    - 2047  → host 2937359488 - 2937361535
container 2048 - 4095  → host 2937359488 - 2937361535  (collision, warning logged)
container 4096 - 6143  → host 2937359488 - 2937361535  (collision, warning logged)
...

Collisions mean two different container uids map to the same host uid. It is akin to everything running as the HPC user, which I think is OK, at least to test. The goal is host escape prevention, not inter-container uid isolation.

Architecture

HPC host (rootless podman, 2048 subuid slots)
  └── usernetes_node container
        ├── systemd
        ├── uid-nri-plugin  ← this binary, at /opt/nri/bin/00-uid-mapper
        │     reads /proc/self/uid_map on startup
        │     connects to containerd NRI socket
        │     injects UIDMappings into every container OCI spec
        └── containerd (NRI enabled by default in v2)
              └── kubernetes workload containers
                    OCI spec has full 0-65535 uid coverage

How NRI pre-started plugins work

NRI in containerd v2 automatically discovers and launches binaries in /opt/nri/bin/ on startup. The naming convention is <index>-<name>. The plugin connects back to containerd over the NRI socket and registers its event subscriptions (CreateContainer in our case).

No containerd config changes are needed — NRI is enabled by default in v2.

Build

go build -o 00-uid-mapper ./cmd/uid-nri-plugin

Or via multi-stage Docker build — I will add this to our Usernetes Dockerfile.

Verify it's working

Inside the usernetes_node container:

# Check the plugin is running
systemctl status uid-nri-plugin

# Check containerd sees it
ctr plugins ls | grep nri

# After starting a pod, check its OCI spec has uid mappings
cat /run/containerd/io.containerd.runtime.v2.task/k8s.io/<id>/config.json \
  | jq '.linux.uidMappings'

License

HPCIC DevTools is distributed under the terms of the MIT license. All new contributions must be made under this license.

See LICENSE, COPYRIGHT, and NOTICE for details.

SPDX-License-Identifier: (MIT)

LLNL-CODE- 842614