Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Observability plugin: node-exporter cannot start in strict mode because it fails to mount the root directory #4783

Open
Dzeri96 opened this issue Dec 14, 2024 · 0 comments

Comments

@Dzeri96
Copy link

Dzeri96 commented Dec 14, 2024

Summary

After installing the observability addon in a microk8s strict snap installation, the node-exporter pod fails to start with the following error: spec: failed to generate spec: path "/" is mounted on "/" but it is not a shared or slave mount.

In dmesg I see messages like these: [2363332.633809] audit: type=1400 audit(1734209045.433:5684): apparmor="DENIED" operation="open" class="file" profile="snap.microk8s.microk8s" name="/usr/games/" pid=1643291 comm="kubectl" requested_mask="r" denied_mask="r" fsuid=0 ouid=0. I find the path weird but it probably correlates to the root mount attempt.

What Should Happen Instead?

Either a root mount is allowed, or the container starts without it.

Reproduction Steps

  1. Install microk8s from the 1.31-strict/stable channel
  2. Enable the observability addon

Introspection Report

inspection-report-20241214_233524.tar.gz

Can you suggest a fix?

To be honest I'm glad the containment worked and no container can read the entire host filesystem.
In this issue in the prometheus repo, people have expressed concerns regarding the full root mount but nothing has been done.
Apart from fixing the issue upstream, I guess all that can be done is to disable this mount altogether and emit a warning that some metrics won't be available in strict confinement mode.

Speaking of the upstream, after removing the root mount from the DaemonSet spec, the only error I see is Failed to open directory, disabling udev device properties" path=/host/root/run/udev/data.
It would seem like we can just mount this one directory directly and everything should work fine.

Are you interested in contributing with a fix?

Yes, but I'd rather this be fixed in the node-exporter repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant