Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BPF Recorder: Exclude Container Initialization from Recorded Profile #2623

Merged
merged 8 commits into from
Jan 8, 2025

Conversation

mhils
Copy link
Contributor

@mhils mhils commented Dec 13, 2024

What type of PR is this?

/kind feature

What this PR does / why we need it:

This PR makes us 1) detect when containers are (about to be) initialized, and 2) makes sure the permissions exercised before this point are not included in the recorded profile.

Which issue(s) this PR fixes:

Fixes #2576

Does this PR have test?

Covered by existing E2E tests.

Special notes for your reviewer:

99% sure that some kernels won't support sched_prepare_exec, but let's see if it works in the first place before we revise.

Switched from sched_prepare_exec to an execve tracepoint.

Switched from execve to setsid to make sure we capture the last part of runc startup for seccomp.

Switched from execve to getppid in an attempt to make things more reliable.

Switched to an approach where we detect the initial unshare and then start suppressing events until runc reaches getppid. This avoids data races on the Go side.

Switched back to an approach where we clean the mount namespace instead of suppressing events.

Does this PR introduce a user-facing change?

The BPF profile recorder now excludes unnecessary permissions exercised during container init.

@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Dec 13, 2024
@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Dec 13, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @mhils. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Dec 13, 2024
@codecov-commenter
Copy link

codecov-commenter commented Dec 13, 2024

Codecov Report

Attention: Patch coverage is 0% with 17 lines in your changes missing coverage. Please review.

Project coverage is 41.36%. Comparing base (11d77f4) to head (ff68cef).
Report is 602 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2623      +/-   ##
==========================================
- Coverage   45.50%   41.36%   -4.14%     
==========================================
  Files          79      109      +30     
  Lines        7782    18231   +10449     
==========================================
+ Hits         3541     7541    +4000     
- Misses       4099    10192    +6093     
- Partials      142      498     +356     

@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 13, 2024
@k8s-ci-robot k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Dec 16, 2024
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Dec 16, 2024
@mhils mhils marked this pull request as draft December 16, 2024 15:00
@k8s-ci-robot k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 16, 2024
@mhils mhils force-pushed the container-init branch 3 times, most recently from d710ac4 to ed678c9 Compare December 18, 2024 09:07
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 18, 2024
@ccojocar
Copy link
Contributor

It is an impressive reduction in permissions in both apapmror and secocmp. 🤞 that this works!

@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 18, 2024
@mhils mhils mentioned this pull request Dec 18, 2024
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jan 7, 2025
@mhils mhils marked this pull request as ready for review January 7, 2025 15:59
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 7, 2025
@k8s-ci-robot k8s-ci-robot requested a review from JAORMX January 7, 2025 15:59
@ccojocar
Copy link
Contributor

ccojocar commented Jan 7, 2025

/test pull-security-profiles-operator-test-e2e

@mhils mhils force-pushed the container-init branch 3 times, most recently from 11071b4 to 5406024 Compare January 8, 2025 09:46
mhils and others added 7 commits January 8, 2025 09:47
testing has shown that we sometimes get reused mount namespaces that are already tainted. we can avoid these issues by clearing when we detect init.
Co-authored-by: Cosmin Cojocar <[email protected]>
for example, `ARGS='-run "TestSuite/TestSecurityProfilesOperator/namespaced:_Seccomp:_Verify_base_profile_merge"' CONTAINER_RUNTIME=docker make test-e2e` can be used to run a particular e2e test
@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jan 8, 2025
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jan 8, 2025
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 8, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ccojocar, mhils

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ccojocar
Copy link
Contributor

ccojocar commented Jan 8, 2025

/test pull-security-profiles-operator-test-e2e

@k8s-ci-robot k8s-ci-robot merged commit 5fafe71 into kubernetes-sigs:main Jan 8, 2025
28 checks passed
@mhils mhils deleted the container-init branch January 8, 2025 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Apparmor recorder allows extra capabilities in the profile which doesn't seem to be required
5 participants