Skip to content

Commit 60bfed8

Browse files
authored
Grant pods/attach only for stdio transport (#6841)
* Grant pods/attach only for stdio transport The proxy-runner Role created for every MCPServer carried pods/attach (create, get) regardless of transport. Only stdio uses it: the runner reaches a stdio MCP server by attaching to its pod through Client.AttachToWorkload, which is called only from pkg/transport/stdio.go. A streamable-http or sse server talks to its pod over the network and never attaches. Security auditors flag pods/attach as equivalent to pod exec, so clusters with strict policies saw a finding on every MCPServer deployment for a permission that could not be used. Split the rule out of defaultRBACRules into stdioRBACRules and choose between them in rbacRulesForTransport. An absent transport is treated as stdio, matching the CRD default, so an object stored before that default existed keeps working. Already-deployed servers are corrected without operator action: EnsureRBACResources replaces Rules wholesale, so the next reconcile rewrites the Role. Closes #6230 Signed-off-by: hellouz818 <hellouz818@gmail.com> * Split proxy-runner Role fixtures by transport The chainsaw operator E2E suites asserted one shared proxyrunner-role.yaml for the stdio, sse, streamable-http and stdio-streamable-http scenarios. That stopped holding once pods/attach became transport-dependent: chainsaw matches the rules array positionally and on length, so the stdio scenarios failed on the rule having moved to the end, and the HTTP ones would fail on it being absent. Replace the shared file with proxyrunner-role-stdio.yaml (six rules, pods/attach last) and proxyrunner-role-http.yaml (the five shared rules), and point each scenario at the one matching the transport in its mcpserver.yaml. Both files still spell out the complete expected Role, so an over-granted rule fails the assertion rather than slipping through. Only the multi-tenancy run reached CI: chainsaw runs with FailFast and both suites share one shell step, so the single-tenancy scenarios never executed. Refs #6230 Signed-off-by: hellouz818 <hellouz818@gmail.com> --------- Signed-off-by: hellouz818 <hellouz818@gmail.com>
1 parent 87ed79d commit 60bfed8

14 files changed

Lines changed: 281 additions & 26 deletions

File tree

‎cmd/thv-operator/controllers/mcpserver_controller.go‎

Lines changed: 34 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ import (
1212
"fmt"
1313
"maps"
1414
"os"
15+
"slices"
1516
"strconv"
1617
"strings"
1718
"time"
@@ -62,9 +63,10 @@ type MCPServerReconciler struct {
6263
ImagePullSecretsDefaults imagepullsecrets.Defaults
6364
}
6465

65-
// defaultRBACRules are the default RBAC rules that the
66-
// ToolHive ProxyRunner and/or MCP server needs to have in order to run.
66+
// defaultRBACRules are the RBAC rules that the ToolHive ProxyRunner and/or MCP
67+
// server needs to have in order to run, whatever transport is configured.
6768
// These permissions are needed for MCPServer which deploys and manages MCP server containers.
69+
// Transport-specific rules are added on top; see rbacRulesForTransport.
6870
var defaultRBACRules = []rbacv1.PolicyRule{
6971
{
7072
APIGroups: []string{"apps"},
@@ -88,16 +90,41 @@ var defaultRBACRules = []rbacv1.PolicyRule{
8890
},
8991
{
9092
APIGroups: []string{""},
91-
Resources: []string{"pods/attach"},
92-
Verbs: []string{"create", "get"},
93+
Resources: []string{"configmaps"},
94+
Verbs: []string{"get", "list", "watch"},
9395
},
96+
}
97+
98+
// stdioRBACRules are granted on top of defaultRBACRules for stdio transport only.
99+
// The proxy runner talks to a stdio MCP server by attaching to the pod's stdio
100+
// (Client.AttachToWorkload, reached only from the stdio transport); the HTTP
101+
// transports speak to the pod over the network and never attach. Security
102+
// auditors treat pods/attach as equivalent to pod exec, so withhold it where it
103+
// cannot be used.
104+
var stdioRBACRules = []rbacv1.PolicyRule{
94105
{
95106
APIGroups: []string{""},
96-
Resources: []string{"configmaps"},
97-
Verbs: []string{"get", "list", "watch"},
107+
Resources: []string{"pods/attach"},
108+
Verbs: []string{"create", "get"},
98109
},
99110
}
100111

112+
// rbacRulesForTransport returns the proxy-runner Role rules for the given
113+
// MCPServer transport. An empty transport is treated as stdio: the CRD defaults
114+
// the field to stdio, so an object can only reach here with an empty value if it
115+
// predates that default or was built in Go, and granting the extra permission
116+
// keeps such a server working rather than breaking it.
117+
//
118+
// The result is always a fresh slice — appending to defaultRBACRules directly
119+
// would let a future rule share its backing array.
120+
func rbacRulesForTransport(mcpTransport string) []rbacv1.PolicyRule {
121+
rules := slices.Clone(defaultRBACRules)
122+
if mcpTransport == stdioTransport || mcpTransport == "" {
123+
rules = append(rules, stdioRBACRules...)
124+
}
125+
return rules
126+
}
127+
101128
// remoteProxyRBACRules defines minimal RBAC permissions for MCPRemoteProxy.
102129
// Remote proxies only connect to external MCP servers and do not deploy containers,
103130
// so they only need read access to ConfigMaps and Secrets (for OIDC/token exchange).
@@ -1105,7 +1132,7 @@ func (r *MCPServerReconciler) ensureRBACResources(ctx context.Context, mcpServer
11051132
if _, err := rbacClient.EnsureRBACResources(ctx, rbac.EnsureRBACResourcesParams{
11061133
Name: proxyRunnerNameForRBAC,
11071134
Namespace: mcpServer.Namespace,
1108-
Rules: defaultRBACRules,
1135+
Rules: rbacRulesForTransport(mcpServer.Spec.Transport),
11091136
Owner: mcpServer,
11101137
ImagePullSecrets: imagePullSecrets,
11111138
}); err != nil {

‎cmd/thv-operator/controllers/mcpserver_rbac_test.go‎

Lines changed: 105 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ package controllers
66
import (
77
"context"
88
"fmt"
9+
"slices"
910
"testing"
1011

1112
"github.com/stretchr/testify/assert"
@@ -70,7 +71,25 @@ func (tc *testContext) assertRoleExists(t *testing.T) {
7071
require.NoError(t, err)
7172
assert.Equal(t, tc.proxyRunnerNameForRBAC, role.Name)
7273
assert.Equal(t, tc.mcpServer.Namespace, role.Namespace)
73-
assert.Equal(t, defaultRBACRules, role.Rules)
74+
assert.Equal(t, expectedRoleRules(tc.mcpServer.Spec.Transport), role.Rules)
75+
}
76+
77+
// podAttachRuleFixture restates the pods/attach rule rather than reusing
78+
// stdioRBACRules, so a change to the production rule's verbs or resources shows
79+
// up as a test failure instead of being mirrored silently.
80+
var podAttachRuleFixture = rbacv1.PolicyRule{
81+
APIGroups: []string{""},
82+
Resources: []string{"pods/attach"},
83+
Verbs: []string{"create", "get"},
84+
}
85+
86+
// expectedRoleRules is what a proxy-runner Role should carry for a transport:
87+
// the shared rules, plus pods/attach only where the proxy actually attaches.
88+
func expectedRoleRules(mcpTransport string) []rbacv1.PolicyRule {
89+
if mcpTransport == "stdio" || mcpTransport == "" {
90+
return append(slices.Clone(defaultRBACRules), podAttachRuleFixture)
91+
}
92+
return defaultRBACRules
7493
}
7594

7695
func (tc *testContext) assertRoleBindingExists(t *testing.T) {
@@ -418,6 +437,91 @@ func TestEnsureRBACResources_ImagePullSecrets(t *testing.T) {
418437
assert.Equal(t, expectedSecrets, mcpServerSA.ImagePullSecrets)
419438
}
420439

440+
// TestEnsureRBACResources_Role_PodsAttachScopedToStdio pins the permission to
441+
// the one transport that uses it. The proxy runner attaches to the MCP pod's
442+
// stdio only for the stdio transport; granting pods/attach for the HTTP
443+
// transports hands every such deployment a permission equivalent to pod exec
444+
// that it can never exercise.
445+
func TestEnsureRBACResources_Role_PodsAttachScopedToStdio(t *testing.T) {
446+
t.Parallel()
447+
448+
tests := []struct {
449+
name string
450+
transport string
451+
wantPodsAttach bool
452+
}{
453+
{
454+
name: "stdio attaches to the pod",
455+
transport: "stdio",
456+
wantPodsAttach: true,
457+
},
458+
{
459+
name: "streamable-http never calls AttachToWorkload",
460+
transport: "streamable-http",
461+
wantPodsAttach: false,
462+
},
463+
{
464+
name: "sse never calls AttachToWorkload",
465+
transport: "sse",
466+
wantPodsAttach: false,
467+
},
468+
{
469+
name: "empty transport is treated as the stdio CRD default",
470+
transport: "",
471+
wantPodsAttach: true,
472+
},
473+
}
474+
475+
for i, tt := range tests {
476+
t.Run(tt.name, func(t *testing.T) {
477+
t.Parallel()
478+
479+
tc := setupTest(t, fmt.Sprintf("test-server-attach-%d", i), "default")
480+
tc.mcpServer.Spec.Transport = tt.transport
481+
482+
require.NoError(t, tc.ensureRBACResources())
483+
484+
role := &rbacv1.Role{}
485+
require.NoError(t, tc.client.Get(context.TODO(), types.NamespacedName{
486+
Name: tc.proxyRunnerNameForRBAC,
487+
Namespace: tc.mcpServer.Namespace,
488+
}, role))
489+
490+
if tt.wantPodsAttach {
491+
assert.Contains(t, role.Rules, podAttachRuleFixture,
492+
"transport %q attaches to the pod and needs pods/attach", tt.transport)
493+
} else {
494+
assert.NotContains(t, role.Rules, podAttachRuleFixture,
495+
"transport %q never attaches and must not be granted pods/attach", tt.transport)
496+
}
497+
498+
// Whatever the transport, the shared rules must survive intact.
499+
for _, want := range defaultRBACRules {
500+
assert.Contains(t, role.Rules, want)
501+
}
502+
})
503+
}
504+
}
505+
506+
// TestRBACRulesForTransport_ReturnsFreshSlice guards the backing array of the
507+
// package-level defaultRBACRules: appending the stdio rule onto it directly
508+
// would let one call mutate what every later call sees.
509+
func TestRBACRulesForTransport_ReturnsFreshSlice(t *testing.T) {
510+
t.Parallel()
511+
512+
before := slices.Clone(defaultRBACRules)
513+
514+
stdio := rbacRulesForTransport("stdio")
515+
require.Contains(t, stdio, podAttachRuleFixture)
516+
517+
assert.Equal(t, before, defaultRBACRules, "defaultRBACRules must not be mutated")
518+
assert.NotContains(t, rbacRulesForTransport("streamable-http"), podAttachRuleFixture)
519+
520+
// Mutating a returned slice must not reach a later caller.
521+
stdio[0] = rbacv1.PolicyRule{APIGroups: []string{"mutated"}}
522+
assert.NotEqual(t, stdio[0], rbacRulesForTransport("stdio")[0])
523+
}
524+
421525
func createTestMCPServer(name, namespace string) *mcpv1beta1.MCPServer {
422526
// Builder defaults (image "test-image:latest", transport "stdio",
423527
// proxyPort 8080) match this fixture exactly.
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Proxy-runner Role for the HTTP transports (sse, streamable-http).
2+
# The proxy reaches such an MCP server over the network, so it is not granted
3+
# pods/attach. Keep in step with defaultRBACRules in
4+
# cmd/thv-operator/controllers/mcpserver_controller.go; chainsaw matches the
5+
# rules positionally, so order matters.
6+
apiVersion: rbac.authorization.k8s.io/v1
7+
kind: Role
8+
metadata:
9+
name: yardstick-proxy-runner
10+
namespace: test-namespace
11+
rules:
12+
- apiGroups:
13+
- apps
14+
resources:
15+
- statefulsets
16+
verbs:
17+
- get
18+
- list
19+
- watch
20+
- create
21+
- update
22+
- patch
23+
- delete
24+
- apiGroups:
25+
- ""
26+
resources:
27+
- services
28+
verbs:
29+
- get
30+
- list
31+
- watch
32+
- create
33+
- update
34+
- patch
35+
- delete
36+
- apiGroups:
37+
- ""
38+
resources:
39+
- pods
40+
verbs:
41+
- get
42+
- list
43+
- watch
44+
- apiGroups:
45+
- ""
46+
resources:
47+
- pods/log
48+
verbs:
49+
- get
50+
- apiGroups:
51+
- ""
52+
resources:
53+
- configmaps
54+
verbs:
55+
- get
56+
- list
57+
- watch

test/e2e/chainsaw/operator/multi-tenancy/test-scenarios/common/proxyrunner-role.yaml renamed to test/e2e/chainsaw/operator/multi-tenancy/test-scenarios/common/proxyrunner-role-stdio.yaml

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,8 @@
1+
# Proxy-runner Role for the stdio transport.
2+
# The proxy attaches to the MCP pod's stdio, so it is granted pods/attach on
3+
# top of the shared rules. Keep in step with defaultRBACRules and
4+
# stdioRBACRules in cmd/thv-operator/controllers/mcpserver_controller.go;
5+
# chainsaw matches the rules positionally, so order matters.
16
apiVersion: rbac.authorization.k8s.io/v1
27
kind: Role
38
metadata:
@@ -45,15 +50,15 @@ rules:
4550
- apiGroups:
4651
- ""
4752
resources:
48-
- pods/attach
53+
- configmaps
4954
verbs:
50-
- create
5155
- get
56+
- list
57+
- watch
5258
- apiGroups:
5359
- ""
5460
resources:
55-
- configmaps
61+
- pods/attach
5662
verbs:
63+
- create
5764
- get
58-
- list
59-
- watch

‎test/e2e/chainsaw/operator/multi-tenancy/test-scenarios/sse/chainsaw-test.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ spec:
3838
- assert:
3939
file: assert-mcpserver-proxy-runner-svc.yaml
4040
- assert:
41-
file: ../common/proxyrunner-role.yaml
41+
file: ../common/proxyrunner-role-http.yaml
4242
- assert:
4343
file: ../common/proxyrunner-rolebinding.yaml
4444
- assert:

‎test/e2e/chainsaw/operator/multi-tenancy/test-scenarios/stdio-streamable-http/chainsaw-test.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ spec:
3434
- assert:
3535
file: assert-mcpserver-proxy-runner-svc.yaml
3636
- assert:
37-
file: ../common/proxyrunner-role.yaml
37+
file: ../common/proxyrunner-role-stdio.yaml
3838
- assert:
3939
file: ../common/proxyrunner-rolebinding.yaml
4040
- assert:

‎test/e2e/chainsaw/operator/multi-tenancy/test-scenarios/stdio/chainsaw-test.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ spec:
3434
- assert:
3535
file: assert-mcpserver-proxy-runner-svc.yaml
3636
- assert:
37-
file: ../common/proxyrunner-role.yaml
37+
file: ../common/proxyrunner-role-stdio.yaml
3838
- assert:
3939
file: ../common/proxyrunner-rolebinding.yaml
4040
- assert:

‎test/e2e/chainsaw/operator/multi-tenancy/test-scenarios/streamable-http/chainsaw-test.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ spec:
3838
- assert:
3939
file: assert-mcpserver-proxy-runner-running.yaml
4040
- assert:
41-
file: ../common/proxyrunner-role.yaml
41+
file: ../common/proxyrunner-role-http.yaml
4242
- assert:
4343
file: ../common/proxyrunner-rolebinding.yaml
4444
- assert:
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Proxy-runner Role for the HTTP transports (sse, streamable-http).
2+
# The proxy reaches such an MCP server over the network, so it is not granted
3+
# pods/attach. Keep in step with defaultRBACRules in
4+
# cmd/thv-operator/controllers/mcpserver_controller.go; chainsaw matches the
5+
# rules positionally, so order matters.
6+
apiVersion: rbac.authorization.k8s.io/v1
7+
kind: Role
8+
metadata:
9+
name: (join('-', [$testPrefix, 'proxy-runner']))
10+
namespace: toolhive-system
11+
rules:
12+
- apiGroups:
13+
- apps
14+
resources:
15+
- statefulsets
16+
verbs:
17+
- get
18+
- list
19+
- watch
20+
- create
21+
- update
22+
- patch
23+
- delete
24+
- apiGroups:
25+
- ""
26+
resources:
27+
- services
28+
verbs:
29+
- get
30+
- list
31+
- watch
32+
- create
33+
- update
34+
- patch
35+
- delete
36+
- apiGroups:
37+
- ""
38+
resources:
39+
- pods
40+
verbs:
41+
- get
42+
- list
43+
- watch
44+
- apiGroups:
45+
- ""
46+
resources:
47+
- pods/log
48+
verbs:
49+
- get
50+
- apiGroups:
51+
- ""
52+
resources:
53+
- configmaps
54+
verbs:
55+
- get
56+
- list
57+
- watch

0 commit comments

Comments
 (0)