Skip to content
This repository was archived by the owner on Dec 29, 2021. It is now read-only.

Commit 8e7b2de

Browse files
scholzjscholzj
committed
Add advanced example
Signed-off-by: Jakub Scholz <[email protected]>
1 parent f695b6b commit 8e7b2de

6 files changed

+405
-32
lines changed

‎README.md

+39-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Demo: Strimzi with Open Policy Agent used for Kafka authorization
1+
# Strimzi with Open Policy Agent used for Kafka authorization
22

33
This repository contains the example files for using Open Policy Agent (OPA) for Apache Kafka authorization.
44
This demo is related to the blog post published on [Strimzi website](https://strimzi.io).
@@ -34,6 +34,9 @@ kubectl apply -f opa-deployment.yaml
3434

3535
## Basic example
3636

37+
The basic example had the groups hardcoded inside the OPA policy.
38+
Any changes to the groups (adding or removing users) would require change fo the policy.
39+
3740
### Deploy Kafka cluster
3841

3942
Deploy the Kafka cluster from the [`basic-example-kafka.yaml`](./basic-example-kafka.yaml) file.
@@ -61,4 +64,39 @@ In the file [`basic-example-clients-denied.yaml`](./basic-example-clients-denied
6164
kubectl apply -f ./basic-example-clients-denied.yaml
6265
```
6366

67+
When you deploy them, you should see that the are allowed to use the Kafka cluster.
68+
69+
## Advanced example
70+
71+
The advanced example is using groups configured as annotations on the `KafkaTopic` and `KafkaUser` resources.
72+
The resources are loaded into OPA using the kube-mgmt sidecar and are used by the policy.
73+
That way, changing the rights, adding or removing users etc. can be done without any changes to the policy.
74+
75+
### Deploy Kafka cluster
76+
77+
Deploy the Kafka cluster from the [`advanced-example-kafka.yaml`](./advanced-example-kafka.yaml) file.
78+
This example is also configured to use the basic example policy.
79+
80+
```
81+
kubectl apply -f advanced-example-kafka.yaml
82+
```
83+
84+
### Deploy allowed clients
85+
86+
In the file [`advanced-example-clients-allowed.yaml`](./advanced-example-clients-allowed.yaml) you can find example consumer and producer which are using users allowed to produce and consumer messages.
87+
88+
```
89+
kubectl apply -f ./advanced-example-clients-allowed.yaml
90+
```
91+
92+
When you deploy them, you should see that the are allowed to run.
93+
94+
### Deploy not allowed clients
95+
96+
In the file [`advanced-example-clients-denied.yaml`](./advanced-example-clients-denied.yaml) you can find example consumer and producer which are using users not allowed to produce and consumer messages.
97+
98+
```
99+
kubectl apply -f ./advanced-example-clients-denied.yaml
100+
```
101+
64102
When you deploy them, you should see that the are allowed to use the Kafka cluster.

‎advanced-example-clients-allowed.yaml

+128
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
apiVersion: kafka.strimzi.io/v1alpha1
2+
kind: KafkaTopic
3+
metadata:
4+
name: my-topic
5+
annotations:
6+
consumer-groups: '["consumers1", "consumers2"]'
7+
producer-groups: '["producers1", "producers2"]'
8+
labels:
9+
strimzi.io/cluster: my-cluster
10+
spec:
11+
replicas: 3
12+
partitions: 12
13+
---
14+
apiVersion: kafka.strimzi.io/v1beta1
15+
kind: KafkaUser
16+
metadata:
17+
name: john
18+
annotations:
19+
groups: '["producers1"]'
20+
labels:
21+
strimzi.io/cluster: my-cluster
22+
spec:
23+
authentication:
24+
type: tls
25+
---
26+
apiVersion: apps/v1
27+
kind: Deployment
28+
metadata:
29+
labels:
30+
app: johns-producer
31+
name: johns-producer
32+
spec:
33+
replicas: 1
34+
selector:
35+
matchLabels:
36+
app: johns-producer
37+
template:
38+
metadata:
39+
labels:
40+
app: johns-producer
41+
spec:
42+
containers:
43+
- name: johns-producer
44+
image: strimzi/hello-world-producer:latest
45+
env:
46+
- name: CA_CRT
47+
valueFrom:
48+
secretKeyRef:
49+
name: my-cluster-cluster-ca-cert
50+
key: ca.crt
51+
- name: USER_CRT
52+
valueFrom:
53+
secretKeyRef:
54+
name: john
55+
key: user.crt
56+
- name: USER_KEY
57+
valueFrom:
58+
secretKeyRef:
59+
name: john
60+
key: user.key
61+
- name: BOOTSTRAP_SERVERS
62+
value: my-cluster-kafka-bootstrap:9093
63+
- name: TOPIC
64+
value: my-topic
65+
- name: DELAY_MS
66+
value: "1000"
67+
- name: LOG_LEVEL
68+
value: "INFO"
69+
- name: MESSAGE_COUNT
70+
value: "1000000"
71+
---
72+
apiVersion: kafka.strimzi.io/v1beta1
73+
kind: KafkaUser
74+
metadata:
75+
name: matt
76+
annotations:
77+
groups: '["consumers2"]'
78+
labels:
79+
strimzi.io/cluster: my-cluster
80+
spec:
81+
authentication:
82+
type: tls
83+
---
84+
apiVersion: apps/v1
85+
kind: Deployment
86+
metadata:
87+
labels:
88+
app: matts-consumer
89+
name: matts-consumer
90+
spec:
91+
replicas: 1
92+
selector:
93+
matchLabels:
94+
app: matts-consumer
95+
template:
96+
metadata:
97+
labels:
98+
app: matts-consumer
99+
spec:
100+
containers:
101+
- name: matts-consumer
102+
image: strimzi/hello-world-consumer:latest
103+
env:
104+
- name: CA_CRT
105+
valueFrom:
106+
secretKeyRef:
107+
name: my-cluster-cluster-ca-cert
108+
key: ca.crt
109+
- name: USER_CRT
110+
valueFrom:
111+
secretKeyRef:
112+
name: matt
113+
key: user.crt
114+
- name: USER_KEY
115+
valueFrom:
116+
secretKeyRef:
117+
name: matt
118+
key: user.key
119+
- name: BOOTSTRAP_SERVERS
120+
value: my-cluster-kafka-bootstrap:9093
121+
- name: TOPIC
122+
value: my-topic
123+
- name: GROUP_ID
124+
value: matts-group
125+
- name: LOG_LEVEL
126+
value: "INFO"
127+
- name: MESSAGE_COUNT
128+
value: "1000000"

‎advanced-example-clients-denied.yaml

+128
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
apiVersion: kafka.strimzi.io/v1alpha1
2+
kind: KafkaTopic
3+
metadata:
4+
name: my-topic
5+
annotations:
6+
consumer-groups: '["consumers1", "consumers2"]'
7+
producer-groups: '["producers1", "producers2"]'
8+
labels:
9+
strimzi.io/cluster: my-cluster
10+
spec:
11+
replicas: 3
12+
partitions: 12
13+
---
14+
apiVersion: kafka.strimzi.io/v1beta1
15+
kind: KafkaUser
16+
metadata:
17+
name: joe
18+
annotations:
19+
groups: '["other-producers"]'
20+
labels:
21+
strimzi.io/cluster: my-cluster
22+
spec:
23+
authentication:
24+
type: tls
25+
---
26+
apiVersion: apps/v1
27+
kind: Deployment
28+
metadata:
29+
labels:
30+
app: joes-producer
31+
name: joes-producer
32+
spec:
33+
replicas: 1
34+
selector:
35+
matchLabels:
36+
app: joes-producer
37+
template:
38+
metadata:
39+
labels:
40+
app: joes-producer
41+
spec:
42+
containers:
43+
- name: joes-producer
44+
image: strimzi/hello-world-producer:latest
45+
env:
46+
- name: CA_CRT
47+
valueFrom:
48+
secretKeyRef:
49+
name: my-cluster-cluster-ca-cert
50+
key: ca.crt
51+
- name: USER_CRT
52+
valueFrom:
53+
secretKeyRef:
54+
name: joe
55+
key: user.crt
56+
- name: USER_KEY
57+
valueFrom:
58+
secretKeyRef:
59+
name: joe
60+
key: user.key
61+
- name: BOOTSTRAP_SERVERS
62+
value: my-cluster-kafka-bootstrap:9093
63+
- name: TOPIC
64+
value: my-topic
65+
- name: DELAY_MS
66+
value: "1000"
67+
- name: LOG_LEVEL
68+
value: "INFO"
69+
- name: MESSAGE_COUNT
70+
value: "1000000"
71+
---
72+
apiVersion: kafka.strimzi.io/v1beta1
73+
kind: KafkaUser
74+
metadata:
75+
name: conor
76+
annotations:
77+
groups: '["other-consumers"]'
78+
labels:
79+
strimzi.io/cluster: my-cluster
80+
spec:
81+
authentication:
82+
type: tls
83+
---
84+
apiVersion: apps/v1
85+
kind: Deployment
86+
metadata:
87+
labels:
88+
app: conors-consumer
89+
name: conors-consumer
90+
spec:
91+
replicas: 1
92+
selector:
93+
matchLabels:
94+
app: conors-consumer
95+
template:
96+
metadata:
97+
labels:
98+
app: conors-consumer
99+
spec:
100+
containers:
101+
- name: conors-consumer
102+
image: strimzi/hello-world-consumer:latest
103+
env:
104+
- name: CA_CRT
105+
valueFrom:
106+
secretKeyRef:
107+
name: my-cluster-cluster-ca-cert
108+
key: ca.crt
109+
- name: USER_CRT
110+
valueFrom:
111+
secretKeyRef:
112+
name: conor
113+
key: user.crt
114+
- name: USER_KEY
115+
valueFrom:
116+
secretKeyRef:
117+
name: conor
118+
key: user.key
119+
- name: BOOTSTRAP_SERVERS
120+
value: my-cluster-kafka-bootstrap:9093
121+
- name: TOPIC
122+
value: my-topic
123+
- name: GROUP_ID
124+
value: conors-group
125+
- name: LOG_LEVEL
126+
value: "INFO"
127+
- name: MESSAGE_COUNT
128+
value: "1000000"

‎advanced-example-kafka.yaml

+68
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
apiVersion: kafka.strimzi.io/v1beta1
2+
kind: Kafka
3+
metadata:
4+
name: my-cluster
5+
labels:
6+
app: my-cluster
7+
spec:
8+
kafka:
9+
replicas: 3
10+
resources:
11+
requests:
12+
memory: 2Gi
13+
cpu: 500m
14+
limits:
15+
memory: 2Gi
16+
cpu: "1"
17+
jvmOptions:
18+
-Xms: 1024m
19+
-Xmx: 1024m
20+
listeners:
21+
tls:
22+
authentication:
23+
type: tls
24+
authorization:
25+
type: opa
26+
url: http://opa:8181/v1/data/kafka/authz/example/crds/allow
27+
expireAfterMs: 60000
28+
config:
29+
offsets.topic.replication.factor: 3
30+
transaction.state.log.replication.factor: 3
31+
transaction.state.log.min.isr: 2
32+
storage:
33+
type: jbod
34+
volumes:
35+
- id: 0
36+
type: persistent-claim
37+
size: 100Gi
38+
deleteClaim: false
39+
zookeeper:
40+
replicas: 3
41+
resources:
42+
requests:
43+
memory: 1Gi
44+
cpu: "0.3"
45+
limits:
46+
memory: 1Gi
47+
cpu: "0.5"
48+
storage:
49+
type: persistent-claim
50+
size: 100Gi
51+
deleteClaim: false
52+
entityOperator:
53+
topicOperator:
54+
resources:
55+
requests:
56+
memory: 256Mi
57+
cpu: "0.1"
58+
limits:
59+
memory: 256Mi
60+
cpu: "0.5"
61+
userOperator:
62+
resources:
63+
requests:
64+
memory: 256Mi
65+
cpu: "0.1"
66+
limits:
67+
memory: 256Mi
68+
cpu: "0.5"

0 commit comments

Comments
 (0)