OCPBUGS-61088: create networkpolicy settings for in-cluster monitoring #2656

juzhao · 2025-09-01T08:38:21Z

see https://issues.redhat.com/browse/OCPBUGS-61088, use this bug to set networkpolicy for in-cluster monitoring
this PR replaced #2645, 2645 does not include networkpolicy settings for alertmanager/prometheus/thanos-querier, since there is default deny network settings for all monitoring pods, if we don't include them, CI jobs would be failed for them

juzhao · 2025-09-01T08:43:14Z

/retitle [WIP] OCPBUGS-61088: create networkpolicy settings for in-cluster monitoring

openshift-ci-robot · 2025-09-01T08:43:23Z

@juzhao: This pull request references Jira Issue OCPBUGS-61088, which is invalid:

expected the bug to target the "4.20.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

see https://issues.redhat.com/browse/OCPBUGS-61088, use this bug to set networkpolicy for in-cluster monitoring
this PR replaced #2645, 2645 does not include networkpolicy settings for alertmanager/prometheus/thanos-querier, since there is default deny network settings for all monitoring pods, if we don't include them, CI jobs would be failed for them

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

juzhao · 2025-09-01T08:44:25Z

/jira refresh

openshift-ci-robot · 2025-09-01T08:44:32Z

@juzhao: This pull request references Jira Issue OCPBUGS-61088, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.20.0) matches configured target version for branch (4.20.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @juzhao

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2025-09-01T08:44:35Z

@openshift-ci-robot: GitHub didn't allow me to request PR reviews from the following users: juzhao.

Note that only openshift members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

@juzhao: This pull request references Jira Issue OCPBUGS-61088, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)

bug target version (4.20.0) matches configured target version for branch (4.20.0)

bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @juzhao

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

jsonnet/components/admission-webhook.libsonnet

simonpasquier · 2025-09-01T15:28:45Z

jsonnet/components/kube-state-metrics.libsonnet

+            ],
+          },
+        ],
+        egress: [


Is there any recommendation to allow egress traffic only to the API service? This is the only destination which ksm should try to connect but I guess that it's not trivial to accommodate classic OCP (self-managed) and hosted clusters (HyperShift).

networkpolicy applied to pod only, seems we can not define for API service, will check again.

we can use pod/namespace selectors in rgress rules https://kubernetes.io/docs/concepts/services-networking/network-policies/#behavior-of-to-and-from-selectors
but yes, it could be tricky to have it right for all the flavors.

maybe we can create a ticket for this, and try to narrow down egress for ksm and others in a follow up PR, with some manual/payload testing we should be able to get it right. (maybe in 4.21)

jsonnet/components/prometheus-operator.libsonnet

juzhao · 2025-09-02T08:14:38Z

/retest

juzhao · 2025-09-03T12:51:10Z

/test verify
/test rules
/test generate

juzhao · 2025-09-05T08:56:28Z

/retest

juzhao · 2025-09-23T02:00:01Z

/retest

juzhao · 2025-09-24T01:54:37Z

/retest

juzhao · 2025-09-24T06:23:44Z

/retest

danielmellado · 2025-09-24T06:49:08Z

go-proxy CI issues are unrelated to the patch

juzhao · 2025-09-24T08:06:57Z

e2e CI error is

: Run multi-stage test e2e-agnostic-operator expand_less 	14s
{  failed to acquire lease for "aws-2-quota-slice": status 503 Service Unavailable, status code 503}

seems prow issue

juzhao · 2025-09-25T01:59:16Z

/retest

juzhao · 2025-09-25T06:19:21Z

/retest-required

juzhao · 2025-09-25T06:29:00Z

e2e-aws-ovn-techpreview failed

: [sig-instrumentation][Late] Alerts shouldn't exceed the series limit of total series sent via telemetry from each cluster [Suite:openshift/conformance/parallel]
    [
        <*errors.errorString | 0xc0074b8030>{
            s: "promQL query returned unexpected results:\navg_over_time(cluster:telemetry_selected_series:count[1h37m56s]) >= 780\n[\n  {\n    \"metric\": {\n      \"prometheus\": \"openshift-monitoring/k8s\"\n    },\n    \"value\": [\n      1758776352.201,\n      \"788.4744897959183\"\n    ]\n  }\n]",
        },
    ]

same issue as #2681 (comment)

e2e-agnostic-operator
failed at image_registry_test.go, PR OCPBUGS-62109: test: remove image registry e2e tests #2681 will remove the tests

    image_registry_test.go:52: 
        	Error Trace:	/go/src/github.com/openshift/cluster-monitoring-operator/test/e2e/image_registry_test.go:52
        	            				/go/src/github.com/openshift/cluster-monitoring-operator/test/e2e/image_registry_test.go:18
        	Error:      	Not equal: 
        	            	expected: "registry.build05.ci.openshift.org"
        	            	actual  : "quay-proxy.ci.openshift.org"

juzhao · 2025-09-25T06:36:42Z

tested with PR, no regression issues.
/verified by @juzhao

openshift-ci-robot · 2025-09-25T06:36:53Z

@juzhao: This PR has been marked as verified by @juzhao.

In response to this:

tested with PR, no regression issues.
/verified by @juzhao

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

juzhao · 2025-09-25T08:00:59Z

/test e2e-aws-ovn-techpreview

juzhao · 2025-09-25T13:35:33Z

/test e2e-agnostic-operator

juzhao · 2025-09-26T01:31:14Z

/test e2e-agnostic-operator

juzhao · 2025-09-26T05:32:58Z

/retest-required

pkg/tasks/alertmanager.go

jsonnet/components/alertmanager.libsonnet

…umber

juzhao · 2025-10-11T06:36:54Z

/retest

openshift-ci · 2025-10-11T10:21:07Z

@juzhao: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/versions	`a6ee40b`	link	false	`/test versions`
ci/prow/okd-scos-e2e-aws-ovn	`a6ee40b`	link	false	`/test okd-scos-e2e-aws-ovn`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

juzhao added 2 commits September 1, 2025 16:22

add networkpolicy settings for in-cluster monitoring

de2f821

update thanos-querier.libsonnet for networkpolicy

91b04be

openshift-ci bot requested review from danielmellado and rexagod September 1, 2025 08:39

juzhao mentioned this pull request Sep 1, 2025

[wip] MON-4297 add networkpolicy for CMO/KSM/metrics-server/monitoring-plugin/OSM/prometheus-operator/telemeter-client #2645

Closed

openshift-ci bot changed the title ~~add networkpolicy settings for in-cluster monitoring~~ [WIP] OCPBUGS-61088: create networkpolicy settings for in-cluster monitoring Sep 1, 2025

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Sep 1, 2025

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 1, 2025

openshift-ci-robot added the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Sep 1, 2025

openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Sep 1, 2025

update admission-webhook.libsonnet

d39c22f

simonpasquier reviewed Sep 1, 2025

View reviewed changes

juzhao added 4 commits September 2, 2025 11:08

fix typo in admission-webhook.libsonnet prometheus-operator.libsonnet

d029248

update monitoring-plugin.libsonnet and gofmt manifests.go

506f269

update format

10baeed

update alertmanager.libsonnet

d881999

update manifests.go to add ThanosQuerierNetworkPolicy func

0da4120

remove networkpolicy.yaml file under each component

02c48aa

juzhao added 3 commits September 5, 2025 22:54

add networkpolicy files back

7f5102e

update main.jsonnet to remove removeNetworkPolicy

d34d3d5

fix error

aa26831

change deploy networkpolicy task to beginning of each task

63867a8

update prometheus 10901 port from UDP to TCP

1fa06d0

openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Sep 25, 2025

machine424 reviewed Sep 26, 2025

View reviewed changes

pkg/tasks/alertmanager.go Outdated Show resolved Hide resolved

machine424 reviewed Sep 26, 2025

View reviewed changes

pkg/tasks/alertmanager.go Outdated Show resolved Hide resolved

machine424 reviewed Sep 26, 2025

View reviewed changes

jsonnet/components/alertmanager.libsonnet Outdated Show resolved Hide resolved

update code and add e2e tests for alertmanager

010c7e9

openshift-ci-robot removed the verified Signifies that the PR passed pre-merge verification criteria label Sep 28, 2025

juzhao added 6 commits September 28, 2025 22:01

fix error

910c61b

add e2e cases and update libsonnet and yaml files to reference port n…

facd3a4

…umber

remove annotations from libsonnet and yaml files

a473694

remove comments

8c65a39

fix error

2ec8ce5

remvove egress for admission-webhook

a6ee40b

OCPBUGS-61088: create networkpolicy settings for in-cluster monitoring #2656

Are you sure you want to change the base?

OCPBUGS-61088: create networkpolicy settings for in-cluster monitoring #2656

Uh oh!

Conversation

juzhao commented Sep 1, 2025

Uh oh!

juzhao commented Sep 1, 2025

Uh oh!

openshift-ci-robot commented Sep 1, 2025

Uh oh!

juzhao commented Sep 1, 2025

Uh oh!

openshift-ci-robot commented Sep 1, 2025

Uh oh!

openshift-ci bot commented Sep 1, 2025

Uh oh!

Uh oh!

Uh oh!

simonpasquier Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

juzhao Sep 2, 2025

Choose a reason for hiding this comment

Uh oh!

machine424 Sep 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

juzhao commented Sep 2, 2025

Uh oh!

juzhao commented Sep 3, 2025

Uh oh!

juzhao commented Sep 5, 2025

Uh oh!

juzhao commented Sep 23, 2025

Uh oh!

juzhao commented Sep 24, 2025

Uh oh!

juzhao commented Sep 24, 2025

Uh oh!

danielmellado commented Sep 24, 2025

Uh oh!

juzhao commented Sep 24, 2025

Uh oh!

juzhao commented Sep 25, 2025

Uh oh!

juzhao commented Sep 25, 2025

Uh oh!

juzhao commented Sep 25, 2025

Uh oh!

juzhao commented Sep 25, 2025

Uh oh!

openshift-ci-robot commented Sep 25, 2025

Uh oh!

juzhao commented Sep 25, 2025

Uh oh!

juzhao commented Sep 25, 2025

Uh oh!

juzhao commented Sep 26, 2025

Uh oh!

juzhao commented Sep 26, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

juzhao commented Oct 11, 2025

Uh oh!

openshift-ci bot commented Oct 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants