Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Server side diff now works correctly with some fields removal #640

Conversation

andrii-korotkov-verkada
Copy link
Contributor

@andrii-korotkov-verkada andrii-korotkov-verkada commented Nov 18, 2024

Helps with argoproj/argo-cd#20792

Removed and modified sets may only contain the fields that changed, not including key fields like "name". This can cause merge to fail, since it expects those fields to be present if they are present in the predicted live. Fortunately, structured merge diff folks have added an option to extracting items to add key fields, and we are going to use that.

Failure of the new test before the fix

            	Error:      	Received unexpected error:
            	            	error removing non config mutations for resource Deployment/nginx-deployment: error reverting webhook removed fields in predicted live resource: .spec.template.spec.containers: element 0: associative list with keys has an element that omits key field "name" (and doesn't have default value)
            	Test:       	TestServerSideDiff/will_test_removing_some_field_with_undoing_changes_done_by_webhook

Copy link
Member

@crenshaw-dev crenshaw-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you open an Argo CD PR to test this? Is there a way to test the higher-level bug in Argo CD so we have coverage of both the gitops-engine behavior and of the user-facing bug?

pkg/diff/diff_test.go Outdated Show resolved Hide resolved
pkg/diff/diff.go Outdated Show resolved Hide resolved
pkg/diff/diff.go Outdated Show resolved Hide resolved
pkg/diff/diff.go Outdated Show resolved Hide resolved
pkg/diff/diff.go Show resolved Hide resolved
@andrii-korotkov-verkada
Copy link
Contributor Author

Can you open an Argo CD PR to test this? Is there a way to test the higher-level bug in Argo CD so we have coverage of both the gitops-engine behavior and of the user-facing bug?

I'll do the testing.

@andrii-korotkov-verkada andrii-korotkov-verkada force-pushed the 20792-fix-server-side-diff-with-fields-removal branch from f171197 to d6cc3eb Compare November 19, 2024 17:50
@andrii-korotkov-verkada
Copy link
Contributor Author

ArgoCD PR argoproj/argo-cd#20842, tests passed.

Copy link
Contributor

@leoluz leoluz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check my comment.

pkg/diff/diff.go Outdated Show resolved Hide resolved
pkg/diff/testdata/smd-deploy-live.yaml Outdated Show resolved Hide resolved
pkg/diff/testdata/smd-deploy-live.yaml Outdated Show resolved Hide resolved
@andrii-korotkov-verkada andrii-korotkov-verkada force-pushed the 20792-fix-server-side-diff-with-fields-removal branch from d6cc3eb to d4b29ea Compare November 19, 2024 21:31
@andrii-korotkov-verkada
Copy link
Contributor Author

@crenshaw-dev, I've addressed your comments.

Code Analysis complains that configs are not perfect, but that's okay since they aren't deployed anywhere.

@crenshaw-dev
Copy link
Member

@andrii-korotkov-verkada wanna try adding this to a new sonar-project.properties file in the root of the repo?

# Exclude test manifests from analysis
sonar.kubernetes.exclusions=pkg/diff/testdata/**

@andrii-korotkov-verkada
Copy link
Contributor Author

Per discussion with @leoluz, I'd spend more time to understand why the Structured Merge Diff library's Merge doesn't derive key fields from paths, since the info is available. I may submit a ticket to their community with a request to update the library.

@andrii-korotkov-verkada
Copy link
Contributor Author

I've filed kubernetes-sigs/structured-merge-diff#273

@andrii-korotkov-verkada andrii-korotkov-verkada force-pushed the 20792-fix-server-side-diff-with-fields-removal branch from d4b29ea to e881697 Compare November 22, 2024 22:23
pkg/diff/testdata/smd-deploy2-config.yaml Fixed Show fixed Hide fixed
pkg/diff/testdata/smd-deploy2-config.yaml Fixed Show fixed Hide fixed
pkg/diff/testdata/smd-deploy2-config.yaml Fixed Show fixed Hide fixed
pkg/diff/testdata/smd-deploy2-live.yaml Fixed Show fixed Hide fixed
pkg/diff/testdata/smd-deploy2-live.yaml Fixed Show fixed Hide fixed
pkg/diff/testdata/smd-deploy2-live.yaml Fixed Show fixed Hide fixed
@leoluz
Copy link
Contributor

leoluz commented Dec 11, 2024

Structured-merge-diff PR with the fix is now merged:
kubernetes-sigs/structured-merge-diff#275

Helps with argoproj/argo-cd#20792

Removed and modified sets may only contain the fields that changed, not including key fields like "name". This can cause merge to fail, since it expects those fields to be present if they are present in the predicted live.
Fortunately, we can inspect the set and derive the key fields necessary. Then they can be added to the set and used during a merge.
Also, have a new test which fails before the fix, but passes now.

Failure of the new test before the fix
```
            	Error:      	Received unexpected error:
            	            	error removing non config mutations for resource Deployment/nginx-deployment: error reverting webhook removed fields in predicted live resource: .spec.template.spec.containers: element 0: associative list with keys has an element that omits key field "name" (and doesn't have default value)
            	Test:       	TestServerSideDiff/will_test_removing_some_field_with_undoing_changes_done_by_webhook
```

Signed-off-by: Andrii Korotkov <[email protected]>
Signed-off-by: Andrii Korotkov <[email protected]>
@andrii-korotkov-verkada andrii-korotkov-verkada force-pushed the 20792-fix-server-side-diff-with-fields-removal branch from 4add644 to 91c72b1 Compare December 11, 2024 20:13
@@ -0,0 +1,2 @@
# Exclude test manifests from analysis
sonar.kubernetes.exclusions=pkg/diff/testdata/**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem to be working

Copy link
Contributor

@leoluz leoluz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: Andrii Korotkov <[email protected]>
@leoluz leoluz dismissed crenshaw-dev’s stale review December 11, 2024 20:27

Confirmed with Michael over Slack

@leoluz leoluz merged commit 8849c3f into argoproj:master Dec 11, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants