Add compatiblity for "source_namespaces" introduced in ArgoCD v2.5.0

[karlschriek]

ArgoCD 2.5.0 has just been released.

One of the features included therein, is the ability to deploy ArgoCD Applications to namespaces outside of the controller's own namespace (which is usually argocd). In order to prevent permission escalation, a new field "source_namespaces" was added to the AppProject resource, which constrains namespaces within which Applications for that AppProject can be deployed and still be picked up by the conroller.

A full discussion on the feature can be found here: argoproj/argo-cd#9755

This PR updates the schemas and other relevant specs by adding support for "source_namespaces".

It also bumps up the argocd-cd Go package to:

github.com/argoproj/argo-cd/v2 v2.5.0

In order to accomodate this, I had to also manually bump the pinned k8s.io dependencies, i.e. k8s.io/api => k8s.io/api v0.23.1 becomes k8s.io/api => k8s.io/api v0.24.2 and so forth.

An issue remains though, as there seems to be some kind of incompatibility with github.com/argoproj/[email protected], which is still the latest version.

go build will cause:

# github.com/argoproj/gitops-engine/pkg/utils/kube
../../go/pkg/mod/github.com/argoproj/[email protected]/pkg/utils/kube/resource_ops.go:270:38: not enough arguments in call to k.fact.Validator
        have (bool)
        want (string, *"k8s.io/cli-runtime/pkg/resource".QueryParamVerifier)
../../go/pkg/mod/github.com/argoproj/[email protected]/pkg/utils/kube/resource_ops.go:278:30: undefined: resource.NewDryRunVerifier
../../go/pkg/mod/github.com/argoproj/[email protected]/pkg/utils/kube/resource_ops.go:326:30: undefined: resource.NewDryRunVerifier
../../go/pkg/mod/github.com/argoproj/[email protected]/pkg/utils/kube/resource_ops.go:375:30: undefined: resource.NewDryRunVerifier

I do not have a good enough understanding of the underlying dependencies to figure out where exactly this problem originates and how to fix or work around it. I am therefore putting this PR up so long in the hope that someone with more context would be able to quickly identify how to deal with this.

[karlschriek]

So I just came across this issue: argoproj/gitops-engine#447, and proceeded to pin the gitops-engine version as follows:

github.com/argoproj/gitops-engine => github.com/argoproj/gitops-engine v0.7.1-0.20220804190909-2bc3fef13e07

With this it compiles correctly. Removing WIP tag.

EDIT: No, keeping WIP for now, hasn't been tested properly yet of course!

[karlschriek]

Ok, I've now added source_namespaces entries to the acceptance tests. Updated scripts/testacc_prepare_env.sh to set up a test environment that uses ArgoCD v2.5.0. Also made some small updates in the readme so that the images and tags line up exactly with what is in scripts/testacc_prepare_env.sh.

Tests all run through for me locally.

[karlschriek]

I've run into an issue in live testing which isn't picked up by the acceptance tests. To read an application it needs to now be referenced as [namespace]/[application], unless the application exists in the root namespace (e.g. argocd), in which case it can be referenced just as [application].

In other words, queries such as this:

	app, err := c.Get(ctx, &applicationClient.ApplicationQuery{
		Name: &objectMeta.Name
	})

will have to be updated to:

	app, err := c.Get(ctx, &applicationClient.ApplicationQuery{
		Name: &objectMeta.Name,
		AppNamespace: &objectMeta.Namespace,
	})

I didn't build any acceptance tests that specifically deploy to other namespaces. The defaults ones still passed because they all deploy to the root namespace and can therefore just use the shorthand [application] instead of [namespace]/[application]. I'll see if I can update all these queries and also add some more acceptance tests. I'll need to alter the dev environment as well though, since server and application-controller need to be deployed with specific flags in order to enable this feature.

[karlschriek]

I accidentally added source_namespaces to schemas V0 and V1. Have removed it now.

[karlschriek]

Ok, I've now dealt with the fact that Applications need to be referenced as [namespace]/[name]. Have also written acceptance tests that will deploy Applications to mynamespace-1 instead of to argocd. All the acceptance tests in argocd/resource_argocd_application_test.go now deploy Applications to the test project (previously no project was defined, i.e. default was used). More on why I did this below.

To support this I also updated the test environment, as follows:

./manifests/install

• Patched the ConfigMap argocd-cmd-params-cm with the entry application.namespaces: mynamespace-*, which will start up argocd-server and argocd-application-controller in a mode where (in addition to the argocd namespace) they will observe any namespaces that match the pattern mynamespace-*
• Added a ClusterRoleBinding that allows argocd-server to create Application resources in any namespace

./manifests/testdata

• Created the namespace mynamespace-1, so that we can deploy Applications there when we run the tests
• Created an AppProject named test and set its sourceNamespaces to *. The default AppProject is scoped to the argocd namespace only. One could attempt to just overwrite the default project instead of creating a new one, but this is probably not a good idea. The default project is not created declaratively; it is created by argocd when it starts up.

I've also updated the GitHub Actions pipeline to use ArgoCD 2.5.0. Not sure if I am supposed to leave the old ones in though? It fails for some reason though; I am not sure why. It seems to not be able to resolve the argocd Go package.

Local tests run through. My live testing also hasn't revealed any further issues.

[karlschriek]

I have now bumped the Go version to 1.19. This resolves the issue with not being able to find the the right go package. Acceptance tests now actually run, but there are some failures. I don't really get why the "argocd_cluster" tests are failing, unless there is something fundamentally different in 2.5.0 there that I haven't accounted for...

EDIT: just to be sure I've redone my local testing env and run acceptance tests again. Locally everything runs through.

[karlschriek]

Have now added back the tests on the older ArgoCD versions. I am pretty sure I am missing something here. How do I set it up so that certain tests only run for certain versions?

And am I actually supposed to create a new applicationSpecSchema (v3), since the spec has now has an additional field?

[onematchfox]

Hi @karlschriek,

Thanks for taking this on. I had hoped to get to looking at this in more detail today but just didn't get the chance. However, I see you have already picked up on the first issue - upgrading the go version - as well as the second, ensuring compatibility with older versions of ArgoCD as per the Compatability promise of the provider. I'd recommend searching for featureProjectScopedClusters within the repository to get a feel of the scope of the change required to do this. On a similar vein, please ensure you add tests for this new attribute (again looking at featureProjectScopedClusters should provide you a good starting point).

[onematchfox]

Just seen that I missed a bunch of comments and additional work since I starting writing that comment - which now isn't necessarily valid. Hopefully my reference to featureProjectScopedClusters points you in the right direction. Otherwise, rest assured this is on my radar and I'll try take a closer look again as soon as I have the chance.

[karlschriek]

@onematchfox yeah, the "argocd_cluster" cluster test problem seems to have gone away. There are other things failing though, in particular on the older versions, although 2.5.0 also fails (although in the latest run it fails on something to do with TLS, for which there is a comment that this doesn't always work).

I think it is almost there now, but two things in particular I am having a hard time figuring out and it might actually be more productive if someone closer to the project takes a look at them:

1. If and how to create a new version for the AppProject schema. Right now I've edited the V2 schema by adding "source_namespaces" to it, but I presume what should have happened instead is that a V3 along with an "upgrade" function gets created.
2. Certain tests will only work on v2.5.0+, but I haven't been able to figure out how to not have them used in earlier versions.

I've also just removed the source_namespaces entries again from all the AppProject tests. I had initially added it to all of them, but have now instead created just one dedicated entry. This revealed another problem, which is that I should have set source_namespaces up to be an optional parameter.

Anyway, I am going to leave it there for now.

[karlschriek]

@onematchfox have you had a chance to look at this yet?

[onematchfox]

@onematchfox have you had a chance to look at this yet?

HI @karlschriek,

Apologies for the delay. I have been trying to look at this when/where possible but between travelling and falling ill I'm afraid I haven't had much time to dedicate to it.

1. If and how to create a new version for the AppProject schema. Right now I've edited the V2 schema by adding "source_namespaces" to it, but I presume what should have happened instead is that a V3 along with an "upgrade" function gets created.
2. Certain tests will only work on v2.5.0+, but I haven't been able to figure out how to not have them used in earlier versions.

I've also just removed the source_namespaces entries again from all the AppProject tests. I had initially added it to all of them, but have now instead created just one dedicated entry. This revealed another problem, which is that I should have set source_namespaces up to be an optional parameter.

I've pushed a commit that fixes up some of the changes to the project schema although this is by no means complete yet. To specifically address your points.

1. There is no need to creation a new schema version since, as you later point out, this needs to be an optional field.
2. I've added the required code (to the project tests at least) that does the required validation of the ArgoCD version to determine whether or not to source namespaces is supported.

Outstanding from my side at the moment is wrapping my head around the rest of the changes. Namely:

• updates to the application resource and associated tests - will need similar treatment to the project resource/tests
• the changes you've made to the setup script including:
  • the addition of testdata_v2.5.0+/token_resource.yml) - my gut feel is that it is going to be better to inline the test project in the application tests
  • the updates to the kustomize overlays
• addressing potential issues with the changes I made to the project resource to handle defaults for source namespaces when running against older ArgoCD versions

[onematchfox]

@karlschriek Think I finally managed to get everything working here.

@oboukili, do you want to take a look at this to double-check I haven't missed anything? One thing worth noting is that I had to adjust the structure of the internal ID for the argocd_application resource to store both the name and namespace in order to support importing resources that have been created in custom/non-default namespaces.

[onematchfox]

@oboukili Do you want to double-check this one or can I go ahead and merge?

[oboukili]

@onematchfox big thanks for handling this PR! @karlschriek we're getting there!