Skip to content

fix: idempotent helm installations for demos/stacks and ignoring of existing jobs #386

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jun 10, 2025

Conversation

xeniape
Copy link
Member

@xeniape xeniape commented Jun 4, 2025

Description

Implementation for #312 and #333

Current procedure to upgrade a demo:

stackablectl demo install [demo] --release x.y
stackablectl release upgrade x.z
stackablectl demo install [demo] --release x.z

Example output of failed Job updates due to immutability:

$ target/debug/stackablectl demo install airflow-scheduled-job --release 25.3 
Deploying Job/start-pyspark-job manifest failed due to immutability
Deploying Job/start-date-job manifest failed due to immutability

Installed demo "airflow-scheduled-job"

Use "stackablectl operator installed" to display the installed operators.
Use "stackablectl stacklet list" to display the installed stacklets.

Problem encountered during development when running helm upgrade --install on opensearch-dashboards

An unrecoverable error occured: failed to execute stack (sub)command

Caused by these errors (recent errors listed first):
 1: failed to install stack "logging"
 2: failed to install stack manifests
 3: failed to install Helm release opensearch-dashboards
 4: failed to upgrade/install Helm release
 5: helm FFI library call failed (failed to replace object: Deployment.apps "opensearch-dashboards" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app.kubernetes.io/instance":"opensearch-dashboards", "app.kubernetes.io/name":"opensearch-dashboards"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable)

Thrown for opensearch-dashboards Helm Chart, because of opensearch-project/helm-charts#655
Should be working due to setting ChartSpec.Force = true in rust/helm-sys/go-helm-wrapper/main.go but does not.

Update/Solution: Due to changes in Helm 3 and how the --force flag works (helm/helm#7082 (comment)), @NickLarsenNZ and me decided to switch from the helm upgrade --install to the strategy of uninstalling the old Helm Chart and installing the new one instead for an upgrade.

Definition of Done Checklist

  • Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
  • Please make sure all these things are done and tick the boxes
# Author
- [ ] Changes are OpenShift compatible
- [ ] CRD changes approved
- [ ] Helm chart can be installed and deployed operator works
- [ ] Integration tests passed (for non trivial changes)
# Reviewer
- [ ] Code contains useful comments
- [ ] (Integration-)Test cases added
- [ ] Documentation added or updated
- [ ] Changelog updated
- [ ] Cargo.toml only contains references to git tags (not specific commits or branches)
# Acceptance
- [ ] Feature Tracker has been updated
- [ ] Proper release label has been added

Copy link
Member

@NickLarsenNZ NickLarsenNZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of suggestions

@xeniape xeniape changed the title Fix/demo installations fix: idempotent helm installations for demos/stacks and ignoring of existing jobs Jun 5, 2025
@xeniape xeniape marked this pull request as ready for review June 5, 2025 14:35
@xeniape xeniape self-assigned this Jun 5, 2025
@xeniape xeniape moved this to Development: Waiting for Review in Stackable Engineering Jun 5, 2025
@xeniape xeniape requested a review from NickLarsenNZ June 10, 2025 09:15
@NickLarsenNZ NickLarsenNZ moved this from Development: Waiting for Review to Development: In Review in Stackable Engineering Jun 10, 2025
Copy link
Member

@NickLarsenNZ NickLarsenNZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Would you mind updating the PR description and adding example output for the Job immutable fields? I was just curious what to expect.

@xeniape xeniape added this pull request to the merge queue Jun 10, 2025
Merged via the queue into main with commit 5c1e86f Jun 10, 2025
21 of 22 checks passed
@xeniape xeniape deleted the fix/demo-installations branch June 10, 2025 18:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Development: In Review
Development

Successfully merging this pull request may close these issues.

stackablectl demo install should optionally ignore existing Jobs fix: Idempotent helm installations
2 participants