Added Vault server integration for values and application specific properties

[Vineet0197]

This Pull Request introduces a new feature for integrating Helm with HashiCorp Vault. The feature allows users to securely retrieve secrets from Vault and use them in their Helm charts, specifically for values.yaml and application-specific properties files.

Changes:

• VaultGetter: Added a new VaultGetter struct to handle retrieving data from Vault.
• Flags: Introduced new flags to support Vault integration:
  • --token: The Vault token used for authentication.
  • --vault-address: The address of the Vault server.
  • --property-file: The path to the application-specific properties file, which can be a Vault path or a local file path.
• Methods: Implemented methods for initializing the Vault client and fetching secrets.
• Unit Tests: Added unit tests for the VaultGetter struct to cover various scenarios, including successful retrieval, no data found, and read errors.

Usage:

Users can specify Vault paths in their Helm charts using the vault:// prefix. The VaultGetter will fetch the secrets from Vault and make them available to the Helm templates. The new flags can be used as follows:

• --token: Specify the Vault token for authentication.
• --vault-address: Specify the Vault server address.
• --property-file: Specify the path to the application-specific properties file, which can be a Vault path or a local file path.

Examples:

Using Vault for values.yaml and properties file:

helm install mychart --values vault://secret/data/myapp/values --token my-vault-token --vault-address https://vault.example.com --property-file vault://secret/data/myapp/conf/application.properties

#### Using a local path for properties file:
helm install mychart --values vault://secret/data/myapp/values --token my-vault-token --vault-address https://vault.example.com --property-file application.properties

[gjenkins8]

Can we fixup/remove copyright notice changes please

[gjenkins8]

For vault integration specifically, it is hard reason that Helm should support specific solution providers like this. Helm authors can't practically support any/all solutions.

Another specific requirement for this change would be test coverage (up to and including integration test coverage IMHO: which calls a different question, as to how this environment would be provisioned, especially given Vault's licensing)

However and ideally, Helm would be able to delegate these situations to a plugin model. Which could then lean on the wider community to support. And not require such functionality to be built into core Helm.

Finally, why does Helm even need to have direct support here? A user could do something like:

helm install mychart --values <(vault-cli --token ${my-vault-token} --vault-address https://vault.example.com secret/data/myapp/values)

(where vault-cli is a tool which would fetch the secret in the same way that the code changes here implement)

[Vineet0197]

For vault integration specifically, it is hard reason that Helm should support specific solution providers like this. Helm authors can't practically support any/all solutions.

Another specific requirement for this change would be test coverage (up to and including integration test coverage IMHO: which calls a different question, as to how this environment would be provisioned, especially given Vault's licensing)

However and ideally, Helm would be able to delegate these situations to a plugin model. Which could then lean on the wider community to support. And not require such functionality to be built into core Helm.

Finally, why does Helm even need to have direct support here? A user could do something like:

helm install mychart --values <(vault-cli --token ${my-vault-token} --vault-address https://vault.example.com secret/data/myapp/values)

(where vault-cli is a tool which would fetch the secret in the same way that the code changes here implement)

Thank you for your detailed feedback. I understand the concern regarding Helm supporting specific solution providers and the challenges it brings in terms of maintenance and extensibility. Here's my perspective on why Vault integration directly within Helm can be a valuable addition:

1. User Convenience and Standardization: While users can utilize external tools like vault-cli for fetching values, this adds additional steps and dependencies to their workflows. Direct integration with Helm simplifies the user experience by providing a seamless, native approach to fetch values.yaml and application properties from Vault without relying on external tools, it reduces the cognitive and operational overhead, particularly for users who are less experienced with managing external CLI tools.
2. Helm as a Centralized Deployment Tool: Helm is often used as a single entry point for managing deployments in Kubernetes. By integrating Vault directly, Helm becomes a more comprehensive tool, covering secrets management as a first-class use case. This aligns with Helm's philosophy of making Kubernetes package management straightforward and efficient.
3. Test Coverage and Maintenance: I acknowledge the need for test coverage and have included extensive unit tests for the Vault integration. For integration tests, I propose using an open-source Vault server (in dev mode) during testing. This would ensure coverage without incurring licensing issues. Additionally, this feature has been implemented in a way that does not affect other parts of Helm, ensuring minimal maintenance overhead.
4. Plugin Model vs. Core Integration: While a plugin model could be an alternative, users often expect essential integrations like this to be natively available within Helm for reliability and consistency. Relying on a plugin model introduces fragmentation and could result in varying levels of quality, support, and user experience. Direct integration avoids these risks.
5. Precedent for Supporting Specific Providers: Helm already supports OCI (Open Container Initiative) registries directly. Similarly, Vault is a widely adopted and industry-standard tool for secrets management. Adding support for Vault aligns with Helm's precedent for integrating with widely used, foundational tools in the Kubernetes ecosystem.
6. Future Scalability: The implementation is designed to be modular and extensible. If the Helm maintainers wish to expand support for other secrets management tools in the future, the groundwork laid here can be adapted for those use cases as well.

By addressing these points, I hope to convey the broader value of this feature and its alignment with Helm's goals. I'd be happy to work on further improving the implementation or documentation to align with the project's standards and priorities.

Thank you for considering this proposal, and I look forward to your thoughts!

[Vineet0197]

This will not require any manual intervention, as most of the organisation want to proceed with end to end automation. With this feature helm can be used as package manager along with security integration. This could be a key feature from security grounds to safeguard the required files and helm will support in fetching and deploying the same securely.

@TerryHowe please check and if needed we can discuss further at the convening time

[TerryHowe]

@TerryHowe please check and if needed we can discuss further at the convening time

I am not a Helm maintainer, but @gjenkins8 is.

As George points out, you could use the vault CLI likevault read secret/data/myapp/values rather than attempt to make helm do everything. There is no real security gain using helm over the vault CLI.

What you are proposing here is a big feature that isn't on the roadmap as far as I know. It will be a long process to convince people and iron out the design.

[Vineet0197]

I understand that Helm maintainers aim to keep Helm lightweight and avoid direct integrations with specific third-party solutions. However, the motivation behind this feature is to provide a seamless way for Helm users to retrieve secrets from Vault without relying on external tools or scripting workarounds.

The primary benefits of this integration include:
1. User Experience & Simplification:
• Many users deploy applications requiring secrets from Vault. Having built-in support avoids extra CLI commands or wrapper scripts, making Helm more user-friendly.
• Instead of requiring users to preprocess values using Vault CLI (vault read ...), Helm can handle it natively, reducing complexity.
2. Consistency & Maintainability:
• If multiple teams or users rely on external scripts to fetch values from Vault before passing them to Helm, maintaining consistency becomes challenging.
• Helm already supports different storage providers (e.g., OCI registry support). Extending support for secrets management aligns with existing functionality.
3. Security Considerations:
• The Vault CLI approach exposes secrets in shell history or process arguments, which could be a security risk.
• Direct Helm integration would allow better secret handling within Helm’s execution flow without intermediate exposure.

I understand this is a significant change, and I’d be happy to work with maintainers to refine the design, add necessary tests, and ensure it aligns with Helm’s principles. If the team still prefers this as a plugin rather than a core Helm feature, I’m open to discussing how Helm could better support such extensions.

@gjenkins8 , Could you please take another look at this? This feature enhances security by allowing Helm to fetch values directly from Vault, reducing the need for external CLI tools or manual scripts that may expose sensitive data. Integrating this functionality within Helm provides a more seamless and secure deployment process.

[TerryHowe]

Copy paste that into a HIP, but maybe have the implementation as a plugin would be my (unimportant) recommendation.

[Vineet0197]

As Per the discussion, I have created a Helm Improvement Proposal (HIP) to formalize this feature request. You can review the HIP here: [Link to HIP PR].

This HIP outlines the motivation, technical details, security considerations, and expected benefits of integrating Vault directly into Helm. Looking forward to feedback and further discussion on how we can proceed with this feature!

@TerryHowe @gjenkins8

[github-actions]

This pull request has been marked as stale because it has been open for 90 days with no activity. This pull request will be automatically closed in 30 days if no further activity occurs.