General question, anybody actively using the vaulttransit secret backend?

[kgorskowski]

For the love of good I cannot get it working and there are not a lot of examples or discussions about it around.
The vault backend is working without any problems as we use it with sops.
But the kapitan params configuration doesnt seem to be picked up when I try to write refs.
Firstly the vault auth isn't picked up and gives me the misleading information that the auth for "vaultkv" is missing (I guess a bug in the codebase). Can be solved by providing "vault-auth" on the cli.
But now I am stuck at trying to point to the right mount and key. And it is just not picking up my configuration.
So, if anyone is actively working with this, I would love to get some directions.
Thanks!

[Moep90]

That is what we do,

1. Create the secret in Kapitan refs

export VAULT_ADDR="https://vault......."
export VAULT_TOKEN="myTokenValue"
echo -n "<value>" | kapitan refs --write vaulttransit:path/to/secret/in/refs -t <target> -f -

2. Use the secret

parameters:
  kapitan:
    secrets:
      vaulttransit:
        auth: token
        mount: some_mount
        crypto_key: ${target_name}-key
        always_latest: true
        update_key: true

  secrets-name:
    some-creds:
      string_data:
        value: ?{vaulttransit:path/to/secret/in/refs||randomstr}

[kgorskowski]

Thanks for your answer, haven't tried to reference a secret with a generated default yet. Following the documentation I am not even able to configure the backend through parameters.kapitan.secrets.
Having defined it on target level to rule out hierarchy problems this is on my "target" definition

parameters:
  kapitan:
    secrets:
      vaulttransit:
        VAULT_ADDR: http://$ADDRESS:8200
        VAULT_SKIP_VERIFY: "True" # for now
        auth: token
        mount: custom-transit # non-default transit mount point
        key: transit # what is actually needed here? key and/or crypto_key?

The kapitan inventory picks it up, but when I try to write a secret I end up on this error

 ~/kapitan  echo "foo:bar" | kapitan refs --write vaulttransit:${target_name}/foo -t ${target_name}  -f -
Traceback (most recent call last):
  File "/Users/.pyenv/versions/3.12.9/bin/kapitan", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/.pyenv/versions/3.12.9/lib/python3.12/site-packages/kapitan/cli.py", line 639, in main
    args.func(args)
  File "/Users/.pyenv/versions/3.12.9/lib/python3.12/site-packages/kapitan/refs/cmd_parser.py", line 36, in handle_refs_command
    ref_write(args, ref_controller)
  File "/Users/.pyenv/versions/3.12.9/lib/python3.12/site-packages/kapitan/refs/cmd_parser.py", line 215, in ref_write
    raise KapitanError(
kapitan.errors.KapitanError: No Authentication type parameter specified. Specify it in parameters.kapitan.secrets.vaultkv.auth and use --target-name or use --vault-auth

When I add the --vault-auth token flag, kapitan tries to communicate with the vault but only under the default "transit" mount point.

edit: the vault is reachable and unsealed, a token is stored in $HOME/.vault-token and also exported as VAULT_TOKEN env variable. Like I said we are using it quite heavily with sops

[Moep90]

I've added our vaulttransit conf to my initial comment.
We also used to export export VAULT_ADDR=.... and export VAULT_TOKEN=... in our terminal

Additionally, I'm not sure if kapitan refs ...... -t ${target_name} actually works (never tried it though)

    key: transit # what is actually needed here? key and/or crypto_key?

re: For us it was the: https://registry.terraform.io/providers/hashicorp/vault/latest/docs/resources/transit_secret_backend_key

[kgorskowski]

ok thanks again for your efforts. But again, it looks to me as if kapitan for some reason is not picking up my parameters.kapitan.secrets.vaulttransit block. the target_name is just an exemplatory environment variable.
The same syntax works without problems if I use it with plain or base64 secrets. So in my mind if the syntax works with one secret backend I could just drop in replace it with a different backend.
I dont know if I configure it at the wrong place or something but as I only use it directly in a "target" yaml and the inventory looks valid to me I am not sure where else I could put the vaulttransit configuration parameters block. Maybe my kapitan experience is cursed but in this case I just cannot pinpoint the error
edit:
so when I just enable a transit engine at the default mount I am able to write a ref with kapitan refs, but like I expected I am not able to reveal it. Compile errors with kapitan.errors.CompileError: Error compiling targets: Error $target: Cannot access vault params althought the vault params look correct in the targets inventory.
kapitan is not picking up the vaulttransit configuration and I have no idea why. Maybe I haven't yet really understood the data structure kapitan expects from me

[ademariag]

Can you show the example of kapitan inventory -t target -p kapitan.secrets

…

On Sat, 1 Mar 2025, 08:25 Karsten Gorskowski, ***@***.***> wrote: ok thanks again for your efforts. But again, it looks to me as if kapitan for some reason is not picking up my parameters.kapitan.secrets.vaulttransit block. the target_name is just an exemplatory environment variable. The same syntax works without problems if I use it with plain or base64 secrets. So in my mind if the syntax works with one secret backend I could just drop in replace it with a different backend. I dont know if I configure it at the wrong place or something but as I only use it directly in a "target" yaml and the inventory looks valid to me I am not sure where else I could put the vaulttransit configuration parameters block. Maybe my kapitan experience is cursed but in this case I just cannot pinpoint the error — Reply to this email directly, view it on GitHub <#1297 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHZB3RQMUWRIJ322TWSRRNT2SFVHLAVCNFSM6AAAAABYCSNYGSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMZVHEYDKMI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[kgorskowski]

Ok will have a look at the slack, but will probably call it a day for the weekend.
You know I really would love to make this work, I am hoping to streamline a pretty complicated manual legacy mess I face in a current project. But right now I must admit I do struggle with the layout, the layering and the documentation of kapitan. It took a while to get the basic grasp of it and I guess my understanding got better, but now I seem to have hit a wall again.

 ~/p/.../...-kapitan  kapitan inventory -t $target -p kapitan.secrets
awskms: null
azkms: null
gkms: null
gpg: null
vaultkv: null
vaulttransit:
  addr: http://192.168.99.24:8200 # my local test instance authenticated via root token
  always_latest: true
  auth: token
  cacert: null
  capath: null
  client_cert: null
  client_key: null
  crypto_key: sops-key # I am not sure if one or both of key/crypto_key should be used, tried all variants
  key: sops-key
  engine: transit
  mount: transit # this is the default that I created to see if it will work, the goal is to have a custom mount
  namespace: null
  skip_verify: true

the general connection to the vault and transit is working.
So from my understanding the inventory looks valid but trying to write a secret against the target backend config doesn't work. If I create a ref "manually" and try to compile/reveal it it also says:
kapitan.errors.CompileError: Error compiling targets: Error compiling $target: Cannot access vault params

Thanks again but I guess I need a break for the weekend.

[ademariag]

Also, please join the kapitan slack channel on the kubernetes slack for quicker interactions On Sat, 1 Mar 2025, 09:11 Alessandro De Maria, ***@***.***> wrote:

…

Can you show the example of kapitan inventory -t target -p kapitan.secrets On Sat, 1 Mar 2025, 08:25 Karsten Gorskowski, ***@***.***> wrote: > ok thanks again for your efforts. But again, it looks to me as if kapitan > for some reason is not picking up my > parameters.kapitan.secrets.vaulttransit block. the target_name is just > an exemplatory environment variable. > The same syntax works without problems if I use it with plain or base64 > secrets. So in my mind if the syntax works with one secret backend I could > just drop in replace it with a different backend. > I dont know if I configure it at the wrong place or something but as I > only use it directly in a "target" yaml and the inventory looks valid to me > I am not sure where else I could put the vaulttransit configuration > parameters block. Maybe my kapitan experience is cursed but in this case I > just cannot pinpoint the error > > — > Reply to this email directly, view it on GitHub > <#1297 (reply in thread)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AHZB3RQMUWRIJ322TWSRRNT2SFVHLAVCNFSM6AAAAABYCSNYGSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEMZVHEYDKMI> > . > You are receiving this because you are subscribed to this thread.Message > ID: ***@***.***> >