I don't think this is the same as #1007 because in general my secrets are working.

The only secret that isn't working is the one that holds a JSON value which contains my Google service account config.
Locally, my features involving the service account are working and i'm using the same secrets manager so I think there is something weird going on with how this secret is handled during a kamal deploy.

Context

• Kamal 2.5.2
• Rails 8.0.1
• agent: Doppler

# kamal/secrets

SECRETS=$(kamal secrets fetch --adapter doppler --from project/prd KAMAL_REGISTRY_PASSWORD RAILS_MASTER_KEY GOOGLE_SERVICE_ACCOUNT_CONFIG)
KAMAL_REGISTRY_PASSWORD=$(kamal secrets extract KAMAL_REGISTRY_PASSWORD ${SECRETS})
RAILS_MASTER_KEY=$(kamal secrets extract RAILS_MASTER_KEY ${SECRETS})
GOOGLE_SERVICE_ACCOUNT_CONFIG=$(kamal secrets extract GOOGLE_SERVICE_ACCOUNT_CONFIG ${SECRETS})

# EXAMPLE - contents of the service account config

GOOGLE_SERVICE_ACCOUNT_CONFIG={
  "type": "service_account",
  "project_id": "project-dev",
  "private_key_id": "*****",
  "private_key": "-----BEGIN PRIVATE KEY-----\n*****\n*****\n*****\n*****\n*****\n*****\n*****\n*****\n*****\n*****\n*****\n*****\n*****\n*****\n*****\n*****\n*****\n*****\n*****\n*****\n*****\n-----END PRIVATE KEY-----\n",
  "client_email": "[email protected]",
  "client_id": "123451133783890925820",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://oauth2.googleapis.com/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/connect%40project-dev.iam.gserviceaccount.com",
  "universe_domain": "googleapis.com"
}

DIFF

Here's the difference in what I'm seeing in console locally vs on the server for ENV['GOOGLE_SERVICE_ACCOUNT_CONFIG']

When I do kamal secret print it looks ok -- as in, it looks like the example i share above. My terminal displays it as properly tabbed JSON and the newlines in the private key are \n.

When I do kamal shell and echo it looks like this:

rails@178:/rails$ echo $GOOGLE_SERVICE_ACCOUNT_CONFIG
{\n "type": "service_account",\n "project_id": "project-dev",\n "private_key_id": "******",\n "private_key": "-----BEGIN PRIVATE KEY-----\\n ... \\n-----END PRIVATE KEY-----\\n",\n "client_email": "[email protected]",\n "client_id": "******",\n "auth_uri": "https://accounts.google.com/o/oauth2/auth",\n "token_uri": "https://oauth2.googleapis.com/token",\n "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",\n "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/connect%40project-dev.iam.gserviceaccount.com",\n "universe_domain": "googleapis.com"\n}

Currently looking for a workaround. Got close with this:

doppler secrets -p project-dev -c prd get GOOGLE_SERVICE_ACCOUNT_CONFIG --plain | jq -c .

but this results in an RSA error. The extra \n are removed at the key level but the private key still has \\\\n in console.

Will post if I find something that works.

Happy dig/share more if I can get some guidance. Thanks!