Skip to content

Support IAM / short-lived token authentication for external Redis (e.g. GCP Memorystore) #6391

Description

@ksk-ktsr

Problem

When self-hosting, Nango can connect to an external Redis only with a static
password
— either NANGO_REDIS_URL or NANGO_REDIS_HOST/PORT/AUTH, which is
baked into a rediss://:<password>@host:port URL once at startup
(packages/kvstore/lib/index.ts).

Managed Redis offerings that use IAM authentication (e.g. GCP Memorystore for
Redis/Valkey) don't use a static password. The "password" is a short-lived OAuth2
access token (~1h TTL) that must be refreshed before it expires. With the current
implementation:

  • A token placed in NANGO_REDIS_AUTH is embedded in the URL and never refreshed,
    so any reconnect after the token expires fails with NOAUTH / WRONGPASS.
  • There is no way to supply a rotating credential.

So Nango cannot be used with IAM-authenticated managed Redis today.

Proposed solution

Add an optional token-file credential source that is re-read on every
(re)connect, using node-redis v5's credentialsProvider
(async-credentials-provider). This is cloud-agnostic: any external process
(sidecar, init job, etc.) that writes the current token to a file works
(GCP/AWS/Azure IAM, Vault, …).

New env vars (system + customer boundary):

  • NANGO_REDIS_AUTH_TOKEN_FILE — path to a file holding the current token;
    re-read on every (re)connect, no restart needed.
  • NANGO_REDIS_USERNAME — optional AUTH username (defaults to default).
  • NANGO_CUSTOMER_REDIS_AUTH_TOKEN_FILE, NANGO_CUSTOMER_REDIS_USERNAME
    same for the customer boundary.

When the token file is set, no credentials are embedded in the URL. When unset,
behaviour is identical to today (static auth).

Scope

  • Centralize Redis client-options building (TLS/socket/auth) in kvstore so the
    server publisher, rate limiter, and jobs runner all share one implementation
    instead of 4 duplicated copies.
  • Add the credentials-provider path.
  • Docs (self-hosting.mdx) + unit tests.
  • (Incidental) fix two node-redis v5 issues surfaced while refactoring:
    scanIterator now yields batches (not single keys), and keepAlive was being
    passed a number instead of keepAliveInitialDelay.

Contribution

I already have a working implementation and would like to contribute it. Happy to
adjust the approach or split the incidental fixes into a separate PR if you prefer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions