Skip to content

Conversation

@jt-poolside
Copy link

Summary

This PR adds explicit credential configuration support to the S3 archiver, enabling easier use with non-AWS S3-compatible object stores like MinIO, Wasabi, DigitalOcean Spaces, etc.

Problem

Currently, the S3 archiver relies solely on AWS SDK's default credential chain (environment variables, ~/.aws/credentials, IAM roles, etc.). This makes it difficult to:

  • Use non-AWS S3-compatible object stores without environment variables
  • Configure different credentials for different archival targets
  • Explicitly specify credentials in configuration files

Solution

Following the existing Elasticsearch AWS request signing pattern, this PR adds three credential provider options:

  1. static - Explicit access key, secret key, and optional session token
  2. environment - Read from AWS environment variables
  3. aws-sdk-default - Use AWS SDK default credential chain (maintains current behavior)

When credentialProvider is not specified, it defaults to aws-sdk-default, ensuring full backward compatibility with existing configurations.

Configuration Examples

Static Credentials

archival:
  history:
    provider:
      s3store:
        region: "us-east-1"
        endpoint: "http://minio:9000"
        credentialProvider: "static"
        static:
          accessKeyID: "AKIAIOSFODNN7EXAMPLE"
          secretAccessKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"

Static Credentials with STS Token

archival:
  history:
    provider:
      s3store:
        region: "us-west-2"
        credentialProvider: "static"
        static:
          accessKeyID: "ASIAIOSFODNN7EXAMPLE"
          secretAccessKey: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
          token: "AQoDYXdzEJr...<remainder of token>"

Environment Credentials

archival:
  history:
    provider:
      s3store:
        region: "eu-west-1"
        credentialProvider: "environment"

Backward Compatible (Unchanged)

archival:
  history:
    provider:
      s3store:
        region: "us-east-1"
        endpoint: "http://localhost:4566"

Changes

  • Config Structure: Added CredentialProvider and Static fields to S3Archiver config
  • New Files:
    • common/archiver/s3store/aws_credentials.go - Credential provider logic
    • common/archiver/s3store/aws_credentials_test.go - Comprehensive unit tests
  • Updated Files:
    • history_archiver.go - Uses new createS3Session() helper
    • visibility_archiver.go - Uses new createS3Session() helper

Testing

✅ All existing tests pass
✅ 13 new unit tests covering:

  • Static credentials (with and without STS token)
  • Environment credentials
  • AWS SDK default provider
  • Backward compatibility (empty provider)
  • Invalid provider error handling
  • Case-insensitive provider names
go test ./common/archiver/s3store -count=1
ok      go.temporal.io/server/common/archiver/s3store   0.298s

Backward Compatibility

This change is fully backward compatible. Existing configurations without credentialProvider will continue to work exactly as before, using the AWS SDK's default credential chain.

The S3 archiver previously relied solely on AWS SDK's default
credential chain (environment variables, ~/.aws/credentials, IAM
roles, etc.), making it difficult to use with non-AWS S3-compatible
object stores like MinIO or Wasabi without relying on environment
variables.

This change adds explicit credential configuration support following
the existing Elasticsearch AWS request signing pattern. Users can now
specify credentials in three ways:

1. static - Explicit access key, secret key, and optional session token
2. environment - Read from AWS environment variables
3. aws-sdk-default - Use AWS SDK default chain (maintains current behavior)

When credentialProvider is not specified, it defaults to
"aws-sdk-default" ensuring full backward compatibility with existing
configurations.

Changes:
- Add CredentialProvider and Static fields to S3Archiver config
- Add S3StaticCredentialProvider struct for static credentials
- Create aws_credentials.go with credential provider logic
- Update history and visibility archivers to use new session helper
- Add comprehensive unit tests for all credential provider types

All existing tests pass, confirming backward compatibility.
@jt-poolside jt-poolside requested review from a team as code owners December 9, 2025 09:35
@CLAassistant
Copy link

CLAassistant commented Dec 9, 2025

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants