Why is there NO way to use a self signed Cert by specifying a certificate file?

[russfellows]

In general, this SDK is great. I have a created an S3 client, and am able to do GET, PUT, DELETE, LIST, and other necessary operations. Great, BUT... so far is has proven impossible to specify a CA bundle to support self signed certificates. Has ANYONE done this? Where are examples or documentation? It doesn't exist as far as I can find.

Ideally, it would be great to support operations EXACTLY as the "aws s3" cli does, but doing it at all would be a great start. For example the following method of specifying a ca_bundle:

aws s3 ca_bundle = /path/to/cabundle-2019mar05.pem ls s3://my-bucket

This is documented here: [(https://docs.aws.amazon.com/cli/v1/userguide/cli-configure-files.html)]
However, there is no documented method, or code that supports this in the Rust aws_sdk_s3

Note: The setting an environment variable with this SDK, as is documented with aws cli does NOT work:
AWS_CA_BUNDLE=/path/to/ca-cert/cacert.pem

[aajtodd]

The AWS_CA_BUNDLE is a CLI specific environment variable.

The Rust SDK developer guide now has examples of providing a custom CA root: https://docs.aws.amazon.com/sdk-for-rust/latest/dg/http.html#customizeCertificatesTls

[russfellows]

Aaron, see my subsequent replies. This code sample is WAY too minimal to be useful.

[russfellows]

Thanks Aaron, I will try this. I could find nothing, and after extensive Chat interactions, pointing it at Github examples, the Rust crates, the aws_sdk_s3, rustls, smithy-aws, etc. etc. I ended up with 40 sets of code, none of which would compile. It would be nice to add something to the high level documentation pointing here also, because searching the docs for “CA” “Certs”, Custom, etc. all turned up nothing. Regards, —Russ Russ Fellows Senior Partner, Evaluator Group ***@***.***

…

On Apr 11, 2025, at 12:20 PM, Aaron Todd ***@***.***> wrote: The AWS_CA_BUNDLE is a CLI specific environment variable. The Rust SDK developer guide now has examples of providing a custom CA root: https://docs.aws.amazon.com/sdk-for-rust/latest/dg/http.html#customizeCertificatesTls — Reply to this email directly, view it on GitHub <#1277 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF64UJ5K4ND7RMHHAGW4O432ZABXRAVCNFSM6AAAAAB26WMOVKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTEOBQG42DCNA>. You are receiving this because you authored the thread.

[russfellows]

Sorry, that is not helpful. The code sample is WAY too minimal. If there are going to be examples, it needs to produce code that someone can actually run. The reference is a small code snipet which is useless without a lot more code.

I am attaching a Rust program that "SHOULD" compile and create a useful example. Except of course it does not because there is not enough information in all the crates to create a working example. This is as close as I can come. If it were to compile, it would use environment variables for KEY_ID and SECRET_KEY along with URL to connect to S3 storage over TLS, and list the contents of a given bucket. Importantly, it should also allow the use of a custom certificate, via CLI or environment variable.

Can ANYONE please correct this, and produce an actual working example? I STILL have NEVER seen a working example.

Code attached:
simple_s3_custom_cert.rs.txt

[russfellows]

Update.

I have been reading docs, and "chatting" with the AI gods. The program example is now closer to compiling. I use the EXACT same function as specified by Aaron above. That code is fine, it's just that there are still no examples of how to use that to build an actual functional S3 client. Note the following version that reports two errors from "cargo build". I will also attach my Cargo.toml file.

Cargo.toml:

[package]
name = "s3_custom"
version = "0.1.0"
edition = "2024"

[dependencies]
tokio = { version = "1.0", features = ["full"] }
aws-sdk-s3 = "1.82.0"
aws-smithy-client = "0.60.3"
clap = "4.5.36"
rustls = "0.23.26"
rustls-pemfile = "2.2.0"
hyper-rustls = "0.27.5"
tokio-rustls = "0.26.2"
aws-smithy-http-client = { version = "1.0.1", features = ["rustls-aws-lc"] }

Rust main.rs:

// src/main.rs
//
// This is intended to be a standalone, "simple" program that can demonstrate
// loading a custom certificate, and then performing a bucket list of an
// S3 provider, that uses a self signed certificate.
//
// This is close to the minimal set of code required to produce a usable
// example.  Currently it does not compile because of a lack of documentation
// and / or examples.
//
// Note: this is getting close, but still has two cargo build errors.  
// 1) The first on line 113: .rustls_connector(tls_context)
// 2) The second on line 119:  .endpoint_resolver(Endpoint::immutable 
//

use aws_sdk_s3::{Client, Config};
use aws_sdk_s3::config::{Credentials, Region};
use aws_sdk_s3::config::endpoint::Endpoint;
use aws_smithy_http_client::{
    tls::{self},
    Builder,
};
use clap::{Arg, Command};
use std::env;
use std::fs;

/// Function to create a TLS context from a PEM file (exactly as specified in the documentation).
fn tls_context_from_pem(filename: &str) -> tls::TlsContext {
    let pem_contents = fs::read(filename).unwrap();

    // Create a new empty trust store (this will not load platform native certificates)
    let trust_store = tls::TrustStore::empty()
        .with_pem_certificate(pem_contents.as_slice());

    tls::TlsContext::builder()
        .with_trust_store(trust_store)
        .build()
        .expect("valid TLS config")
}

/// Lists objects in the specified S3 bucket.
async fn list_bucket_objects(client: &Client, bucket: &str) {
    match client.list_objects_v2().bucket(bucket).send().await {
        Ok(output) => {
            if let Some(contents) = output.contents {
                println!("Objects in bucket '{}':", bucket);
                for object in contents {
                    if let Some(key) = object.key {
                        println!("- {}", key);
                    }
                }
            } else {
                println!("No objects found in bucket '{}'.", bucket);
            }
        }
        Err(err) => {
            eprintln!("Error listing objects in bucket '{}': {:?}", bucket, err);
        }
    }
}

#[tokio::main]
async fn main() {
    // Command-line argument parsing using Clap
    let matches = Command::new("S3 Client with Custom TLS")
        .arg(
            Arg::new("bucket")
                .short('b')
                .long("bucket")
                .value_parser(clap::value_parser!(String))
                .required(true)
                .help("The S3 bucket to list objects from"),
        )
        .arg(
            Arg::new("endpoint")
                .short('e')
                .long("endpoint")
                .value_parser(clap::value_parser!(String))
                .help("The custom S3 endpoint (e.g., http://localhost:9000)"),
        )
        .arg(
            Arg::new("cert")
                .short('c')
                .long("cert")
                .value_parser(clap::value_parser!(String))
                .required(true)
                .help("Path to the custom CA certificate file"),
        )
        .get_matches();

    // Parse command-line arguments
    let bucket = matches
        .get_one::<String>("bucket")
        .expect("Bucket name is required");
    let endpoint = matches
        .get_one::<String>("endpoint")
        .cloned()
        .unwrap_or_else(|| env::var("S3_ENDPOINT").unwrap_or_else(|_| "http://localhost:9000".to_string()));
    let cert_path = matches
        .get_one::<String>("cert")
        .expect("Certificate path is required");

    // Load AWS credentials from environment variables
    let aws_access_key_id = env::var("AWS_ACCESS_KEY_ID").expect("AWS_ACCESS_KEY_ID must be set");
    let aws_secret_access_key =
        env::var("AWS_SECRET_ACCESS_KEY").expect("AWS_SECRET_ACCESS_KEY must be set");

    // Create the TLS context using the custom CA certificate
    let tls_context = tls_context_from_pem(cert_path);

    // Create an AWS Smithy HTTP client builder and configure it
    let smithy_client = Builder::new()
        .rustls_connector(tls_context) // Reference: https://docs.rs/aws-smithy-http/0.53.0/aws_smithy_http/client/struct.Builder.html#method.rustls_connector
        .build();

    // Build the AWS SDK configuration
    let aws_config = Config::builder()
        .region(Region::new("us-east-1"))
        .endpoint_resolver(Endpoint::immutable(endpoint.parse().expect("Invalid endpoint URL"))) // Reference: https://docs.rs/aws-sdk-s3/1.82.0/aws_sdk_s3/config/endpoint/trait.ResolveEndpoint.html
        .credentials_provider(Credentials::new(
            aws_access_key_id,
            aws_secret_access_key,
            None,
            None,
            "custom",
        ))
        .build();

    // Create the S3 client using the custom HTTP connector
    let s3_client = Client::from_conf(aws_config);

    // List objects in the specified bucket
    list_bucket_objects(&s3_client, bucket).await;
}

[aajtodd]

Couple things stand out:

1. There is no rustls_connector function on the aws_smithy_http_client::Builder type. The comment with the reference is referencing the aws_smithy_http crate which is not the same thing.
2. You are not setting the http_client on the client config
3. If you are overriding the endpoint you don't need an endpoint resolver usually, you just need to override the endpoint_url (you can do this in code or via env variables globally or per/service FYI). Also see the Rust developer guide section on endpoint config
4. Highly recommend you use the aws_config crate for resolving most of your configuration and only overriding the parts you need. In particular using this will remove the need for you to even set the credentials provider as the default credential chain will already handle sourcing it from environment. See the developer guide section on credential providers

The developer guide provides the necessary code for all of these things.

    let http_client = Builder::new()
        .tls_provider(tls::Provider::Rustls(CryptoMode::AwsLc))
        .tls_context(tls_context_from_pem("my-custom-ca.pem"))
        .build_https();
    
    let sdk_config = aws_config::defaults(
        aws_config::BehaviorVersion::latest()
    )
    .http_client(http_client)
    .endpoint_url("https://localhost:9000")
    .load()
    .await;
    
    // create client(s) using sdk_config
    let s3_client = aws_sdk_s3::Client::new(&sdk_config);


    // List objects in the specified bucket
    list_bucket_objects(&s3_client, bucket).await;

[russfellows]

I will try this, thank you for the explanation and code example.