feat: S3 Express support #1906

jbelkins · 2025-03-14T22:08:54Z

Description of changes

Provides customizations needed to utilize the S3 Express feature.

New/existing dependencies impact assessment, if applicable

No new dependencies were added to this change.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

jbelkins · 2025-03-14T22:12:33Z

...ore/AWSSDKCommon/Sources/AWSSDKCommon/FileBasedConfiguration/CRTFileBasedConfiguration.swift

+                continuation.resume(returning: fileBasedConfig)
+            } catch {
+                continuation.resume(throwing: error)
+            }


This just fixes a concurrency issue; the nonconcurrent code called here should be called using withCheckedThrowingContinuation.

jbelkins · 2025-03-14T22:15:46Z

Sources/Core/AWSSDKHTTPAuth/Sources/AWSSDKHTTPAuth/AWSSigV4Signer.swift

@@ -49,7 +50,7 @@ public class AWSSigV4Signer: SmithyHTTPAuthAPI.Signer {
            )
        }

-        guard let identity = identity as? AWSCredentialIdentity else {
+        guard let identity = (identity as? AWSCredentialIdentity) ?? (identity as? S3ExpressIdentity)?.awsCredentialIdentity else {


When the CRT signer signs in S3 Express mode, it intakes the S3 Express credentials using the same Credentials type as with any other signing operation. So, when an S3ExpressIdentity is supplied, convert it to an AWS credential identity to pass into CRT (see private helper method below.)

jbelkins · 2025-03-14T22:17:05Z

Sources/Core/AWSSDKHTTPAuth/Sources/AWSSDKHTTPAuth/SigV4S3ExpressAuthScheme.swift

+import protocol SmithyHTTPAuthAPI.Signer
+import struct Smithy.Attributes
+
+public struct SigV4S3ExpressAuthScheme: AuthScheme {


This AuthScheme for S3 Express is basically the sigv4 auth scheme, but with a different scheme ID & signing algorithm.

jbelkins · 2025-03-14T22:18:38Z

Sources/Core/AWSSDKIdentity/Sources/AWSSDKIdentity/AWSIdentityPropertyKeys.swift

+
+import struct Smithy.AttributeKey
+
+public enum AWSIdentityPropertyKeys {


These keys are used to pass data & components into identity resolvers in a type-safe manner.

jbelkins · 2025-03-14T22:23:17Z

Sources/Core/AWSSDKIdentity/Sources/AWSSDKIdentity/S3Express/S3ExpressCreateSessionClient.swift

+        clientConfig: DefaultClientConfiguration,
+        bucket: String
+    ) async throws -> S3ExpressIdentity
+}


This protocol provides an interface for the S3 Express identity resolver to call S3 CreateSession to get credentials, without depending on the code-generated S3 client.

A SwiftIntegration on the S3 service provides conformance with this protocol.

jbelkins · 2025-03-14T22:24:05Z

Sources/Core/AWSSDKIdentity/Sources/AWSSDKIdentity/S3Express/S3ExpressIdentity.swift

+
+import protocol SmithyIdentityAPI.Identity
+import struct Foundation.Date
+


A type to hold a S3 Express identity.

jbelkins · 2025-03-14T22:25:24Z

Sources/Core/AWSSDKIdentity/Sources/AWSSDKIdentity/S3Express/S3ExpressIdentityResolver.swift

+import struct Smithy.Attributes
+import protocol SmithyIdentityAPI.IdentityResolver
+
+public protocol S3ExpressIdentityResolver: IdentityResolver where IdentityT == S3ExpressIdentity {}


A convenience declaration for the type of an S3 Express-specialized identity resolver.

jbelkins · 2025-03-14T22:27:47Z

Sources/Services/AWSS3/Sources/AWSS3/AuthSchemeResolver.swift

+                    authOption.signingProperties.set(key: SmithyHTTPAuthAPI.SigningPropertyKeys.signingRegion, value: param.signingRegion)
+                    authOption.identityProperties.set(key: AWSSDKIdentity.AWSIdentityPropertyKeys.bucket, value: serviceParams.bucket)
+                    authOption.identityProperties.set(key: AWSSDKIdentity.AWSIdentityPropertyKeys.s3ExpressClient, value: S3ExpressCreateSessionClient())
+                    validAuthOptions.append(authOption)


[Note: this is generated code]

An auth option is added for S3 Express. The bucket name and a S3ExpressCreateSessionClient are added to identity properties for use by the S3 Express identity resolver.

jbelkins · 2025-03-14T22:34:16Z

Sources/Services/AWSS3/Sources/AWSS3/Models.swift

@@ -22522,6 +22523,7 @@ extension GetObjectInput {
                      .withExpiration(value: expiration)
                      .withIdentityResolver(value: config.awsCredentialIdentityResolver, schemeID: "aws.auth#sigv4")
                      .withIdentityResolver(value: config.awsCredentialIdentityResolver, schemeID: "aws.auth#sigv4a")
+                      .withIdentityResolver(value: config.s3ExpressIdentityResolver, schemeID: "aws.auth#sigv4-s3express")


[Note: this is generated code]

For this & several other presigned URLs:

The client config is added to context so it can be used for the S3 CreateSession operation to get credentials later on.

The S3 Express identity resolver is added to the operation.

jbelkins · 2025-03-14T22:35:28Z

Sources/Services/AWSS3/Sources/AWSS3/Plugins.swift

@@ -44,7 +45,7 @@ public class DefaultAWSAuthSchemePlugin: ClientRuntime.Plugin {
    public func configureClient(clientConfiguration: ClientRuntime.ClientConfiguration) throws {
        if let config = clientConfiguration as? S3Client.S3ClientConfiguration {
            config.authSchemeResolver = DefaultS3AuthSchemeResolver()
-            config.authSchemes = [AWSSDKHTTPAuth.SigV4AuthScheme(), AWSSDKHTTPAuth.SigV4AAuthScheme()]
+            config.authSchemes = [AWSSDKHTTPAuth.SigV4AuthScheme(), AWSSDKHTTPAuth.SigV4AAuthScheme(), AWSSDKHTTPAuth.SigV4S3ExpressAuthScheme()]


[Note: this is generated code]

SigV4S3ExpressAuthScheme is added to the service's auth schemes

jbelkins · 2025-03-14T22:36:52Z

...y-aws-swift-codegen/src/main/kotlin/software/amazon/smithy/aws/swift/codegen/AWSAuthUtils.kt

-
+        if (ctx.service.isS3) {
+            updatedAuthSchemeList += writer.format("\$N()", AWSSDKHTTPAuthTypes.SigV4S3ExpressAuthScheme)
+        }


SigV4S3ExpressAuthScheme is added, but only to the S3 service client.

jbelkins · 2025-03-14T22:37:34Z

...en/src/main/kotlin/software/amazon/smithy/aws/swift/codegen/AWSHTTPProtocolCustomizations.kt

@@ -36,6 +37,9 @@ abstract class AWSHTTPProtocolCustomizations : DefaultHTTPProtocolCustomizations
        }
        writer.write("  .withIdentityResolver(value: config.awsCredentialIdentityResolver, schemeID: \$S)", "aws.auth#sigv4")
        writer.write("  .withIdentityResolver(value: config.awsCredentialIdentityResolver, schemeID: \$S)", "aws.auth#sigv4a")
+        if (ctx.service.isS3) {
+            writer.write("  .withIdentityResolver(value: config.s3ExpressIdentityResolver, schemeID: \$S)", "aws.auth#sigv4-s3express")
+        }


S3 Express identity resolver is added to the context for an operation, but only to the S3 service client.

jbelkins · 2025-03-14T22:39:01Z

...tware/amazon/smithy/aws/swift/codegen/customization/RulesBasedAuthSchemeResolverGenerator.kt

+                            )
+                            write("validAuthOptions.append(authOption)")
+                            dedent()
+                        }


In the S3 rules-based auth scheme resolver only, the S3 Express auth option is added.

(See the generated code above for more discussion of the content here.)

jbelkins · 2025-03-14T22:41:43Z

...ain/kotlin/software/amazon/smithy/aws/swift/codegen/customization/s3/S3ExpressIntegration.kt

+        ctx: SwiftCodegenContext,
+        protocolGenerationContext: ProtocolGenerator.GenerationContext,
+        delegator: SwiftDelegator
+    ) {


This method generates a file that creates a concrete implementation of the S3ExpressCreateSessionClient protocol.

An instance of this class is passed into the S3 credential provider, and allows that credential provider to call S3 without actually depending on S3.

jbelkins added 2 commits March 14, 2025 17:02

Add S3 codegen

76b7286

Runtime & codegen changes

fec058f

jbelkins commented Mar 14, 2025

View reviewed changes

jbelkins added 4 commits March 14, 2025 17:58

Fix codegen tests

9bf1e9f

Fix ktlint

6171042

fix Swiftlint

91d5131

Merge branch 'main' into jbe/s3_express

1c23aeb

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: S3 Express support #1906

feat: S3 Express support #1906

jbelkins commented Mar 14, 2025

jbelkins Mar 14, 2025

jbelkins Mar 14, 2025

jbelkins Mar 14, 2025

jbelkins Mar 14, 2025

jbelkins Mar 14, 2025

jbelkins Mar 14, 2025

jbelkins Mar 14, 2025

jbelkins Mar 14, 2025

jbelkins Mar 14, 2025

jbelkins Mar 14, 2025

jbelkins Mar 14, 2025

jbelkins Mar 14, 2025

jbelkins Mar 14, 2025

jbelkins Mar 14, 2025


		import struct Smithy.AttributeKey

		public enum AWSIdentityPropertyKeys {


		import protocol SmithyIdentityAPI.Identity
		import struct Foundation.Date

feat: S3 Express support #1906

Are you sure you want to change the base?

feat: S3 Express support #1906

Conversation

jbelkins commented Mar 14, 2025

Description of changes

New/existing dependencies impact assessment, if applicable

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment