Add Configuration to Handle AWS SDK Checksum Changes for Third-Party S3-Compatible Services #2490
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@JoshData Is there any chance to get this fix into the next release or have it released as a minor fix release? |
MrWinux
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This pull request addresses recent changes in the AWS SDKs that enable checksum validations by default. These changes have caused compatibility issues with third-party S3-compatible services that do not support these checksum headers. To resolve these issues, we've introduced configurations to control checksum behaviors, ensuring compatibility with such services.
Background:
AWS SDKs have introduced default integrity protections for Amazon S3 operations, automatically enabling checksum validations. While this enhances data integrity, it has led to compatibility problems with third-party S3-compatible services that do not support these checksum features. Users have reported errors when interacting with these services. (See: https://discourse.mailinabox.email/t/third-party-s3-and-aws-sdk-changes-how-to-fix/12792)
Changes Introduced:
request_checksum_calculationandresponse_checksum_validationtoWHEN_REQUIRED. This ensures that checksums are only calculated or validated when explicitly required, maintaining compatibility with services that do not support these features. (See: fix: MissingContentLength in boto3 version 1.36.1 boto/boto3#4398 (comment))