Skip to content

[BUG] S3 DeleteObjects (batch) ignores per-object VersionId on versioned buckets #3355

Description

@jwoa95

Service

S3

AWS API Action

DeleteObjects (batch delete, POST /{bucket}?delete)

Expected behavior

When a <Object> element carries a <VersionId>, AWS permanently deletes that specific version and echoes it back:

{"Deleted": [{"Key": "a.txt", "VersionId": "5be89dc4-..."}]}

No delete marker is created, and the version disappears from list-object-versions. This matches Floci's own singular DeleteObject --version-id, which is correct.

Actual behavior

Floci ignores <VersionId> entirely and performs an unversioned delete, so it creates a delete marker per object and leaves every named version readable:

{"Deleted": [{"Key": "a.txt", "DeleteMarker": true, "DeleteMarkerVersionId": "be05e755-..."}]}

Afterwards both original versions are still listed by list-object-versions, plus two new delete markers. <VersionId> is never emitted in the response at all.

Two deviations, then:

  1. the requested version is not deleted (a marker is added instead), and
  2. Deleted.VersionId is always absent, even for a permanent delete.

Where it happens

The VersionId is discarded during request parsing, before any logic runs — S3Controller.java:1305:

List<String> keys = XmlParser.extractAll(xml, "Key");   // <VersionId> siblings never read
...
S3Service.DeleteObjectsResult result = s3Service.deleteObjects(bucket, authorizedKeys);

and S3Service.java:1484:

public DeleteObjectsResult deleteObjects(String bucketName, List<String> keys) {
    for (String key : keys) {
        S3Object result = deleteObject(bucketName, key);       // 2-arg overload
        if (result != null && result.isDeleteMarker()) {
            deleted.add(new DeleteResult(key, null, true, result.getVersionId()));
        } else {
            deleted.add(new DeleteResult(key, null, false, null));
        }               //             ^^^^ DeleteResult.versionId is hardcoded null on both paths

The version-aware overloads already exist a few hundred lines above — deleteObject(bucket, key, versionId) at S3Service.java:1018 and deleteObject(bucket, key, versionId, bypassGovernance) at :1022. The batch path just never reaches them, and because deleteObjects takes List<String>, the version cannot be threaded through without a signature change.

A possible fix, roughly:

  • parse <Object> elements as (Key, VersionId) pairs instead of flattening to List<String> (this also fixes the case of the same key appearing twice with different versions, which currently collapses into two identical unversioned deletes);
  • pass them to a version-aware deleteObjects, dispatching to the 3-arg deleteObject when a version is present;
  • populate DeleteResult.versionId and emit <VersionId> in handleDeleteObjects' non-quiet branch when the delete was permanent.

x-amz-bypass-governance-retention on the batch request appears to be dropped on the same path, since the 4-arg overload is likewise unreachable — untested, mentioning it only because a fix would touch the same lines.

Reproduction

export AWS_ACCESS_KEY_ID=local AWS_SECRET_ACCESS_KEY=local AWS_DEFAULT_REGION=eu-west-2
EP=http://localhost:4566; B=verprobe

aws --endpoint-url $EP s3api create-bucket --bucket $B \
  --create-bucket-configuration LocationConstraint=eu-west-2
aws --endpoint-url $EP s3api put-bucket-versioning --bucket $B \
  --versioning-configuration Status=Enabled

echo one > a.txt && aws --endpoint-url $EP s3api put-object --bucket $B --key a.txt --body a.txt
echo two > a.txt && aws --endpoint-url $EP s3api put-object --bucket $B --key a.txt --body a.txt

VIDS=$(aws --endpoint-url $EP s3api list-object-versions --bucket $B \
  --query 'Versions[].{Key:Key,VersionId:VersionId}' --output json)
aws --endpoint-url $EP s3api delete-objects --bucket $B \
  --delete "{\"Objects\":$VIDS,\"Quiet\":false}"

# Expected: both versions gone. Actual: both still present, plus two delete markers.
aws --endpoint-url $EP s3api list-object-versions --bucket $B

The same script against MinIO returns {"Key":"a.txt","VersionId":"5be89dc4-..."} and leaves the bucket empty.

Environment

  • Floci version / image tag: 2.0.1
  • Java SDK version (if applicable): n/a — reproduced with AWS CLI v2 and @aws-sdk/client-s3 v3 (JS)
  • How you're running Floci: Docker (docker compose, single container, default config)

Why it matters to us

We use the batch delete to permanently erase all versions under a prefix for GDPR right-to-erasure, carrying VersionId on every entry specifically because a versioned delete without one only writes a marker and leaves the bytes readable. Under Floci that code cannot empty a versioned prefix, so we currently have to point those S3 clients at MinIO while everything else stays on Floci. Happy to test a patch.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workings3Amazon Simple Storage Service (S3)

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions