Skip to content

[BUG] DynamoDB Query silently ignores an ExclusiveStartKey from a different partition #3322

Description

@rfoel

Service

DynamoDB

AWS API Action

Query with ExclusiveStartKey

Expected behavior

When the ExclusiveStartKey belongs to a different partition than the one the query asks for, DynamoDB rejects the request with HTTP 400 ValidationException:

The provided starting key is outside query boundaries based on provided condition

AWS-published DynamoDB Local 3.3.1 rejects it the same way.

Actual behavior

Floci accepts the request and ignores the cursor. The query returns the target partition from the first item, as if no ExclusiveStartKey had been passed at all.

This is worse than a missing check. Code that paginates in a loop and accidentally reuses a cursor across partitions gets duplicate items back instead of an error, so the bug is silent.

This looks like the same code path as #2468 (DynamoDbJsonHandler.validateExclusiveStartKey). That issue was about wrong value types in the cursor. Here the types are correct and match the key schema, but the partition key value does not match the KeyConditionExpression.

Reproduction

aws dynamodb create-table --table-name repro-cursor \
  --attribute-definitions AttributeName=pk,AttributeType=S AttributeName=sk,AttributeType=S \
  --key-schema AttributeName=pk,KeyType=HASH AttributeName=sk,KeyType=RANGE \
  --billing-mode PAY_PER_REQUEST
aws dynamodb wait table-exists --table-name repro-cursor

for s in 1 2; do
  aws dynamodb put-item --table-name repro-cursor --item "{\"pk\":{\"S\":\"A\"},\"sk\":{\"S\":\"$s\"}}"
  aws dynamodb put-item --table-name repro-cursor --item "{\"pk\":{\"S\":\"B\"},\"sk\":{\"S\":\"$s\"}}"
done

# cursor from partition A
CUR=$(aws dynamodb query --table-name repro-cursor \
  --key-condition-expression "pk = :p" \
  --expression-attribute-values '{":p":{"S":"A"}}' \
  --limit 1 --query LastEvaluatedKey --output json)

# same cursor, but querying partition B
aws dynamodb query --table-name repro-cursor \
  --key-condition-expression "pk = :p" \
  --expression-attribute-values '{":p":{"S":"B"}}' \
  --exclusive-start-key "$CUR"

Real AWS and DynamoDB Local: ValidationException.

Floci returns:

{
  "Items": [
    { "pk": { "S": "B" }, "sk": { "S": "1" } },
    { "pk": { "S": "B" }, "sk": { "S": "2" } }
  ],
  "Count": 2
}

Note that sk = 1 comes back even though the cursor should have skipped past the first item.

Environment

  • Floci version / image tag: 2.0.1, floci/floci:latest (sha256:4e451c39c7bb88e3cd4f87e8fc0c25d5b47695a51185d521e2241fa00486e8eb)
  • Java SDK version (if applicable): n/a, reproduced with AWS CLI v2 and @aws-sdk/client-dynamodb v3
  • How you're running Floci: Docker, via docker compose

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdynamodbAmazon DynamoDB

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions