Skip to content

Support DISTINCT queries #4754

Description

@tvaron3

Feature Summary

Add ordered and unordered cross-partition DISTINCT execution to the Cosmos DB Rust driver.

Feature Description

Implement structural, type-aware deduplication that works across partitions and pages and can be reused by GROUP BY.

Validation and acceptance criteria:

  • Implement unordered DISTINCT using structural, type-aware JSON hashing and equality.
  • Bound retained deduplication state and define an explicit continuation contract rather than silently emitting incomplete state.
  • Implement ordered DISTINCT as adjacency deduplication over streaming ORDER BY output.
  • Reuse the structural hash/equality implementation for GROUP BY keys.
  • Cover every JSON type, object and array structural equality, duplicate values across partitions and pages, and ordered versus unordered behavior.
  • Cover resume and split behavior according to the explicit policy, including rejection where safe continuation is unsupported.

Use Case

Applications need SQL DISTINCT to remove structurally equal Cosmos JSON values globally, not merely within individual partition pages.

Alternatives

String serialization or per-page sets do not provide type-aware structural equality and allow duplicates to leak across partitions or continuation boundaries.

Additional Context

Implement under sdk/cosmos/azure_data_cosmos_driver/src/driver/dataflow/. This depends on the parent issue's envelopes, buffering limits, continuation policy, and reusable tests; ordered DISTINCT additionally depends on streaming ORDER BY. Cross-SDK references: .NET DistinctQueryPipelineStage, DistinctMap, DistinctHash, and DistinctQueryTests; Java DistinctDocumentQueryExecutionContext, OrderedDistinctMap, UnorderedDistinctMap, DistinctHash, and DistinctContinuationToken.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ClientThis issue points to a problem in the data-plane of the library.CosmosThe azure_cosmos crate

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions