Implementation of in-memory-emulator with external endpoint by FabianMeiswinkel · Pull Request #4786 · Azure/azure-sdk-for-rust

FabianMeiswinkel · 2026-07-15T22:49:31Z

Adds an out-of-process Cosmos DB in-memory emulator with Gateway V1 and cleartext Gateway 2.0 support.

Changes:

Adds the emulator host, configuration, management API, and RNTBD bridge.
Extends driver emulator functionality and integration coverage.
Adds non-blocking CI legs for both gateway modes.

See spec at sdk\cosmos\azure_data_cosmos_emulator\docs

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

…-008 Address reviewer feedback: key/entra auth must require HTTPS (else credentials leak over plaintext h2c), and entra mode must define the full set of JWT validation trust inputs (JWKS source, issuer, audience, allowed OIDs) rather than rely on an allow-list alone. Co-authored-by: FabianMeiswinkel <19165014+FabianMeiswinkel@users.noreply.github.com>

Normalize ADR-003 to the repository's canonical Gateway 2.0 terminology. This keeps the document aligned with the Cosmos Gateway naming guidance without changing any other ADR content. Co-authored-by: FabianMeiswinkel <19165014+FabianMeiswinkel@users.noreply.github.com>

Co-authored-by: FabianMeiswinkel <19165014+FabianMeiswinkel@users.noreply.github.com>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

FabianMeiswinkel · 2026-07-17T23:03:35Z

/azp run rust - cosmos - weekly

azure-pipelines · 2026-07-17T23:03:54Z

Azure Pipelines: Successfully started running 1 pipeline(s).

FabianMeiswinkel · 2026-07-17T23:13:55Z

/azp run rust - cosmos - weekly

azure-pipelines · 2026-07-17T23:14:13Z

Azure Pipelines: Successfully started running 1 pipeline(s).

FabianMeiswinkel · 2026-07-18T10:08:10Z

/azp run rust - cosmos - weekly

azure-pipelines · 2026-07-18T10:08:29Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Copilot

Review details

Comments suppressed due to low confidence (3)

sdk/cosmos/azure_data_cosmos_driver/tests/in_memory_emulator_tests/split_merge.rs:186

This reads from the source region, where the item was written before replication, so the test still passes if the deferred replication is lost during the split. Read from ctx.west_url as the merge test does; that makes the assertion actually verify replication into the target region.
sdk/cosmos/ci.yml:74
The PR promises non-blocking emulator legs, but ci.tests.yml:28-31 defaults jobs to blocking unless their matrix sets ContinueOnError. The new in-memory matrix omits it, and this PR also removes it from the existing vnext matrix, so emulator failures now block the build. Restore "ContinueOnError": ["true"] in both emulator matrices (or update the stated intent if blocking is deliberate).
sdk/cosmos/azure_data_cosmos_driver/src/driver/transport/gateway_v2_dispatch.rs:501
These mappings make additional Gateway 2.0 metadata visible through public response headers, which is customer-visible driver behavior. The Cosmos changelog rule requires every customer-visible change to be recorded (sdk/cosmos/.github/instructions/cosmos.changelog.instructions.md:6-12), but this PR has no unreleased changelog entry; add one describing the newly surfaced Gateway 2.0 metadata.

    if let Some(item_count) = response.item_count {
        headers.insert(response_header_names::ITEM_COUNT, item_count.to_string());
    }
    if let Some(query_metrics) = response.query_metrics {
        headers.insert(response_header_names::QUERY_METRICS, query_metrics);

Files reviewed: 60/61 changed files
Comments generated: 0 new
Review effort level: Medium

FabianMeiswinkel · 2026-07-20T08:17:01Z

/azp run rust - cosmos - weekly

azure-pipelines · 2026-07-20T08:17:21Z

Azure Pipelines: Successfully started running 1 pipeline(s).

Copilot

Review details

Comments suppressed due to low confidence (1)

sdk/cosmos/ci.yml:75

The PR description promises non-blocking emulator CI legs, but this change explicitly makes both hosted-emulator jobs blocking and also moves the vNext leg out of AdditionalMatrixConfigs after removing its ContinueOnError setting. This changes release gating rather than merely adding non-blocking coverage. Please either restore the non-blocking matrix configuration or update the stated requirement if blocking CI is intentional.

Files reviewed: 61/62 changed files
Comments generated: 0 new
Review effort level: Medium

heaths · 2026-07-20T18:31:34Z

You might consider naming this AGENTS.md and putting it in the crate root. Gets discovered automatically. I used to do this whole plan.md thing as well, but you have to remember to refer to it each time...unless that's your intention.

heaths · 2026-07-20T18:35:17Z

+complete design (configuration schema, management REST API, Gateway 2.0
+support, and CI integration).
+
+`publish = false` — this crate is never published to crates.io.


This seems like something that should be in AGENTS.md, not a customer-focused README.md.

heaths · 2026-07-20T18:37:19Z

+cargo build -p azure_data_cosmos_emulator
+./target/debug/azure_data_cosmos_emulator --config sdk/cosmos/azure_data_cosmos_emulator/config/ci-gateway-v1.json


Suggestion: depends on PWD, path delimiter (depending on your shell), etc. As long as you're config path is normalizing paths (it should), just the following would work more universally:

Suggested change

cargo build -p azure_data_cosmos_emulator

./target/debug/azure_data_cosmos_emulator --config sdk/cosmos/azure_data_cosmos_emulator/config/ci-gateway-v1.json

cargo run -p azure_data_cosmos_emulator -- --config sdk/cosmos/azure_data_cosmos_emulator/config/ci-gateway-v1.json

heaths · 2026-07-20T18:38:04Z

+`"emulator_inmemory_gateway_v2"` for Gateway 2.0-specific tests). See
+`sdk/cosmos/eng/scripts/Invoke-CosmosTestSetup.ps1`
+for how CI builds, starts, and health-checks the host process before running
+those suites.


Even though you're not publishing, the README should still probably have the security, license, etc. footers as other READMEs generally do.

FabianMeiswinkel and others added 30 commits July 14, 2026 12:24

Plan for in-memory-emulator with its own http(s) endpoint

65bc4b9

Update plan.md

3ce86e8

Update plan.md

ee5fc91

Potential fix for pull request finding

7720563

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

9dc8ec4

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

8863aa9

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

3c2b1a3

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

ca92a9f

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

6795fe1

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

docs: normalize Gateway 2.0 naming

80b550e

Co-authored-by: FabianMeiswinkel <19165014+FabianMeiswinkel@users.noreply.github.com>

Adjust split/merge plan contract

2399f36

Co-authored-by: FabianMeiswinkel <19165014+FabianMeiswinkel@users.noreply.github.com>

Normalize Gateway 2.0 plan naming

bf4c5df

Co-authored-by: FabianMeiswinkel <19165014+FabianMeiswinkel@users.noreply.github.com>

Potential fix for pull request finding

909c6cd

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

e15a002

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

b81eb51

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

e006e99

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

67e5fb2

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

2ecd0e0

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Potential fix for pull request finding

94af359

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Updating ADRs

2dd214a

Unlinking links

5be250e

Updating ADRs

f3577b8

Addressing code review comments

37f0049

Update 007_gateway_v2_rntbd.md

6332052

Addressing code review feedback

d9e1ad8

Update 002_separate_host_crate.md

7216572

Iterating on doc fixes

a168d02

Merge branch 'main' into users/fabianm/ExternalInMemoryEmulator

561c820

Copilot started reviewing on behalf of FabianMeiswinkel July 17, 2026 22:56 View session

Update validation.rs

99498e0

This comment was marked as spam.

Sign in to view

Copilot AI review requested due to automatic review settings July 17, 2026 23:06

Copilot started reviewing on behalf of FabianMeiswinkel July 17, 2026 23:07 View session

Fix accidental yml changes

95eda00

This comment was marked as spam.

Sign in to view

Copilot AI review requested due to automatic review settings July 17, 2026 23:13

Copilot started reviewing on behalf of FabianMeiswinkel July 17, 2026 23:15 View session

This comment was marked as spam.

Sign in to view

FabianMeiswinkel added 3 commits July 18, 2026 09:38

Update gateway_v2_dispatch.rs

33b985e

Update dual_backend.rs

ce144b2

Update test_client.rs

e59c8a1

Copilot AI review requested due to automatic review settings July 18, 2026 10:07

Copilot started reviewing on behalf of FabianMeiswinkel July 18, 2026 10:08 View session

Copilot AI reviewed Jul 18, 2026

View reviewed changes

Update validation.rs

32cc9bc

Copilot AI review requested due to automatic review settings July 20, 2026 08:16

Copilot started reviewing on behalf of FabianMeiswinkel July 20, 2026 08:16 View session

Copilot AI reviewed Jul 20, 2026

View reviewed changes

analogrelay moved this from In Progress to Needs Review in CosmosDB Rust SDK and Driver Jul 20, 2026

heaths reviewed Jul 20, 2026

View reviewed changes

		cargo build -p azure_data_cosmos_emulator
		./target/debug/azure_data_cosmos_emulator --config sdk/cosmos/azure_data_cosmos_emulator/config/ci-gateway-v1.json

	cargo build -p azure_data_cosmos_emulator
	./target/debug/azure_data_cosmos_emulator --config sdk/cosmos/azure_data_cosmos_emulator/config/ci-gateway-v1.json
	cargo run -p azure_data_cosmos_emulator -- --config sdk/cosmos/azure_data_cosmos_emulator/config/ci-gateway-v1.json

Uh oh!

Conversation

FabianMeiswinkel commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

FabianMeiswinkel commented Jul 17, 2026

Uh oh!

azure-pipelines Bot commented Jul 17, 2026

Uh oh!

This comment was marked as spam.

Uh oh!

This comment was marked as spam.

Uh oh!

FabianMeiswinkel commented Jul 17, 2026

Uh oh!

azure-pipelines Bot commented Jul 17, 2026

Uh oh!

This comment was marked as spam.

Uh oh!

FabianMeiswinkel commented Jul 18, 2026

Uh oh!

azure-pipelines Bot commented Jul 18, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

FabianMeiswinkel commented Jul 20, 2026

Uh oh!

azure-pipelines Bot commented Jul 20, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

heaths Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

heaths Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

heaths Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

heaths Jul 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

FabianMeiswinkel commented Jul 15, 2026 •

edited

Loading