Improving Test robustness after changing configured Regions for live tests by FabianMeiswinkel · Pull Request #49639 · Azure/azure-sdk-for-java

FabianMeiswinkel · 2026-06-25T23:43:02Z

No description provided.

FabianMeiswinkel · 2026-06-25T23:45:08Z

/azp run java - cosmos - tests

azure-pipelines · 2026-06-25T23:45:42Z

Azure Pipelines successfully started running 1 pipeline(s).

Copilot

Pull request overview

This PR improves Cosmos E2E test diagnosability by replacing terse region-contact assertions with helper methods that generate richer failure messages including preferred/service-ordered regions and diagnostics context.

Changes:

Added reusable assertion helpers for validating contacted region sets/counts with detailed failure output.
Updated multiple retry/failover E2E tests to use the new helpers for clearer intent and debugging.

Show a summary per file

File	Description
sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/rx/ClientRetryPolicyE2ETests.java	Introduces contacted-region assertion helpers and applies them across client retry policy E2E scenarios.
sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/PerPartitionCircuitBreakerE2ETests.java	Refactors diagnostics-context contacted-region asserts into helper methods with more informative failure messages.

Review details

Files reviewed: 2/2 changed files
Comments generated: 2
Review effort level: Low

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot

Review details

Files reviewed: 2/2 changed files
Comments generated: 1
Review effort level: Low

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

FabianMeiswinkel · 2026-06-26T11:11:48Z

/azp run java - cosmos - tests

azure-pipelines · 2026-06-26T11:12:17Z

Azure Pipelines successfully started running 1 pipeline(s).

FabianMeiswinkel · 2026-06-26T13:36:15Z

/azp run java - cosmos - tests

azure-pipelines · 2026-06-26T13:36:52Z

Azure Pipelines successfully started running 1 pipeline(s).

FabianMeiswinkel · 2026-07-02T15:04:07Z

Resolution — findings triaged ✅

Worked through all 9 findings one-by-one. Dispositions:

Addressed in code (this branch):

🔴 P1 — Multiplicative retry latency: RxDocumentClientImpl outer routing-map retry reduced 10 → 1 (worst-case ~60–90s → ~15s) so it no longer multiplies with InCompleteRoutingMapRetryPolicy's backoff; kept the collection-cache refresh. Added a bounded-attempts unit test that also asserts the 404/1024 terminal.
🟡 P2 — 404/1024 terminal error: documented in azure-cosmos/CHANGELOG.md.
🟡 P2 — Cross-region-on-404 rationale: added an explanatory comment in ClientRetryPolicy.
🟡 P2 — Missing unit tests: added a ClientRetryPolicy new-branch test (sub-status 0/1003/1013 + negative gating), expanded InCompleteRoutingMapRetryPolicyTest (1003/1013 + exhaustion, renamed the misleading test), and added a BulkOperationRetryPolicy 2-attempt-bound test. All pass.

Intentional / no change (won't fix):

ServerConnectionTimeout and warmup assertion loosenings — deliberate CI-stability changes for the new higher-latency regions.
PerPartitionCircuitBreaker 5-min transient-404 retry and satellite-region 404/1002 batch tolerance — deliberate, scoped tolerances for the post-create metadata race (the circuit-breaker retry skips fault-injected failures; the batch tolerance is region-guarded).
ClientMetricsTest region-metric — the loosening is a correctness fix (read/query routing can legitimately contact a non-first region).

Resolving this thread.

_{⚠️ AI-generated review — resolution recorded by the PR author.}

FabianMeiswinkel · 2026-07-02T15:53:58Z

/azp run java - cosmos - tests

FabianMeiswinkel · 2026-07-02T15:54:05Z

/azp run java - cosmos - kafka

FabianMeiswinkel · 2026-07-02T15:54:11Z

/azp run java - cosmos - spark

azure-pipelines · 2026-07-02T15:54:21Z

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines · 2026-07-02T15:54:34Z

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines · 2026-07-02T15:54:47Z

Azure Pipelines successfully started running 1 pipeline(s).

FabianMeiswinkel · 2026-07-02T19:36:05Z

/azp run java - cosmos - tests

FabianMeiswinkel · 2026-07-02T19:36:11Z

/azp run java - cosmos - kafka

FabianMeiswinkel · 2026-07-02T19:36:17Z

/azp run java - cosmos - spark

azure-pipelines · 2026-07-02T19:36:23Z

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines · 2026-07-02T19:36:27Z

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines · 2026-07-02T19:36:44Z

Azure Pipelines successfully started running 1 pipeline(s).

xinlian12

LGTM, thanks

Add more verbose asserts

c880fbf

Copilot AI review requested due to automatic review settings June 25, 2026 23:43

FabianMeiswinkel requested review from a team and kirankumarkolli as code owners June 25, 2026 23:43

github-actions Bot added the Cosmos label Jun 25, 2026

Copilot started reviewing on behalf of FabianMeiswinkel June 25, 2026 23:43 View session

Copilot AI reviewed Jun 25, 2026

View reviewed changes

Comment thread sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/rx/ClientRetryPolicyE2ETests.java Outdated

Comment thread sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/rx/ClientRetryPolicyE2ETests.java Outdated

Potential fix for pull request finding

a60a7c0

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

FabianMeiswinkel requested a review from Copilot June 26, 2026 10:50

Copilot started reviewing on behalf of FabianMeiswinkel June 26, 2026 10:51 View session

Merge branch 'main' into users/fabianm/Testfix

e9c3e35

Copilot AI mentioned this pull request Jun 26, 2026

[WIP] Fix code based on review comments #49646

Closed

Copilot AI reviewed Jun 26, 2026

View reviewed changes

Comment thread sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/rx/ClientRetryPolicyE2ETests.java Outdated

FabianMeiswinkel and others added 2 commits June 26, 2026 12:54

Potential fix for pull request finding

157a4ad

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Fix NITs in tests

3d34939

FabianMeiswinkel added 7 commits June 26, 2026 13:02

Update PerPartitionCircuitBreakerE2ETests.java

431ea68

Update ReadFeedStoredProceduresTest.java

8849c5a

Update CosmosItemTest.java

19f800a

Update StoredProcedureUpsertReplaceTest.java

1e5d841

Update FaultInjectionServerErrorRuleOnDirectTests.java

6dabaae

Update FaultInjectionServerErrorRuleOnDirectTests.java

bff6574

Update StoredProcedureQueryTest.java

9ec325c

Fix SP tests

42db971

Reacting to code review feedback

70bd3a1

FabianMeiswinkel added 2 commits July 2, 2026 15:48

Fixing remaining test failures

22c0cdc

Merge branch 'main' into users/fabianm/Testfix

fb7344d

jeet1995 reviewed Jul 2, 2026

View reviewed changes

Comment thread ...ure-cosmos/src/main/java/com/azure/cosmos/implementation/batch/BulkOperationRetryPolicy.java

jeet1995 reviewed Jul 2, 2026

View reviewed changes

Comment thread sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/ClientRetryPolicy.java

jeet1995 reviewed Jul 2, 2026

View reviewed changes

Comment thread ...azure/cosmos/test/implementation/faultinjection/FaultInjectionServerErrorResultInternal.java

jeet1995 approved these changes Jul 2, 2026

View reviewed changes

Fixing remaining test flakiness

ce20908

tvaron3 approved these changes Jul 2, 2026

View reviewed changes

jeet1995 approved these changes Jul 2, 2026

View reviewed changes

xinlian12 reviewed Jul 2, 2026

View reviewed changes

Comment thread ...rc/test/java/com/azure/cosmos/faultinjection/FaultInjectionServerErrorRuleOnDirectTests.java

xinlian12 reviewed Jul 2, 2026

View reviewed changes

Comment thread ...rc/test/java/com/azure/cosmos/faultinjection/FaultInjectionServerErrorRuleOnDirectTests.java

xinlian12 reviewed Jul 2, 2026

View reviewed changes

Comment thread sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/rx/ChangeFeedTest.java

xinlian12 reviewed Jul 2, 2026

View reviewed changes

Comment thread ...os/src/main/java/com/azure/cosmos/implementation/caches/InCompleteRoutingMapRetryPolicy.java

xinlian12 approved these changes Jul 2, 2026

View reviewed changes

FabianMeiswinkel merged commit a79d582 into Azure:main Jul 2, 2026
135 checks passed

FabianMeiswinkel mentioned this pull request Jul 6, 2026

Addressing left-over test code review comments from PR#49639 #49728

Merged

Uh oh!

Conversation

FabianMeiswinkel commented Jun 25, 2026

Uh oh!

FabianMeiswinkel commented Jun 25, 2026

Uh oh!

azure-pipelines Bot commented Jun 25, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Review details

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Review details

Uh oh!

Uh oh!

FabianMeiswinkel commented Jun 26, 2026

Uh oh!

azure-pipelines Bot commented Jun 26, 2026

Uh oh!

FabianMeiswinkel commented Jun 26, 2026

Uh oh!

azure-pipelines Bot commented Jun 26, 2026

Uh oh!

FabianMeiswinkel commented Jul 2, 2026

Resolution — findings triaged ✅

Uh oh!

FabianMeiswinkel commented Jul 2, 2026

Uh oh!

FabianMeiswinkel commented Jul 2, 2026

Uh oh!

FabianMeiswinkel commented Jul 2, 2026

Uh oh!

azure-pipelines Bot commented Jul 2, 2026

Uh oh!

azure-pipelines Bot commented Jul 2, 2026

Uh oh!

azure-pipelines Bot commented Jul 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

FabianMeiswinkel commented Jul 2, 2026

Uh oh!

FabianMeiswinkel commented Jul 2, 2026

Uh oh!

FabianMeiswinkel commented Jul 2, 2026

Uh oh!

azure-pipelines Bot commented Jul 2, 2026

Uh oh!

azure-pipelines Bot commented Jul 2, 2026

Uh oh!

azure-pipelines Bot commented Jul 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

xinlian12 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants