Fix core crate version resolution when changes are needed

[heaths]

Summary

We are hitting duplicate core crate graphs when a crate needs to use in-repo sdk/core and sdk/identity sources while most of the workspace still depends on the latest shipped crate versions.

A concrete symptom is that commands like:

cargo tree -p azure_storage_blob_test -i azure_core

show multiple azure_core versions in the same package graph, which then causes type and trait mismatches (TokenCredential, AsyncRawResponse, StatusCode, error conversions, etc.).

What we found

The split graph was introduced because different parts of the workspace resolved through different core stacks:

• shipped graph: typespec 1.1.0, typespec_client_core 1.1.0, azure_core 1.1.0, azure_identity 1.0.0
• local graph: in-repo typespec 1.2.0-beta.1, typespec_client_core 1.2.0-beta.1, azure_core 1.2.0-beta.1, azure_identity 1.1.0-beta.1

One observed case was:

• azure_storage_blob_test depended on workspace azure_core 1.1.0
• azure_core_test depended on local azure_core 1.2.0-beta.1
• azure_core_test also pulled local azure_identity 1.1.0-beta.1
• this introduced duplicate typespec* and azure_core* instances in downstream test graphs

Experiments and results

1. Update only storage crates to use local sdk/core and sdk/identity paths

Tried changing sdk/storage/* manifests to point at local sdk/core / sdk/identity crates using path dependencies.

Result:

• This resolved the duplication for azure_storage_blob_test specifically.
• It did not solve the workspace-wide problem.
• cargo build --all-features --all-targets still failed in other crates (for example, azure_security_keyvault_certificates) because they were still mixing shipped and local core graphs.

Takeaway: fixing one area by hand is not enough; the problem is cross-workspace.

2. Keep shipped workspace dependency versions and add [patch.crates-io] for core crates

Tried a separate worktree experiment with [patch.crates-io] entries for:

• typespec
• typespec_client_core
• typespec_macros
• azure_core
• azure_core_macros
• azure_identity

First attempt added patch entries only.

Result:

• This did not work by itself.
• Cargo continued resolving many consumers to the published crates because the in-repo crate versions (1.2.0-beta.1, 1.1.0-beta.1, etc.) did not satisfy the workspace's current shipped-version requirements.

Takeaway: patching alone does not help when the local crate versions do not match the requested versions.

3. Patch-based override with local crates advertising the shipped versions

In a second worktree experiment, we kept workspace dependency requirements at the currently shipped versions and added [patch.crates-io] overrides, but also changed the in-repo patched crates to advertise those same shipped versions.

Examples of the version alignment used in the experiment:

• typespec -> 1.1.0
• typespec_client_core -> 1.1.0
• typespec_macros -> 1.0.0
• azure_core -> 1.1.0
• azure_core_macros -> 1.0.0
• azure_identity -> 1.0.0

We also kept unpublished test-only crates as local workspace path dependencies.

Result:

• azure_storage_blob_test resolved to a single local azure_core graph.
• azure_security_keyvault_certificates resolved through the same local core stack instead of mixing local and published crates.
• cargo build --all-features --all-targets succeeded in the experiment.

Takeaway: this is the first approach that made the whole workspace build cleanly in the test setup.

Important constraint

We do not want to ship azure_core_test as a supported published crate just to make this work.

The good news is that the successful patch-based experiment does not require publishing azure_core_test.

azure_core_test and azure_core_test_macros can remain unpublished local workspace crates. The important requirement is that the crates resolved from crates.io and overridden by [patch.crates-io] advertise versions that satisfy the workspace dependency requirements.

Recommendation

The most promising direction looks like this:

1. Keep normal workspace dependency requirements pinned to the latest shipped core crate versions.
2. Keep azure_core_test and azure_core_test_macros as unpublished local workspace crates.
3. Add a coordinated [patch.crates-io] strategy for the published core stack (typespec*, azure_core*, azure_identity, and any other related crates that participate in the same public type graph).
4. Define a clear repo strategy for when local development should temporarily make the in-repo crates advertise the currently shipped versions so Cargo resolves the entire workspace to one local graph.

Open questions:

• Should this be a documented local-development workflow only?
• Should the version-alignment step be automated by a script/tool?
• Which crates must always participate together in the patched graph to avoid duplicate public types?

This issue is for tracking the resolution strategy and documenting a workspace-wide approach rather than continuing to patch individual service crates by hand.

[heaths]

Option 3 would require we also change our publishing pipeline to bump versions just prior to publishing, which I'm not crazy about.

[heaths]

Alternate ideas

Remove workspace

Ultimately, the problem is the pervasiveness of the workspace. A problem - if even a problem - with one crate affects the whole workspace. If we removed the workspace, that would isolate problems. And then the problems described above might not even be problems because, for example, Storage could take a path dependency on core crates without affecting anyone else. Cross-crate dependency cycles might still be a problem, though, so it doesn't eliminate the problem entirely but at least then we don't have to worry about them impacting the whole workspace.

The downside is that all metadata, lints, etc. have to be copied. We'd also want to copy the root Cargo.toml initially so that we keep the same locked versions of transitive dependencies, but a build of each should fix that. But we do lose centralized management entirely. Fortunately, tools like dependabot already handle this, and that's already the case for most other Azure SDK languages.

Variant

Would it be possible to retain the workspace to inherit metadata, lints, etc., but separate dependencies? Worth trying, but I imagine it wouldn't work because then we're back to one crate affecting all.

Release branch

We require any changes to core or that depend on changes to core to be in a release/* branch for vNext. Once published, we can merge that into main after changing the dependencies to the shipped version.

The main problem here is that we disable a true merge. We'd have to squash merge and lose any history. Maybe we could use a branch policy to allow only maintainers to merge but no one else...if GitHub supports that. Care would need to be taken anyway because we have to change the dependencies from path to version for published versions first before merging of any sort. This should be an infrequent occurrence and become more infrequent over time.