Specifying dependency paths relative to workspace root

[bearcage]

Describe the problem you are trying to solve

tl;dr: If you have a lot of crates in a workspace, maintaining the levels of parent-directory traversal to wire them up with path dependencies is a little tedious.

Both at work and at home I use cargo a bit like a rust-only buck or bazel -- code grows in a big tree with many (often small) crates broken apart to make it easy to reuse things across the repository, and no versions. There are other languages and build systems mixed in there too, and most of our rust can build with Bazel too, but I'd prefer to stick with Cargo where possible.

When working in a workspace with many interdependent crates like this it gets pretty tedious to match up the number of ../../../ sections in path dependencies to walk back out to the root, then back in to another portion of the tree. It also means that you can't blindly copy and paste a dependency entry from one crate's Cargo.toml to another without checking/modifying the number of leading "parent directory" ../ segments to match the location of the file you're copying in to.

For a concrete example, this is taken from one of my crates:

[dependencies]
# crates.io deps elided for brevity
cli = { path = "../../common/cli", features = ["native"]}
buildkite = { path = "../../meta/buildkite"}
log_config = { path = "../../common/log_config" }
panic_config = { path = "../../common/panic_config" }
workspace = { path = "../../meta/workspace" }

which exists in a directory tree like this:

meta
├── buildkite
├── cargo_manifest
└── workspace

common
├── audio
├── cli
├── ... a dozen or more, some nested
└── time

net
├── nsframe
└── slack

projects
├── demos
│   ├── .. many others
├── lit
├── lit-* .. many subcommands, some nested
└── turbobunny

with a single Cargo.toml at the root that does nothing but anchor the workspace, configure some settings, set some excludes, etc...

I frequently need to copy a few common dependencies over to a new crate to start working on it, and if it happens to be at a different level of depth it'll require some fixing-up of the paths. The same is true for relocating a crate in the tree -- for example promoting a binary out of a bin directory to its own crate in a different subdirectory.

Describe the solution you'd like

I'd like to propose a way to specify workspace-relative dependency paths. How is a pretty bikesheddable question, so I've whipped up three ideas to get the conversation started:

1. Workspace Relative as a kind of Simple dependency

I doubt this'll be a popular idea since I suspect the way I use cargo is pretty different from most folks, but my personal favorite approach would be a new TomlDependency enum variant, or an extension to the existing Simple parser, for workspace relative paths denoted by a punctuation character such that a dependency block like this would work:

[dependencies]
buildkite = "@/meta/buildkite"
cli = "@/common/cli"
workspace = "@/meta/workspace"

I've chosen @ for illustration since it doesn't intersect with punctuation used in the version parser, but it could be anything, or even multiple characters like // or something.

2. New detailed dependency key: workspace_path

For an alternative that doesn't complicate the common case, we could add a new DetailedTomlDependency key for workspace_path, which would function similarly to the existing path key but operate relative to the workspace root.

[dependencies]
buildkite = { workspace_path = "/meta/buildkite" }
cli = { workspace_path = "/common/cli" }
workspace = { workspace_path = "/meta/workspace" }

3. Special parse mode on the existing path key

Somewhere in between the previous two ideas, we could extend the parser for the existing path key on DetailedTomlDependency to recognize a prefix as indicating a path is relative to the workspace root. Illustrated with @ again, for consistency's sake:

[dependencies]
buildkite = { path = "@/meta/buildkite" }
cli = { path = "@/common/cli" }
workspace = { path = "@/meta/workspace" }

Notes

It doesn't look like this is feasible to do with a cargo subcommand for trying-it-out, or I'd have done that first to prove the value.

I recognize this is an ease-of-use papercut at best, not a huge problem, but if I'm not the only one it bugs I'd love to smooth it over upstream. I don't have a great sense for what's a reasonable thing to do with just a feature request ticket and a PR versus writing up an RFC, so please don't hesitate to correct me if this is too large a change to try doing this way. Likewise if this is just something there's not much maintainer or community interest in: lmk. no hard feelings.

If, otoh, y'all like this idea I'm happy to do the work to make it happen.

[Eh2406]

Would your use case be helped by rust-lang/rfcs#2906?

[bearcage]

Somehow I completely missed that RFC when I searched the issue board, my apologies!

That'd entirely address what I'm after here, and then some.

[ehuss]

Closing as it looks like #8415 should address this.

[matts1]

While #8415 resolves this for the OP, I still believe there's value to be had in this feature request if you happen to use virtual manifests to organize large unrelated projects. In this example below, we have two crates which happen to have the same name, but have different paths to the crates. In this case, #8415 doesn't solve this issue, because writing cli_common.workspace = true is ambiguous.

├── common
│   └── crate
├── team1
│   ├── bin
│   │   ├── bin1
│   │   └── bin2
│   └── libs
│       └── cli_common
└── team2
    ├── bin
    │   ├── bin1
    │   └── bin2
    └── libs
        ├── cli_common
        └── path_to_other
            └── lib

You can imagine a situation where each team has a crate containing some common code for all their CLI applications (maybe a team dealing with web servers has a port flag for all their CLI applications). In this situation, it would be rather unreasonable of us to say that your crate names cannot conflict with other team's crate names, given that team1/libs/cli_common is clearly distinct from team2/libs/cli_common. Thus, we'd like to be able to refer to the crates by their absolute path.

This is a massive oversimplification of what can happen, and we have more complex cases such as path/to/crate and path/to/other/crate, which are distinguished by path rather than team. In general, however, I believe that a lack of namespacing on crate names will fail to support large-scale codebases, where it is inevitable that eventually you will get crate name conflicts within deeply nested packages.