Skip to content

Commit 1fc0630

Browse files
committed
Auto merge of #7977 - ehuss:unit-graph, r=alexcrichton
Add unit-graph JSON output. This adds a `--unit-graph` flag that will emit a JSON object of Cargo's internal build graph. See unstable.md for more details. The primary motivator is to provide an accurate picture of which features are set. With the new feature resolver it is not possible to properly represent the features in the `cargo metadata` structure, because features are no longer unified. Also, features selected depend on the command, and exactly which packages are being built. To handle that in `cargo metadata`, it would need to add a "mode" flag, and a superset of flags for all build commands (test, check, build, etc.). To me that seemed like a difficult path to take. This may also be helpful for making visualizations of the true dependencies. `cargo metadata` doesn't show the intra-package dependencies like build scripts or test units, and walking the `cargo metadata` graph correctly isn't always obvious. This initial concept exposes almost all of the fields. That may be a little too much, but I imagine we could always trim it before stabilizing. This structure also has a high risk of being unstable, since it has a good chance of changing form in the future. I figure that can be addressed with documentation emphasizing that it may change and we may not always provide backwards-compatibility (though we will try if it is not too much burden). This could also potentially be extended in the future to include things like artifact paths, or "freshness", if we'd like to.
2 parents 3d39211 + 4107872 commit 1fc0630

24 files changed

+540
-37
lines changed

src/bin/cargo/commands/bench.rs

+1
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ pub fn cli() -> App {
4444
"no-fail-fast",
4545
"Run all benchmarks regardless of failure",
4646
))
47+
.arg_unit_graph()
4748
.after_help(
4849
"\
4950
The benchmark filtering argument BENCHNAME and all the arguments following the

src/bin/cargo/commands/build.rs

+1
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@ pub fn cli() -> App {
4141
.arg_manifest_path()
4242
.arg_message_format()
4343
.arg_build_plan()
44+
.arg_unit_graph()
4445
.after_help(
4546
"\
4647
All packages in the workspace are built if the `--workspace` flag is supplied. The

src/bin/cargo/commands/check.rs

+1
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ pub fn cli() -> App {
3333
.arg_target_dir()
3434
.arg_manifest_path()
3535
.arg_message_format()
36+
.arg_unit_graph()
3637
.after_help(
3738
"\
3839
If the `--package` argument is given, then SPEC is a package ID specification

src/bin/cargo/commands/doc.rs

+1
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ pub fn cli() -> App {
3030
.arg_target_dir()
3131
.arg_manifest_path()
3232
.arg_message_format()
33+
.arg_unit_graph()
3334
.after_help(
3435
"\
3536
By default the documentation for the local package and all dependencies is

src/bin/cargo/commands/run.rs

+1
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ pub fn cli() -> App {
2424
.arg_target_dir()
2525
.arg_manifest_path()
2626
.arg_message_format()
27+
.arg_unit_graph()
2728
.after_help(
2829
"\
2930
If neither `--bin` nor `--example` are given, then if the package only has one

src/bin/cargo/commands/rustc.rs

+1
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ pub fn cli() -> App {
2929
.arg_target_dir()
3030
.arg_manifest_path()
3131
.arg_message_format()
32+
.arg_unit_graph()
3233
.after_help(
3334
"\
3435
The specified target for the current package (or package specified by SPEC if

src/bin/cargo/commands/rustdoc.rs

+1
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,7 @@ pub fn cli() -> App {
3333
.arg_target_dir()
3434
.arg_manifest_path()
3535
.arg_message_format()
36+
.arg_unit_graph()
3637
.after_help(
3738
"\
3839
The specified target for the current package (or package specified by SPEC if

src/bin/cargo/commands/test.rs

+1
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@ pub fn cli() -> App {
5454
.arg_target_dir()
5555
.arg_manifest_path()
5656
.arg_message_format()
57+
.arg_unit_graph()
5758
.after_help(
5859
"\
5960
The test filtering argument TESTNAME and all the arguments following the

src/cargo/core/compiler/build_config.rs

+3
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ pub struct BuildConfig {
2222
pub force_rebuild: bool,
2323
/// Output a build plan to stdout instead of actually compiling.
2424
pub build_plan: bool,
25+
/// Output the unit graph to stdout instead of actually compiling.
26+
pub unit_graph: bool,
2527
/// An optional override of the rustc process for primary units
2628
pub primary_unit_rustc: Option<ProcessBuilder>,
2729
pub rustfix_diagnostic_server: RefCell<Option<RustfixDiagnosticServer>>,
@@ -79,6 +81,7 @@ impl BuildConfig {
7981
message_format: MessageFormat::Human,
8082
force_rebuild: false,
8183
build_plan: false,
84+
unit_graph: false,
8285
primary_unit_rustc: None,
8386
rustfix_diagnostic_server: RefCell::new(None),
8487
})

src/cargo/core/compiler/compile_kind.rs

+13-1
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ use std::path::Path;
99
/// compilations, where cross compilations happen at the request of `--target`
1010
/// and host compilations happen for things like build scripts and procedural
1111
/// macros.
12-
#[derive(PartialEq, Eq, Hash, Debug, Clone, Copy, PartialOrd, Ord, Serialize)]
12+
#[derive(PartialEq, Eq, Hash, Debug, Clone, Copy, PartialOrd, Ord)]
1313
pub enum CompileKind {
1414
/// Attached to a unit that is compiled for the "host" system or otherwise
1515
/// is compiled without a `--target` flag. This is used for procedural
@@ -41,6 +41,18 @@ impl CompileKind {
4141
}
4242
}
4343

44+
impl serde::ser::Serialize for CompileKind {
45+
fn serialize<S>(&self, s: S) -> Result<S::Ok, S::Error>
46+
where
47+
S: serde::ser::Serializer,
48+
{
49+
match self {
50+
CompileKind::Host => None::<&str>.serialize(s),
51+
CompileKind::Target(t) => Some(t.name).serialize(s),
52+
}
53+
}
54+
}
55+
4456
/// Abstraction for the representation of a compilation target that Cargo has.
4557
///
4658
/// Compilation targets are one of two things right now:

src/cargo/core/compiler/context/mod.rs

+1-1
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ use super::custom_build::{self, BuildDeps, BuildScriptOutputs, BuildScripts};
1616
use super::fingerprint::Fingerprint;
1717
use super::job_queue::JobQueue;
1818
use super::layout::Layout;
19-
use super::unit_dependencies::{UnitDep, UnitGraph};
19+
use super::unit_graph::{UnitDep, UnitGraph};
2020
use super::{BuildContext, Compilation, CompileKind, CompileMode, Executor, FileFlavor};
2121

2222
mod compilation_files;

src/cargo/core/compiler/fingerprint.rs

+1-1
Original file line numberDiff line numberDiff line change
@@ -202,7 +202,7 @@ use serde::de;
202202
use serde::ser;
203203
use serde::{Deserialize, Serialize};
204204

205-
use crate::core::compiler::unit_dependencies::UnitDep;
205+
use crate::core::compiler::unit_graph::UnitDep;
206206
use crate::core::{InternedString, Package};
207207
use crate::util;
208208
use crate::util::errors::{CargoResult, CargoResultExt};

src/cargo/core/compiler/links.rs

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
use super::unit_dependencies::UnitGraph;
1+
use super::unit_graph::UnitGraph;
22
use crate::core::{PackageId, Resolve};
33
use crate::util::errors::CargoResult;
44
use std::collections::{HashMap, HashSet};

src/cargo/core/compiler/mod.rs

+2-1
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ pub mod standard_lib;
1515
mod timings;
1616
mod unit;
1717
pub mod unit_dependencies;
18+
pub mod unit_graph;
1819

1920
use std::env;
2021
use std::ffi::{OsStr, OsString};
@@ -38,7 +39,7 @@ pub use self::job::Freshness;
3839
use self::job::{Job, Work};
3940
use self::job_queue::{JobQueue, JobState};
4041
use self::output_depinfo::output_depinfo;
41-
use self::unit_dependencies::UnitDep;
42+
use self::unit_graph::UnitDep;
4243
pub use crate::core::compiler::unit::{Unit, UnitInterner};
4344
use crate::core::manifest::TargetSourcePath;
4445
use crate::core::profiles::{Lto, PanicStrategy, Profile};

src/cargo/core/compiler/unit_dependencies.rs

+1-19
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515
//! (for example, with and without tests), so we actually build a dependency
1616
//! graph of `Unit`s, which capture these properties.
1717
18+
use crate::core::compiler::unit_graph::{UnitDep, UnitGraph};
1819
use crate::core::compiler::Unit;
1920
use crate::core::compiler::{BuildContext, CompileKind, CompileMode};
2021
use crate::core::dependency::DepKind;
@@ -27,25 +28,6 @@ use crate::CargoResult;
2728
use log::trace;
2829
use std::collections::{HashMap, HashSet};
2930

30-
/// The dependency graph of Units.
31-
pub type UnitGraph<'a> = HashMap<Unit<'a>, Vec<UnitDep<'a>>>;
32-
33-
/// A unit dependency.
34-
#[derive(Debug, Clone, Hash, Eq, PartialEq, PartialOrd, Ord)]
35-
pub struct UnitDep<'a> {
36-
/// The dependency unit.
37-
pub unit: Unit<'a>,
38-
/// The purpose of this dependency (a dependency for a test, or a build
39-
/// script, etc.).
40-
pub unit_for: UnitFor,
41-
/// The name the parent uses to refer to this dependency.
42-
pub extern_crate_name: InternedString,
43-
/// Whether or not this is a public dependency.
44-
pub public: bool,
45-
/// If `true`, the dependency should not be added to Rust's prelude.
46-
pub noprelude: bool,
47-
}
48-
4931
/// Collection of stuff used while creating the `UnitGraph`.
5032
struct State<'a, 'cfg> {
5133
bcx: &'a BuildContext<'a, 'cfg>,

src/cargo/core/compiler/unit_graph.rs

+121
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
use crate::core::compiler::Unit;
2+
use crate::core::compiler::{CompileKind, CompileMode};
3+
use crate::core::profiles::{Profile, UnitFor};
4+
use crate::core::{nightly_features_allowed, InternedString, PackageId, Target};
5+
use crate::util::CargoResult;
6+
use std::collections::HashMap;
7+
use std::io::Write;
8+
9+
/// The dependency graph of Units.
10+
pub type UnitGraph<'a> = HashMap<Unit<'a>, Vec<UnitDep<'a>>>;
11+
12+
/// A unit dependency.
13+
#[derive(Debug, Clone, Hash, Eq, PartialEq, PartialOrd, Ord)]
14+
pub struct UnitDep<'a> {
15+
/// The dependency unit.
16+
pub unit: Unit<'a>,
17+
/// The purpose of this dependency (a dependency for a test, or a build
18+
/// script, etc.).
19+
pub unit_for: UnitFor,
20+
/// The name the parent uses to refer to this dependency.
21+
pub extern_crate_name: InternedString,
22+
/// Whether or not this is a public dependency.
23+
pub public: bool,
24+
/// If `true`, the dependency should not be added to Rust's prelude.
25+
pub noprelude: bool,
26+
}
27+
28+
const VERSION: u32 = 1;
29+
30+
#[derive(serde::Serialize)]
31+
struct SerializedUnitGraph<'a> {
32+
version: u32,
33+
units: Vec<SerializedUnit<'a>>,
34+
roots: Vec<usize>,
35+
}
36+
37+
#[derive(serde::Serialize)]
38+
struct SerializedUnit<'a> {
39+
pkg_id: PackageId,
40+
target: &'a Target,
41+
profile: &'a Profile,
42+
platform: CompileKind,
43+
mode: CompileMode,
44+
features: &'a Vec<InternedString>,
45+
#[serde(skip_serializing_if = "std::ops::Not::not")] // hide for unstable build-std
46+
is_std: bool,
47+
dependencies: Vec<SerializedUnitDep>,
48+
}
49+
50+
#[derive(serde::Serialize)]
51+
struct SerializedUnitDep {
52+
index: usize,
53+
extern_crate_name: InternedString,
54+
// This is only set on nightly since it is unstable.
55+
#[serde(skip_serializing_if = "Option::is_none")]
56+
public: Option<bool>,
57+
// This is only set on nightly since it is unstable.
58+
#[serde(skip_serializing_if = "Option::is_none")]
59+
noprelude: Option<bool>,
60+
// Intentionally not including `unit_for` because it is a low-level
61+
// internal detail that is mostly used for building the graph.
62+
}
63+
64+
pub fn emit_serialized_unit_graph(
65+
root_units: &[Unit<'_>],
66+
unit_graph: &UnitGraph<'_>,
67+
) -> CargoResult<()> {
68+
let is_nightly = nightly_features_allowed();
69+
let mut units: Vec<(&Unit<'_>, &Vec<UnitDep<'_>>)> = unit_graph.iter().collect();
70+
units.sort_unstable();
71+
// Create a map for quick lookup for dependencies.
72+
let indices: HashMap<&Unit<'_>, usize> = units
73+
.iter()
74+
.enumerate()
75+
.map(|(i, val)| (val.0, i))
76+
.collect();
77+
let roots = root_units.iter().map(|root| indices[root]).collect();
78+
let ser_units = units
79+
.iter()
80+
.map(|(unit, unit_deps)| {
81+
let dependencies = unit_deps
82+
.iter()
83+
.map(|unit_dep| {
84+
// https://github.com/rust-lang/rust/issues/64260 when stabilized.
85+
let (public, noprelude) = if is_nightly {
86+
(Some(unit_dep.public), Some(unit_dep.noprelude))
87+
} else {
88+
(None, None)
89+
};
90+
SerializedUnitDep {
91+
index: indices[&unit_dep.unit],
92+
extern_crate_name: unit_dep.extern_crate_name,
93+
public,
94+
noprelude,
95+
}
96+
})
97+
.collect();
98+
SerializedUnit {
99+
pkg_id: unit.pkg.package_id(),
100+
target: unit.target,
101+
profile: &unit.profile,
102+
platform: unit.kind,
103+
mode: unit.mode,
104+
features: &unit.features,
105+
is_std: unit.is_std,
106+
dependencies,
107+
}
108+
})
109+
.collect();
110+
let s = SerializedUnitGraph {
111+
version: VERSION,
112+
units: ser_units,
113+
roots,
114+
};
115+
116+
let stdout = std::io::stdout();
117+
let mut lock = stdout.lock();
118+
serde_json::to_writer(&mut lock, &s)?;
119+
write!(lock, "\n")?;
120+
Ok(())
121+
}

src/cargo/core/profiles.rs

+17-2
Original file line numberDiff line numberDiff line change
@@ -374,6 +374,7 @@ impl Profiles {
374374
/// times).
375375
pub fn get_profile_run_custom_build(&self, for_unit_profile: &Profile) -> Profile {
376376
let mut result = Profile::default();
377+
result.name = for_unit_profile.name;
377378
result.root = for_unit_profile.root;
378379
result.debuginfo = for_unit_profile.debuginfo;
379380
result.opt_level = for_unit_profile.opt_level;
@@ -578,10 +579,11 @@ pub enum ProfileRoot {
578579

579580
/// Profile settings used to determine which compiler flags to use for a
580581
/// target.
581-
#[derive(Clone, Copy, Eq, PartialOrd, Ord)]
582+
#[derive(Clone, Copy, Eq, PartialOrd, Ord, serde::Serialize)]
582583
pub struct Profile {
583584
pub name: InternedString,
584585
pub opt_level: InternedString,
586+
#[serde(skip)] // named profiles are unstable
585587
pub root: ProfileRoot,
586588
pub lto: Lto,
587589
// `None` means use rustc default.
@@ -743,8 +745,21 @@ pub enum Lto {
743745
Named(InternedString),
744746
}
745747

748+
impl serde::ser::Serialize for Lto {
749+
fn serialize<S>(&self, s: S) -> Result<S::Ok, S::Error>
750+
where
751+
S: serde::ser::Serializer,
752+
{
753+
match self {
754+
Lto::Bool(b) => b.to_string().serialize(s),
755+
Lto::Named(n) => n.serialize(s),
756+
}
757+
}
758+
}
759+
746760
/// The `panic` setting.
747-
#[derive(Clone, Copy, PartialEq, Eq, Debug, Hash, PartialOrd, Ord)]
761+
#[derive(Clone, Copy, PartialEq, Eq, Debug, Hash, PartialOrd, Ord, serde::Serialize)]
762+
#[serde(rename_all = "lowercase")]
748763
pub enum PanicStrategy {
749764
Unwind,
750765
Abort,

src/cargo/ops/cargo_compile.rs

+6
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ use std::sync::Arc;
3030

3131
use crate::core::compiler::standard_lib;
3232
use crate::core::compiler::unit_dependencies::build_unit_dependencies;
33+
use crate::core::compiler::unit_graph;
3334
use crate::core::compiler::{BuildConfig, BuildContext, Compilation, Context};
3435
use crate::core::compiler::{CompileKind, CompileMode, RustcTargetData, Unit};
3536
use crate::core::compiler::{DefaultExecutor, Executor, UnitInterner};
@@ -490,6 +491,11 @@ pub fn compile_ws<'a>(
490491
&std_roots,
491492
)?;
492493

494+
if bcx.build_config.unit_graph {
495+
unit_graph::emit_serialized_unit_graph(&units, &unit_dependencies)?;
496+
return Ok(Compilation::new(&bcx, build_config.requested_kind)?);
497+
}
498+
493499
let ret = {
494500
let _p = profile::start("compiling");
495501
let cx = Context::new(config, &bcx, unit_dependencies, build_config.requested_kind)?;

src/cargo/util/command_prelude.rs

+10
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,10 @@ pub trait AppExt: Sized {
150150
))
151151
}
152152

153+
fn arg_unit_graph(self) -> Self {
154+
self._arg(opt("unit-graph", "Output build graph in JSON (unstable)").hidden(true))
155+
}
156+
153157
fn arg_new_opts(self) -> Self {
154158
self._arg(
155159
opt(
@@ -438,11 +442,17 @@ pub trait ArgMatchesExt {
438442
build_config.message_format = message_format.unwrap_or(MessageFormat::Human);
439443
build_config.requested_profile = self.get_profile_name(config, "dev", profile_checking)?;
440444
build_config.build_plan = self._is_present("build-plan");
445+
build_config.unit_graph = self._is_present("unit-graph");
441446
if build_config.build_plan {
442447
config
443448
.cli_unstable()
444449
.fail_if_stable_opt("--build-plan", 5579)?;
445450
};
451+
if build_config.unit_graph {
452+
config
453+
.cli_unstable()
454+
.fail_if_stable_opt("--unit-graph", 8002)?;
455+
}
446456

447457
let opts = CompileOptions {
448458
config,

0 commit comments

Comments
 (0)