Close the gaps between what Rust source contains and what the graph records - #384
Merged
Conversation
Measured over 4,000 files of real crate source, the Rust extractor was blind to whole categories of declaration: 10,094 type aliases, 4,206 modules, 1,148 macro_rules! definitions, 416 extern crate declarations and 38,973 extern blocks produced zero graph nodes between them. Rust also does not parse macro arguments as expressions — every macro_invocation carries an opaque token_tree — so 68,551 invocations concealed 65,525 call sites against the 238,457 the extractor found everywhere else. Test suites suffered worst: 42,511 of those invocations are assert*!, so the call a #[test] actually exercises was precisely the one never recorded. A trailing ::<T> likewise wraps a callee in a generic_function node that none of the three call patterns matched, hiding another 5,301 sites such as xs.collect::<Vec<_>>(). Adds query patterns and emitters for mod (KindPackage, following the C++/C# namespace precedent — KindModule is reserved for package-manager dependencies), type aliases, unions and their fields, macro_rules!, extern-block functions with their ABI, extern crate, and associated types. Recovers macro-body call sites by walking the token stream for the three shapes Rust can spell, labelling each with via_macro so a consumer wanting only grammar-certain edges can filter; pattern and cfg-predicate macros are excluded so matches!(x, Some(_)) does not report a call to Some. Two same-named types in one file previously collapsed into one node and the loser's fields were re-parented onto the survivor, which then reported members it does not have. Types, enums, traits and unions now route through disambiguateID like methods already did, and members bind to the id actually minted for their container rather than reconstructing it by name. Items carry scope_mod so the two remain distinguishable. Also stamps real signatures in place of the "fn name(...)" placeholder every function carried, records async/unsafe/const/extern qualifiers, gives free functions the return_type meta methods already had, stops rustCollectAttributes from discarding an attribute stack when a comment sits between it and its item, and fixes Result error extraction, which split on every comma (making Result<(u8, u16), E> emit a throws edge to the literal target "u16)") and anchored on the last "::" in the string (so std::result::Result<T, io::Error> emitted no edge at all).
Every const_item and static_item was emitted at file scope as a bare
KindVariable with empty Meta. Two consequences:
An associated const belongs to its impl, so `impl A { const LIMIT }` and
`impl B { const LIMIT }` both minted `<file>::LIMIT`. The dedup guard
dropped the second outright — no node, no edge — and the survivor
reported no owner. They now scope to the container that declares them,
carry a receiver, and link to it with EdgeMemberOf.
A const is a constant, not a variable: it now lands as KindConstant, so
the dead-code pass's IncludeConstants switch and the constant-value
sidecar apply to Rust the way they already do elsewhere. `static` stays
KindVariable and gains a `mutable` marker for `static mut`, the one form
that is genuinely mutable shared state.
Both kinds now record visibility, declared type, doc comment, enclosing
module and their attributes, which is what makes clap and serde
metadata on a const reachable at all.
`struct Meters(f64)` and `Msg::Failed(Error)` produced no member node at
all, so a newtype's wrapped type was invisible: find_usages on Error
could not see that Msg::Failed carries one, and the whole newtype idiom
contributed nothing to the type graph.
Positional fields are now emitted under the name Rust uses to address
them (self.0), carrying field_type and a typed_as edge to the wrapped
type. A payload nests under its variant rather than the enum, so two
payload-carrying variants cannot collide on ".0".
Struct-shaped variants (`Rich { code: u16 }`) were dropped too — the
named-field pass only ever accepted a struct_item parent. It now
resolves struct, union and enum-variant owners alike.
Type canonicalisation stripped a tuple's parentheses and left the comma behind, and had no case at all for slices, arrays, raw pointers, or a trait-object bound list. The result was edge targets that name no possible node — unresolved::"u8, u16" from Result<(u8, u16), E>, unresolved::"[Item]" from &[Item], unresolved::"Error + Send + 'static" from Box<dyn Error + Send>. They can never resolve, and they crowd out real misses in every unresolved-reference report. Slices, arrays, raw pointers and bound lists now unwrap to the type they carry, and anything still holding punctuation after canonicalisation is rejected rather than emitted. A tuple names one type per slot, so parameters, returns and let-bindings expand into one edge per element — `-> (Item, Cfg)` now returns both at their own positions instead of neither. The same non-nominal spellings broke impl receivers: `impl Trait for &'a Buf` minted the method id `<file>::&'a Buf.show` and pointed its member_of at a node that never exists, so the method was orphaned from the type it belongs to. Reference, pointer, slice and array impl targets now reduce to the named type. The generic spelling is deliberately left alone — `Replacer<M>.replace_all` is the established id shape and rebindRustLocalGenericMemberOwners already repairs its member_of.
Neither of the two ways a Rust type implements a trait produced an
implements edge. `impl Display for Buf` emitted only a method-level
EdgeOverrides, so a marker impl carrying no methods — `impl Send for
Handle {}` — left no trace whatsoever, and #[derive(Debug, Clone)]
emitted an annotation and nothing else. Since derive is how most Rust
types acquire Debug, Clone and Serialize, find_implementations and
get_class_hierarchy were blind to the bulk of the trait hierarchy.
Both now emit EdgeImplements from the type to the trait, tagged with
whether it came from an impl block or a derive, and a path-qualified
trait (std::fmt::Display, serde::Serialize) reduces to its base name the
way the rest of the extractor already does.
Attributes written on an impl block were dropped wholesale. That is how
#[async_trait], #[pymethods] and #[wasm_bindgen] impl blocks went
unrecorded — including the file-level wasm_bindgen interop sentinel,
which only ever saw function-level attributes and so missed the form the
wasm-bindgen guide actually recommends for methods.
entrypoints.Detect had no rust case, so nothing in a Rust repo was ever stamped as a live root. fn main, every #[test], and every #[no_mangle] export have no in-crate caller by construction, so a call-graph walk found them unreachable and the dead-code analyzer reported a binary crate's entire call tree as dead. Stamps the symbols a runtime other than application code calls: fn main in a Cargo binary target (src/main.rs, src/bin/*.rs, examples/, benches/, tests/ — a fn main in a library module stays an ordinary function), async-runtime mains behind #[tokio::main] / #[actix_web::main] / #[async_std::main] / #[rocket::main], test and bench harness attributes, and FFI exports reached from another language through #[no_mangle], #[wasm_bindgen], pyo3 and napi. Declarations inside an extern block are stamped rust:ffi-import for the opposite reason: their body lives in C, so a missing definition is not a missing implementation and must not be reported as one.
A trait member carries no visibility modifier of its own; it is exactly as reachable as the trait that declares it. Reading the absent modifier as "private" meant every method of every `pub trait` was reported private, which understates the public API surface and invites the dead-code pass to over-reach on symbols that are part of a crate's contract. Trait members now inherit their trait's visibility, and a method in `impl Trait for Type` — callable by anyone who can name the trait — is public, while an inherent impl keeps ordinary per-method visibility. Trait declarations also still carried the "fn name(...)" placeholder signature after the free-function and impl-method paths had stopped using it, and gained none of the async/unsafe/const qualifiers. Both now go through the same builder as every other function.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Rust extraction looked complete —
docs/languages.mdrated it "Full" across theboard — but measuring it against real crate source told a different story. I ran
the extractor over 4,000
.rsfiles (68 MB, ~577 crates from the local Cargoregistry), counting tree-sitter node types present in the source against graph
nodes actually emitted.
Whole categories of declaration produced zero nodes:
macro_invocation(concealing call sites)extern "C"blockstypealiasesgeneric_functioncalls (xs.collect::<Vec<_>>())modmacro_rules!extern crate/associated_type/unionOthers were silently wrong, which is worse than missing — a confidently
incorrect edge is one nobody re-checks.
What changed
Eight commits, one per behaviour.
Coverage. Query patterns and emitters for
mod(indexed as a package node,following the C++/C# namespace precedent —
KindModuleis reserved forpackage-manager dependencies), type aliases, unions and their fields,
macro_rules!,extern-block functions with their ABI,extern crate,associated types, and tuple-struct / enum-variant payload fields.
Calls hidden in macros. Rust does not parse macro arguments as expressions —
every
macro_invocationcarries an opaque token tree — so any query overcall_expressionis blind to calls written inside one. 42,511 of the corpus'sinvocations are
assert*!, meaning the call a#[test]actually exercises wasprecisely the one never recorded. Recovered by a token scan over the three call
shapes Rust can spell, labelled
Meta["via_macro"]so consumers wanting onlygrammar-certain edges can filter. Pattern and cfg-predicate macros are excluded,
so
matches!(x, Some(_))does not report a call toSome.Correctness fixes, each reproduced with a probe before being fixed:
fields were re-parented onto the survivor, which then reported members it
does not have. Now routed through the existing
disambiguateIDhelper, withscope_moddistinguishing them.impl A { const LIMIT }andimpl B { const LIMIT }fought over one id and the loser vanished entirely.unresolved::"u8, u16"fromResult<(u8, u16), E>,unresolved::"[Item]"from
&[Item],unresolved::"Error + Send + 'static"from a boxed traitobject. A tuple now expands to one edge per slot.
impl Trait for &'a Bufminted the method id<file>::&'a Buf.showwith amember_ofedge to a node that never exists, orphaning the method.rustCollectAttributesstopped at the first non-attribute sibling, so acomment between an attribute stack and its item discarded every attribute
above it.
Resulterror extraction split on every comma and anchored on the last::in the string, so
Result<(u8, u16), E>emitted a throws edge to the literal"u16)"andstd::result::Result<T, io::Error>emitted none at all.privateeven when their trait waspub,understating the public API surface.
signaturewas the hardcoded placeholderfn name(...)on every Rustfunction;
async/unsafe/const/externqualifiers were never recorded.Relationships. Neither way a Rust type acquires a trait produced an
implements edge:
impl Display for Bufemitted only a method-levelEdgeOverrides(so a marker impl left no trace at all) and#[derive(...)]emitted only an annotation — despite derive being how most Rust types get
Debug,CloneandSerialize. Attributes on impl blocks were droppedwholesale, which is why the file-level
wasm_bindgensentinel missed the formthe wasm-bindgen guide recommends for methods.
Entry points.
entrypoints.Detecthad norustcase, so nothing in a Rustrepo was ever a live root and the dead-code analyzer reported every binary
crate's entire call tree as dead. Now stamps
fn mainin Cargo binary targets,async-runtime mains, test/bench harnesses, and FFI exports;
externblockdeclarations are stamped
rust:ffi-importbecause their body lives in anotherlanguage, so a missing definition is not a missing implementation.
Result
Re-measured over the same 4,000 files:
implementsedgestyped_asedgesreturnsedgesannotatededgesgo build ./...andgo test -race ./...pass. 663 lines of new tests, eachasserting the specific defect it covers.
Deliberately not in this change
The audit also surfaced problems outside the parsing layer, listed here so they
are not lost. The resolver ones matter most because they produce confidently
wrong results rather than missing ones:
use super::Xfrom a non-mod.rsmodule file resolves one directory too highand binds to the wrong file at full
ast_resolvedconfidence.alpha::helper()) bind to an unrelated module'ssame-named function.
#[path = "..."]is ignored; glob re-exports (pub use inner::*) are dropped;group imports (
use crate::{a, b}) never resolve.[workspace.dependencies]is never parsed.sqlxSQL extraction, or ORM lane for diesel/sea-orm.I also verified and rejected one reported issue: a field's doc comment bleeding
onto the next undocumented field does not reproduce.