Skip to content

Close the gaps between what Rust source contains and what the graph records - #384

Merged
zzet merged 8 commits into
mainfrom
feat/rust-parser-coverage
Jul 28, 2026
Merged

Close the gaps between what Rust source contains and what the graph records#384
zzet merged 8 commits into
mainfrom
feat/rust-parser-coverage

Conversation

@zzet

@zzet zzet commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Why

Rust extraction looked complete — docs/languages.md rated it "Full" across the
board — but measuring it against real crate source told a different story. I ran
the extractor over 4,000 .rs files (68 MB, ~577 crates from the local Cargo
registry), counting tree-sitter node types present in the source against graph
nodes actually emitted.

Whole categories of declaration produced zero nodes:

construct present in corpus nodes emitted
macro_invocation (concealing call sites) 68,551 0
extern "C" blocks 38,973 0
type aliases 10,094 0
generic_function calls (xs.collect::<Vec<_>>()) 5,301 0
mod 4,206 0
macro_rules! 1,148 0
extern crate / associated_type / union 416 / 225 / 162 0

Others were silently wrong, which is worse than missing — a confidently
incorrect edge is one nobody re-checks.

What changed

Eight commits, one per behaviour.

Coverage. Query patterns and emitters for mod (indexed as a package node,
following the C++/C# namespace precedent — KindModule is reserved for
package-manager dependencies), type aliases, unions and their fields,
macro_rules!, extern-block functions with their ABI, extern crate,
associated types, and tuple-struct / enum-variant payload fields.

Calls hidden in macros. Rust does not parse macro arguments as expressions —
every macro_invocation carries an opaque token tree — so any query over
call_expression is blind to calls written inside one. 42,511 of the corpus's
invocations are assert*!, meaning the call a #[test] actually exercises was
precisely the one never recorded. Recovered by a token scan over the three call
shapes Rust can spell, labelled Meta["via_macro"] so consumers wanting only
grammar-certain edges can filter. Pattern and cfg-predicate macros are excluded,
so matches!(x, Some(_)) does not report a call to Some.

Correctness fixes, each reproduced with a probe before being fixed:

  • Two same-named types in one file collapsed into one node and the loser's
    fields were re-parented onto the survivor
    , which then reported members it
    does not have. Now routed through the existing disambiguateID helper, with
    scope_mod distinguishing them.
  • Associated consts were emitted at file scope, so impl A { const LIMIT } and
    impl B { const LIMIT } fought over one id and the loser vanished entirely.
  • Type canonicalisation emitted targets that name no possible node —
    unresolved::"u8, u16" from Result<(u8, u16), E>, unresolved::"[Item]"
    from &[Item], unresolved::"Error + Send + 'static" from a boxed trait
    object. A tuple now expands to one edge per slot.
  • impl Trait for &'a Buf minted the method id <file>::&'a Buf.show with a
    member_of edge to a node that never exists, orphaning the method.
  • rustCollectAttributes stopped at the first non-attribute sibling, so a
    comment between an attribute stack and its item discarded every attribute
    above it.
  • Result error extraction split on every comma and anchored on the last ::
    in the string, so Result<(u8, u16), E> emitted a throws edge to the literal
    "u16)" and std::result::Result<T, io::Error> emitted none at all.
  • Trait members were reported private even when their trait was pub,
    understating the public API surface.
  • signature was the hardcoded placeholder fn name(...) on every Rust
    function; async/unsafe/const/extern qualifiers were never recorded.

Relationships. Neither way a Rust type acquires a trait produced an
implements edge: impl Display for Buf emitted only a method-level
EdgeOverrides (so a marker impl left no trace at all) and #[derive(...)]
emitted only an annotation — despite derive being how most Rust types get
Debug, Clone and Serialize. Attributes on impl blocks were dropped
wholesale, which is why the file-level wasm_bindgen sentinel missed the form
the wasm-bindgen guide recommends for methods.

Entry points. entrypoints.Detect had no rust case, so nothing in a Rust
repo was ever a live root and the dead-code analyzer reported every binary
crate's entire call tree as dead. Now stamps fn main in Cargo binary targets,
async-runtime mains, test/bench harnesses, and FFI exports; extern block
declarations are stamped rust:ffi-import because their body lives in another
language, so a missing definition is not a missing implementation.

Result

Re-measured over the same 4,000 files:

before after
call edges 238,457 381,608 (+60%)
total nodes 221,676 370,822 (+67%)
implements edges 0 35,604
typed_as edges 30,544 103,130
returns edges 21,325 51,819
annotated edges 54,545 99,778

go build ./... and go test -race ./... pass. 663 lines of new tests, each
asserting the specific defect it covers.

Deliberately not in this change

The audit also surfaced problems outside the parsing layer, listed here so they
are not lost. The resolver ones matter most because they produce confidently
wrong results rather than missing ones:

  • use super::X from a non-mod.rs module file resolves one directory too high
    and binds to the wrong file at full ast_resolved confidence.
  • Module-qualified path calls (alpha::helper()) bind to an unrelated module's
    same-named function.
  • #[path = "..."] is ignored; glob re-exports (pub use inner::*) are dropped;
    group imports (use crate::{a, b}) never resolve.
  • Cargo workspace siblings are misclassified as external dependencies, and
    [workspace.dependencies] is never parsed.
  • Axum/actix attribute routes bind to the function preceding the attribute.
  • No tokio channel-ops, sqlx SQL extraction, or ORM lane for diesel/sea-orm.

I also verified and rejected one reported issue: a field's doc comment bleeding
onto the next undocumented field does not reproduce.

zzet added 8 commits July 28, 2026 01:54
Measured over 4,000 files of real crate source, the Rust extractor was
blind to whole categories of declaration: 10,094 type aliases, 4,206
modules, 1,148 macro_rules! definitions, 416 extern crate declarations
and 38,973 extern blocks produced zero graph nodes between them.

Rust also does not parse macro arguments as expressions — every
macro_invocation carries an opaque token_tree — so 68,551 invocations
concealed 65,525 call sites against the 238,457 the extractor found
everywhere else. Test suites suffered worst: 42,511 of those
invocations are assert*!, so the call a #[test] actually exercises was
precisely the one never recorded. A trailing ::<T> likewise wraps a
callee in a generic_function node that none of the three call patterns
matched, hiding another 5,301 sites such as xs.collect::<Vec<_>>().

Adds query patterns and emitters for mod (KindPackage, following the
C++/C# namespace precedent — KindModule is reserved for package-manager
dependencies), type aliases, unions and their fields, macro_rules!,
extern-block functions with their ABI, extern crate, and associated
types. Recovers macro-body call sites by walking the token stream for
the three shapes Rust can spell, labelling each with via_macro so a
consumer wanting only grammar-certain edges can filter; pattern and
cfg-predicate macros are excluded so matches!(x, Some(_)) does not
report a call to Some.

Two same-named types in one file previously collapsed into one node and
the loser's fields were re-parented onto the survivor, which then
reported members it does not have. Types, enums, traits and unions now
route through disambiguateID like methods already did, and members bind
to the id actually minted for their container rather than reconstructing
it by name. Items carry scope_mod so the two remain distinguishable.

Also stamps real signatures in place of the "fn name(...)" placeholder
every function carried, records async/unsafe/const/extern qualifiers,
gives free functions the return_type meta methods already had, stops
rustCollectAttributes from discarding an attribute stack when a comment
sits between it and its item, and fixes Result error extraction, which
split on every comma (making Result<(u8, u16), E> emit a throws edge to
the literal target "u16)") and anchored on the last "::" in the string
(so std::result::Result<T, io::Error> emitted no edge at all).
Every const_item and static_item was emitted at file scope as a bare
KindVariable with empty Meta. Two consequences:

An associated const belongs to its impl, so `impl A { const LIMIT }` and
`impl B { const LIMIT }` both minted `<file>::LIMIT`. The dedup guard
dropped the second outright — no node, no edge — and the survivor
reported no owner. They now scope to the container that declares them,
carry a receiver, and link to it with EdgeMemberOf.

A const is a constant, not a variable: it now lands as KindConstant, so
the dead-code pass's IncludeConstants switch and the constant-value
sidecar apply to Rust the way they already do elsewhere. `static` stays
KindVariable and gains a `mutable` marker for `static mut`, the one form
that is genuinely mutable shared state.

Both kinds now record visibility, declared type, doc comment, enclosing
module and their attributes, which is what makes clap and serde
metadata on a const reachable at all.
`struct Meters(f64)` and `Msg::Failed(Error)` produced no member node at
all, so a newtype's wrapped type was invisible: find_usages on Error
could not see that Msg::Failed carries one, and the whole newtype idiom
contributed nothing to the type graph.

Positional fields are now emitted under the name Rust uses to address
them (self.0), carrying field_type and a typed_as edge to the wrapped
type. A payload nests under its variant rather than the enum, so two
payload-carrying variants cannot collide on ".0".

Struct-shaped variants (`Rich { code: u16 }`) were dropped too — the
named-field pass only ever accepted a struct_item parent. It now
resolves struct, union and enum-variant owners alike.
Type canonicalisation stripped a tuple's parentheses and left the comma
behind, and had no case at all for slices, arrays, raw pointers, or a
trait-object bound list. The result was edge targets that name no
possible node — unresolved::"u8, u16" from Result<(u8, u16), E>,
unresolved::"[Item]" from &[Item], unresolved::"Error + Send + 'static"
from Box<dyn Error + Send>. They can never resolve, and they crowd out
real misses in every unresolved-reference report.

Slices, arrays, raw pointers and bound lists now unwrap to the type they
carry, and anything still holding punctuation after canonicalisation is
rejected rather than emitted. A tuple names one type per slot, so
parameters, returns and let-bindings expand into one edge per element —
`-> (Item, Cfg)` now returns both at their own positions instead of
neither.

The same non-nominal spellings broke impl receivers: `impl Trait for
&'a Buf` minted the method id `<file>::&'a Buf.show` and pointed its
member_of at a node that never exists, so the method was orphaned from
the type it belongs to. Reference, pointer, slice and array impl targets
now reduce to the named type. The generic spelling is deliberately left
alone — `Replacer<M>.replace_all` is the established id shape and
rebindRustLocalGenericMemberOwners already repairs its member_of.
Neither of the two ways a Rust type implements a trait produced an
implements edge. `impl Display for Buf` emitted only a method-level
EdgeOverrides, so a marker impl carrying no methods — `impl Send for
Handle {}` — left no trace whatsoever, and #[derive(Debug, Clone)]
emitted an annotation and nothing else. Since derive is how most Rust
types acquire Debug, Clone and Serialize, find_implementations and
get_class_hierarchy were blind to the bulk of the trait hierarchy.

Both now emit EdgeImplements from the type to the trait, tagged with
whether it came from an impl block or a derive, and a path-qualified
trait (std::fmt::Display, serde::Serialize) reduces to its base name the
way the rest of the extractor already does.

Attributes written on an impl block were dropped wholesale. That is how
#[async_trait], #[pymethods] and #[wasm_bindgen] impl blocks went
unrecorded — including the file-level wasm_bindgen interop sentinel,
which only ever saw function-level attributes and so missed the form the
wasm-bindgen guide actually recommends for methods.
entrypoints.Detect had no rust case, so nothing in a Rust repo was ever
stamped as a live root. fn main, every #[test], and every #[no_mangle]
export have no in-crate caller by construction, so a call-graph walk
found them unreachable and the dead-code analyzer reported a binary
crate's entire call tree as dead.

Stamps the symbols a runtime other than application code calls: fn main
in a Cargo binary target (src/main.rs, src/bin/*.rs, examples/, benches/,
tests/ — a fn main in a library module stays an ordinary function),
async-runtime mains behind #[tokio::main] / #[actix_web::main] /
#[async_std::main] / #[rocket::main], test and bench harness attributes,
and FFI exports reached from another language through #[no_mangle],
#[wasm_bindgen], pyo3 and napi.

Declarations inside an extern block are stamped rust:ffi-import for the
opposite reason: their body lives in C, so a missing definition is not a
missing implementation and must not be reported as one.
A trait member carries no visibility modifier of its own; it is exactly
as reachable as the trait that declares it. Reading the absent modifier
as "private" meant every method of every `pub trait` was reported
private, which understates the public API surface and invites the
dead-code pass to over-reach on symbols that are part of a crate's
contract. Trait members now inherit their trait's visibility, and a
method in `impl Trait for Type` — callable by anyone who can name the
trait — is public, while an inherent impl keeps ordinary per-method
visibility.

Trait declarations also still carried the "fn name(...)" placeholder
signature after the free-function and impl-method paths had stopped
using it, and gained none of the async/unsafe/const qualifiers. Both now
go through the same builder as every other function.
@zzet
zzet merged commit 903194e into main Jul 28, 2026
10 checks passed
@zzet
zzet deleted the feat/rust-parser-coverage branch July 28, 2026 08:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant