enhancement(compiler): cache external query paths

[gwenaskell]

Summary

VRL currently clones the lookup path every time it reads a field such as .foo.bar. Those allocations add up in programs that read several fields per event.

This PR builds the full path once during compilation and reuses it for event and metadata reads. Each query owns a single path. exists and del also borrow that stored path through Query::external_path.

Rust API change and Vector follow-up

Query::external_path now returns Option<&OwnedTargetPath>. Callers that need an owned path can use .cloned().

Vector's set_semantic_meaning needs this update when bumping VRL:

-        let Some(path) = query.external_path() else {
+        let Some(path) = query.external_path().cloned() else {

That clone occurs once during compilation, when Vector stores the path in MeaningList.

Performance

Recorded AST measurements on Apple M4 Max, Rust 1.96.1, with the system allocator: 100 samples, 3 s warmup, 5 s measurement. Compilation and fresh target setup are excluded; runtime reset, execution and result/target drops are included.

Field reads and remap, compared with upstream f363989fbebdbf63f2bcd2905d6daee0ccfa6d00:

Program	Before mean	After mean	Change
.a	92.0 ns	64.4 ns	-30.0%
.a.b	142.6 ns	98.6 ns	-30.9%
.a.b.c	206.6 ns	135.9 ns	-34.2%
%a.b.c	200.6 ns	142.8 ns	-28.8%
. (root control)	270.9 ns	275.1 ns	No significant change (p = 0.07)
Eight-field remap	1.640 µs	1.227 µs	-25.2%

Borrowed accessor, compared with the initial PR commit f57fcc003:

Program	Cloning accessor mean	Borrowed accessor mean	Change
exists(.a.b.c)	216.3 ns	132.6 ns	-38.7%
del(.a.b.c)	244.0 ns	171.7 ns	-29.7%

These recorded results used the original standalone runner with stdlib-base. On the current revision, the Cargo benchmark target is stdlib and requires the default and test features. external_field_reads is the Criterion filter for cases registered from benches/runtime/mod.rs:

cargo bench --bench stdlib --features default,test -- external_field_reads --warm-up-time 3 --measurement-time 5 --sample-size 100

For a new comparison, use the same benches/runtime/mod.rs module and runtime::external_field_reads registration in benches/stdlib.rs on both revisions. Add --save-baseline before to the command on the baseline revision, then --baseline before on the changed revision. Results cover VRL execution, not a full Vector pipeline.

How did you test this PR?

• make fmt, make fmt-check, and make check-clippy (workspace, all targets/features).
• cargo test --lib --no-default-features --features stdlib-base,enable_system_functions compiler::expression (92 passed).
• cargo run --package vrl-tests --bin vrl-tests -- --pattern expressions/query (15 passed).
• cargo run --package vrl-tests --bin vrl-tests -- --pattern metadata (6 passed).
• cargo run --package vrl-tests --bin vrl-tests -- --pattern exists (3 passed).
• cargo run --package vrl-tests --bin vrl-tests -- --pattern functions/del (9 passed).
• cargo test --lib --no-default-features --features stdlib-base,enable_system_functions stdlib::del (1 passed).
• cargo run -p release -- check-changelog.
• Before/after Criterion comparisons above.
• cargo bench --bench stdlib --features default,test -- external_field_reads --test (8 cases passed after moving them into the shared runner).

One new VRL fixture checks event and metadata collection copy independence after mutations, plus root, indexed and missing reads.

Does this PR include user-facing changes?

• Yes. Includes performance and Rust API migration changelog fragments.
• No. A maintainer will apply the "no-changelog" label to this PR.

Checklist

• Read CONTRIBUTING.md.
• If this PR introduces changes to LICENSE-3rdparty.csv, run dd-rust-license-tool write and commit the changes. No dependencies changed.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-13T13:53:14.921098Z	6cddab8	New commits

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[datadoghq-integration]

More details

The cached paths preserve event and metadata prefixes, cloned read values, and existing query APIs. The completed static review identified no reportable regression.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit f57fcc0 · @DataDog review to ask questions}

[bruceg]

LGTM but I have a question about a clone that remains

[github-actions]

⚠️ Vector compatibility advisory

This check tests whether Vector can update to the VRL version in this PR without additional changes. Dependency changes can make that update significantly harder. If the required Vector changes are reasonable, you can ignore this advisory. Review the workflow run for details.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e9284f2fb7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".