Fix sanitized LLM codec annotations#248
Conversation
Signed-off-by: Minh Vu <vuhoangminh97@gmail.com>
github-actions
Bot
added
size:M
WalkthroughRequest and response codec annotations are refactored to derive from sanitized guardrailed payloads. End-event response decoding becomes configurable for error fatality and cost attachment. Request codec is propagated through both execution paths, and streaming response decoding is moved to occur post-sanitization. ChangesCodec Annotation from Sanitized Payloads
🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
⚔️ Resolve merge conflicts
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
crates/core/src/api/llm.rs (1)
1-3:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftRun the required Rust/core validation matrix before merge.
The listed checks in this PR are missing required core-change validation coverage (
just test-rust,cargo fmt --all,cargo clippy --workspace --all-targets -- -D warnings, plus thecrates/corefull language matrix guidance). Please add these results to the PR.
As per coding guidelines,**/*.rsrequires runningjust test-rust,cargo fmt --all, andcargo clippy --workspace --all-targets -- -D warnings, andcrates/{core,adaptive}/**/*requires the full language matrix for core changes.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/core/src/api/llm.rs` around lines 1 - 3, This PR is missing required Rust/core validation results for core changes: run "just test-rust", then "cargo fmt --all", and "cargo clippy --workspace --all-targets -- -D warnings" in the workspace, and also execute the full language matrix required for core changes (the core crate validation matrix); capture and attach the test, format, and clippy outputs (logs or CI artifacts) to the PR so reviewers can verify the checks passed.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@crates/core/src/api/llm.rs`:
- Around line 1-3: This PR is missing required Rust/core validation results for
core changes: run "just test-rust", then "cargo fmt --all", and "cargo clippy
--workspace --all-targets -- -D warnings" in the workspace, and also execute the
full language matrix required for core changes (the core crate validation
matrix); capture and attach the test, format, and clippy outputs (logs or CI
artifacts) to the PR so reviewers can verify the checks passed.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 0dfffbe3-d0d7-4432-859e-94e4f0e7f3d5
📒 Files selected for processing (3)
crates/core/src/api/llm.rscrates/core/src/stream.rscrates/core/tests/integration/pipeline_tests.rs
📜 Review details
🧰 Additional context used
📓 Path-based instructions (16)
**/*.rs
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Use
snake_casenaming convention for Rust identifiers (e.g.,nemo_relay_tool_call)
**/*.rs: Any Rust change must runjust test-rust
Any Rust change must runcargo fmt --all
Any Rust change must runcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allfor all FFI work since it is Rust work
Runjust test-rustto validate FFI changes
Runcargo clippy --workspace --all-targets -- -D warningsto enforce strict linting on FFI workWhen Rust files changed as part of Go work, also run
cargo fmt --all,just test-rust, andcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Runcargo fmt --allwhen Rust files are changed as part of Node work
Runcargo clippy --workspace --all-targets -- -D warningswhen Rust files are changed as part of Node work
Runjust test-rustwhen Rust files are changed as part of Node work
**/*.rs: Runcargo fmt --allto format all Rust code
Runcargo clippy --workspace --all-targets -- -D warningsto enforce all clippy lints as errors
**/*.rs: Runcargo fmt --allwhen Rust files changed as part of WebAssembly work
Runcargo clippy --workspace --all-targets -- -D warningswhen Rust files changed as part of WebAssembly work
**/*.rs: If any Rust code changed, always runjust test-rust
If any Rust code changed, also runcargo fmt --all
If any Rust code changed, also runcargo clippy --workspace --all-targets -- -D warnings
Run Rust formatting withcargo fmt --all
Run Rust linting withcargo clippy --workspace --all-targets -- -D warnings
**/*.rs: Usecargo fmtfor Rust code formatting
Runcargo clippy -- -D warningsto lint Rust code and treat all warnings as errors
Use Rust snake_case naming convention for Rust identifiers
Include SPDX license header in all Rust source files using double-slash comment syntax
Validate Rust code withuv run pre-commit run --all-filesto enforce cargo fmt formatting check, cargo clippy lints, and cargo deny aud...
Files:
crates/core/tests/integration/pipeline_tests.rscrates/core/src/api/llm.rscrates/core/src/stream.rs
{crates/adaptive/**/*.rs,**/*test*.{rs,py,go,ts,js},**/*adaptive*test*.{rs,py,go,ts,js},docs/plugins/adaptive/**}
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Maintain documented and tested validation and report behavior for adaptive surfaces
Files:
crates/core/tests/integration/pipeline_tests.rs
**/{Cargo.toml,**/*.rs}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Maintain consistency between Rust package names in
Cargo.tomland their actual usage across the codebase
Files:
crates/core/tests/integration/pipeline_tests.rscrates/core/src/api/llm.rscrates/core/src/stream.rs
**/*.{h,hpp,c,cpp,rs}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Ensure FFI header and library naming follows consistent conventions across platform-specific builds
Files:
crates/core/tests/integration/pipeline_tests.rscrates/core/src/api/llm.rscrates/core/src/stream.rs
{crates/core,crates/adaptive}/**/*
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changes to
crates/coreorcrates/adaptivemust run the full language matrix
Files:
crates/core/tests/integration/pipeline_tests.rscrates/core/src/api/llm.rscrates/core/src/stream.rs
**/*.{rs,toml}
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update Rust crate names and module prefixes during coordinated rename operations
Files:
crates/core/tests/integration/pipeline_tests.rscrates/core/src/api/llm.rscrates/core/src/stream.rs
crates/core/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/test-go-binding/SKILL.md)
If the change touched
crates/coreor shared runtime semantics, also usevalidate-changefor broader validation
crates/core/**/*.rs: UseJson = serde_json::Valuein Rust-facing runtime APIs where the existing code expects JSON payloads.
UseResult<T>withFlowErrorin core runtime paths. Keep errors explicit and binding-appropriate at the wrapper layer.
Files:
crates/core/tests/integration/pipeline_tests.rscrates/core/src/api/llm.rscrates/core/src/stream.rs
crates/{core,adaptive}/**
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If
crates/coreorcrates/adaptivechanged, run the full matrix across Rust, Python, Go, Node.js, and WebAssembly
Files:
crates/core/tests/integration/pipeline_tests.rscrates/core/src/api/llm.rscrates/core/src/stream.rs
**/*.{rs,py,js,ts,tsx,jsx,go,sh,toml,yaml,yml,md}
📄 CodeRabbit inference engine (AGENTS.md)
Keep SPDX headers on source, docs, scripts, and configuration files. The project is Apache-2.0.
Files:
crates/core/tests/integration/pipeline_tests.rscrates/core/src/api/llm.rscrates/core/src/stream.rs
**/*.{rs,py,go,js,ts,tsx}
📄 CodeRabbit inference engine (AGENTS.md)
Follow binding naming conventions: Rust and Python use
snake_case, C FFI exports prefixednemo_relay_, Go usesPascalCasefor public APIs, Node.js usescamelCase.
Files:
crates/core/tests/integration/pipeline_tests.rscrates/core/src/api/llm.rscrates/core/src/stream.rs
crates/**/*.rs
📄 CodeRabbit inference engine (AGENTS.md)
crates/**/*.rs: Keep async behavior on the existing tokio-based model. Bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
UseJson = serde_json::Valuein Rust-facing runtime APIs for JSON payload handling.
Files:
crates/core/tests/integration/pipeline_tests.rscrates/core/src/api/llm.rscrates/core/src/stream.rs
crates/{core,adaptive}/**/*.rs
⚙️ CodeRabbit configuration file
crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.
Files:
crates/core/tests/integration/pipeline_tests.rscrates/core/src/api/llm.rscrates/core/src/stream.rs
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}
⚙️ CodeRabbit configuration file
{crates/**/tests/**,python/tests/**,go/nemo_relay/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.
Files:
crates/core/tests/integration/pipeline_tests.rs
**
⚙️ CodeRabbit configuration file
**:AGENTS.md
This file provides guidance to agents, including Claude Code and OpenAI Codex, when working in this repository.
Project Overview
NeMo Relay is a multi-language agent runtime framework for execution scopes, lifecycle events, middleware, plugins, and observability around tool and LLM calls. The core runtime is Rust. Primary supported bindings are Rust, Python, and Node.js. Go, WebAssembly, and the raw C FFI are experimental and source-first.
The shared runtime model is:
- Scope stacks decide where work belongs and which scope-local behavior is visible.
- Middleware registries decide what guardrails and intercepts run around managed calls.
- Plugins install reusable runtime behavior from configuration.
- Events record runtime behavior in ATOF form.
- Subscribers and exporters consume events in-process or export them to ATIF, OpenTelemetry, OpenInference, or other backends.
Repository Structure
The repository layout separates the Rust runtime, language bindings, documentation,
integration patches, and agent-facing skills.crates/ core/ # Rust core runtime crate, published as nemo-relay adaptive/ # Adaptive runtime primitives and plugin components python/ # PyO3 native extension for the Python package ffi/ # Raw C ABI layer used by downstream bindings such as Go node/ # NAPI Node.js binding and JavaScript/TypeScript entry points wasm/ # wasm-bindgen WebAssembly binding and JS wrappers python/ nemo_relay/ # Python wrapper package: scopes, tools, LLM, middleware, typed helpers, plugins, adaptive helpers tests/ # Python tests go/ nemo_relay/ # Experimental Go CGo binding and tests fern/ # Fern documentation site scripts/ # Stable wrappers and helper scripts; build/test/docs entry points live in justfile third_party/ # P...
Files:
crates/core/tests/integration/pipeline_tests.rscrates/core/src/api/llm.rscrates/core/src/stream.rs
crates/core/src/api/**/*.rs
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Implement behavior first in Rust core API modules:
crates/core/src/api/and related core modules such ascrates/core/src/api/runtime/,crates/core/src/codec/, orcrates/core/src/json.rs
Files:
crates/core/src/api/llm.rs
crates/core/src/api/{tool,llm}.rs
📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)
Wire the new middleware chain into the execute path in
crates/core/src/api/tool.rsorcrates/core/src/api/llm.rsat the appropriate pipeline stage
Files:
crates/core/src/api/llm.rs
🔇 Additional comments (3)
crates/core/src/api/llm.rs (1)
275-311: LGTM!Also applies to: 358-476, 593-612, 628-640, 747-766
crates/core/src/stream.rs (1)
159-170: LGTM!crates/core/tests/integration/pipeline_tests.rs (1)
22-26: LGTM!Also applies to: 33-35, 233-275, 1052-1110, 1252-1311, 1376-1438
|
@fallintoplace how does this differ from #241 |
2 participants
Summary
Ensure LLM start/end annotated codec fields are decoded from the sanitized event payloads when sanitize guardrails rewrite request or response content. This keeps structured annotations from leaking raw content that the emitted event output has already redacted.
Tests
cargo test -p nemo-relay --test pipeline_integration sanitizedcargo test -p nemo-relay --test pipeline_integrationcargo fmt --all --checkSummary by CodeRabbit
Refactor
Tests