feat: isolate and attest OpenShell jobs#298
Conversation
Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughOpenShell deep-research sandboxing now uses per-job policy-bound sandboxes with attestation and allowlist networking. Cleanup now flows through ChangesPer-job OpenShell sandbox and finalize lifecycle
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
204-216: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
_cleanup_failedis sticky across mid-life retries, causing false-negative terminal cleanup reports.
_safe_close(and its cousin inopenshell.py's_exit_context) setsself._cleanup_failed = Trueon any close exception, but_safe_closeis also invoked from_reset_session()when tearing down a stale session before recreating it on a recoverable error — a mid-job event unrelated to terminal cleanup. Since_cleanup_failedis never reset, a transient failure during that retry path permanently poisonscleanup_succeeded, sofinalize()will report "failed" for a job whose actual terminalclose()succeeded without error. This undermines the truthful-cleanup-reporting goal called out in the PR description.Consider tracking retry-teardown failures separately from terminal-cleanup failures, or resetting
_cleanup_failedat the start of_reset_session()since a fresh session is about to replace the stale one.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/sandbox/base.py` around lines 204 - 216, The cleanup failure flag in `_safe_close` is being left sticky across session retries, so a stale-session close error can incorrectly make `cleanup_succeeded` fail later. Update `BaseSandbox._safe_close` and the retry path in `_reset_session()` to distinguish mid-life teardown from final terminal cleanup, or reset `_cleanup_failed` when starting a fresh session replacement. Keep the terminal reporting used by `cleanup_succeeded`/`finalize()` tied only to the actual final close, and apply the same handling to the corresponding `_exit_context` logic in `openshell.py` if it shares the same flag.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@frontends/aiq_api/src/aiq_api/jobs/runner.py`:
- Around line 667-673: In runner.py, the sandbox cleanup path around finalize()
only logs when an exception is raised, so a falsy cleanup result is missed.
Update the finalize handling in the job runner to inspect the return value from
sandbox_runtime.finalize(...) and emit the same warning path when it returns
False or another falsy cleanup_succeeded value, while still keeping exception
handling non-fatal. Use the existing finalize, job_id, and logger.warning call
site to keep the behavior aligned with the current cleanup flow.
In `@src/aiq_agent/agents/deep_researcher/deepagents_runtime.py`:
- Around line 278-298: The finalize() idempotency guard in DeepResearcherRuntime
only protects the _finalized flag, so concurrent calls can skip the actual
cleanup and return a stale success from cleanup_succeeded. Update finalize() to
keep the same lock across the full cleanup path, including terminate()/close(),
the cleanup result read, and the cleanup event emission, so only one caller
performs cleanup and others wait for the real outcome. Use the existing
finalize(), _finalize_lock, _finalized, _emit_cleanup, terminate(), and close()
symbols to make the change.
In `@src/aiq_agent/agents/deep_researcher/register.py`:
- Around line 298-306: Add a unit test in test_agent.py that covers _run
cancellation behavior in DeepResearcher register.py: simulate
asyncio.CancelledError during _run, assert the exception is re-raised, and
verify active_agent.finalize is called with interrupted=True only when
owns_active_agent is true. Use the _run coroutine and the owns_active_agent /
active_agent.finalize path to confirm no finalization happens for non-owned
agents and that the finally block triggers the correct interrupted flag for
owned agents.
In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py`:
- Around line 368-423: `_os_context` is being accessed concurrently between
`_create_session()` and out-of-band cleanup paths like `close()`, `terminate()`,
and `_terminate_session()`, which can race during cancellation. Protect all
reads/writes/clears of `self._os_context` with the existing `self._state_lock`,
matching how `self._session` is guarded in the base class, and ensure
`_exit_context()` and the `_create_session()` setup/teardown path in
`OpenShellSandboxProvider` use the same lock consistently.
- Around line 406-424: The attestation success event is emitted too early in
OpenShellSandbox creation, before session construction is guaranteed to succeed.
In openshell.py, adjust the _create_session flow so _attest(self, os_sandbox)
and its “succeeded” emission happen only after OpenShellSandbox(...) is
successfully constructed, or add a failure compensation path if adapter
construction throws. Keep the logging and cleanup around os_sandbox.__enter__
and _exit_context() unchanged, but ensure the attestation state reflects the
final outcome of _create_session rather than the pre-construction step.
In `@tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py`:
- Around line 151-441: Add a regression test around OpenShellSandboxProvider
that exercises _reset_session() after a stale-session close failure, then
performs a later successful terminal close and verifies cleanup_succeeded
remains True. Use the existing _reset_session and close behavior in the
provider/test setup to force the first cleanup failure, then assert the second
close does not stay poisoned by _cleanup_failed.
---
Outside diff comments:
In `@src/aiq_agent/agents/deep_researcher/sandbox/base.py`:
- Around line 204-216: The cleanup failure flag in `_safe_close` is being left
sticky across session retries, so a stale-session close error can incorrectly
make `cleanup_succeeded` fail later. Update `BaseSandbox._safe_close` and the
retry path in `_reset_session()` to distinguish mid-life teardown from final
terminal cleanup, or reset `_cleanup_failed` when starting a fresh session
replacement. Keep the terminal reporting used by
`cleanup_succeeded`/`finalize()` tied only to the actual final close, and apply
the same handling to the corresponding `_exit_context` logic in `openshell.py`
if it shares the same flag.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 63446849-608f-4056-8b4b-58895c90df18
📒 Files selected for processing (16)
configs/config_openshell.ymlconfigs/openshell/aiq-research-policy.yamldocs/source/architecture/agents/sandbox.mdfrontends/aiq_api/src/aiq_api/jobs/runner.pyscripts/README.mdscripts/setup_openshell.shsrc/aiq_agent/agents/deep_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/sandbox/README.mdsrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/config.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/jobs/test_runner.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (10)
**
⚙️ CodeRabbit configuration file
**:AI-Q Agent Guidance
Repository-global instructions for coding agents and for humans reviewing
agent-authored changes. These rules apply to every task in this repository.
Task-specific runbooks live in.agents/skills/— load the
relevant skill before starting a workflow it covers.Project overview
AI-Q is an NVIDIA AI Blueprint: an enterprise research agent built on the
NeMo Agent Toolkit (NAT). The deployed product is a research blueprint, not
a general skill runtime. New retrieval sources and tools are NAT functions;
agent behavior is driven by workflow YAML, Jinja2 prompts, and a data-source
registry — not by hard-coded logic.Primary boundaries:
- Backend Python package:
src/aiq_agent/.- Data-source and tool packages:
sources/(each is its own package).- Frontends and tooling:
frontends/(web UI infrontends/ui/, eval harnesses
infrontends/benchmarks/).- Configs, deployment, docs:
configs/,deploy/,docs/.Stay inside this repository. If your workspace also contains adjacent repos
(for example a sibling NeMo-Relay checkout), do not edit them as part of an AI-Q
change. Treatsources/*as independent packages: prefer the smallest change
scoped to the package you are touching.Repository structure
Path Purpose src/aiq_agent/Backend agent, FastAPI extensions, auth, observability, knowledge sources/Data-source / tool packages (e.g. tavily_web_search,google_scholar_paper_search)configs/Workflow YAML configs (e.g. config_cli_default.yml)frontends/ui/Next.js / React / TypeScript / Tailwind / KUI web UI frontends/benchmarks/Eval harnesses: freshqa,deepsearch_qa,deepresearch_benchdeploy/Docker Compose and Helm/Kubernetes assets; deploy/.envfor secretsdocs/source/...
Files:
scripts/README.mdtests/aiq_agent/jobs/test_runner.pysrc/aiq_agent/agents/deep_researcher/agent.pyconfigs/config_openshell.ymlfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/register.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pyconfigs/openshell/aiq-research-policy.yamldocs/source/architecture/agents/sandbox.mdsrc/aiq_agent/agents/deep_researcher/sandbox/config.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pyscripts/setup_openshell.shsrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/README.md
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run ruff check and ruff format validation for Python code changes
**/*.py: Python code must be linted and formatted with Ruff using line length 120, target Python 3.11, rule sets E,F,W,I,PL,UP, and isort force-single-line configuration
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr instead, resolving API keys at runtime
Never print or log secret values, including in tool output or error messages
Missing-secret paths must degrade gracefully (stub/skip), not crash or leak
Do not hand-reformat unrelated code when making changes; match the existing import and formatting style
Files:
tests/aiq_agent/jobs/test_runner.pysrc/aiq_agent/agents/deep_researcher/agent.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/register.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/config.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
**/*test*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run pytest for all behavior changes in Python code
Files:
tests/aiq_agent/jobs/test_runner.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py
src/aiq_agent/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/aiq_agent/**/*.py: Respect authenticated data sources by honoring requires_auth, per-user token pass-through, and backend token validators; apply owner guardrails before loading protected report or artifact context into an agent
Do not weaken or bypass AuthMiddleware, validators, or auth gating without a prior design discussion
Files:
src/aiq_agent/agents/deep_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/sandbox/config.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/deep_researcher/agent.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/register.pysrc/aiq_agent/agents/deep_researcher/sandbox/config.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/README.md
{deploy/**,configs/**}
⚙️ CodeRabbit configuration file
{deploy/**,configs/**}: Review deployment and config changes for secret separation, safe defaults, local-vs-production behavior, Helm and
Docker portability, and documentation parity. Flag committed credentials, environment-specific NVIDIA internals in
public defaults, and changes that make examples diverge from CI-tested paths.
Files:
configs/config_openshell.ymlconfigs/openshell/aiq-research-policy.yaml
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.py
docs/source/**/*
📄 CodeRabbit inference engine (AGENTS.md)
Update the docs under docs/source/ when behavior, configuration, or workflows change
Files:
docs/source/architecture/agents/sandbox.md
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,SECURITY.md,CODE-OF-CONDUCT.md}: Review documentation for command accuracy, branch-name consistency, current CI and copy-pr-bot behavior, public
vs internal boundary clarity, stale examples, and links that no longer match the repository layout.
Files:
docs/source/architecture/agents/sandbox.md
**/*config*.py
📄 CodeRabbit inference engine (AGENTS.md)
Config schemas must inherit from FunctionBaseConfig and YAML _type names must come from the registered config class
Files:
src/aiq_agent/agents/deep_researcher/sandbox/config.py
🔇 Additional comments (20)
configs/config_openshell.yml (1)
1-2: LGTM!Also applies to: 142-165
configs/openshell/aiq-research-policy.yaml (1)
24-29: LGTM!scripts/setup_openshell.sh (2)
17-17: LGTM!Also applies to: 51-51, 61-61, 80-80, 93-95, 108-108, 1010-1010, 1062-1062, 1075-1075
155-162: LGTM!Validation of
LANDLOCK_COMPATIBILITYruns inresolve_policy()beforeemit_policy_header()writes it into the now-unquoted heredoc, so only the two whitelisted literals ever reach the generated policy file — no injection risk from the quoting change.Also applies to: 878-884, 898-898, 922-925
docs/source/architecture/agents/sandbox.md (1)
9-11: LGTM!Description of per-job physical sandboxes, attestation, fail-closed network/Landlock behavior, and
sandbox.attestation/sandbox.cleanupevents is consistent with the config and setup-script changes in this cohort.Also applies to: 24-30, 40-46, 58-58
scripts/README.md (1)
54-65: LGTM!Matches the setup script's new default (
CREATE_SANDBOX=false),--landlock-compatibility/--create-shared-debug-sandboxflags, and Landlock defaults.Also applies to: 190-190
src/aiq_agent/agents/deep_researcher/sandbox/README.md (2)
40-50: LGTM!Fail-closed conditions, attestation/policy-revision requirements, debug shared-sandbox opt-in, and troubleshooting guidance are consistent with the per-job sandbox design described across the config, policy, and setup-script changes in this cohort.
Also applies to: 65-65, 160-165, 211-217, 238-254, 277-279, 287-302
122-152: 📐 Maintainability & Code QualityThe sandbox example matches the current schema.
SandboxConfigexposesnetwork: NetworkPolicywithmode/allow, andproviders.openshell.imageis a real field, so this example is consistent with the code.> Likely an incorrect or invalid review comment.src/aiq_agent/agents/deep_researcher/sandbox/config.py (1)
93-152: LGTM!src/aiq_agent/agents/deep_researcher/deepagents_runtime.py (1)
22-22: LGTM!Also applies to: 36-36, 90-127, 136-162, 191-193, 207-208, 495-504, 517-517
src/aiq_agent/agents/deep_researcher/agent.py (1)
155-158: LGTM!src/aiq_agent/agents/deep_researcher/register.py (1)
18-18: LGTM!Also applies to: 240-245, 272-272
frontends/aiq_api/src/aiq_api/jobs/runner.py (1)
649-650: LGTM!src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (3)
130-196: LGTM!
199-262: LGTM!Also applies to: 291-292, 359-359
231-243: 🎯 Functional Correctness
openis an explicit unrestricted mode, so_validate_policy_networkonly needs to enforceblockedandallowlist.> Likely an incorrect or invalid review comment.src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
102-103: LGTM!Also applies to: 145-157
tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py (1)
16-40: LGTM!Also applies to: 68-148, 564-578
tests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py (1)
35-35: LGTM!Also applies to: 306-377
tests/aiq_agent/jobs/test_runner.py (1)
1911-1920: LGTM!
|
/ok to test b6b058e |
|
Final CodeRabbit remediation is in |
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
390-390: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winDo not delete attached debug sandboxes by default.
This shared-name branch attaches to an existing sandbox, but
oscfg.delete_on_exitnow defaults toTrue, so normal cleanup can delete a sandbox the job did not create. Keep deletion disabled for shared/debug attachment unless there is a separate explicit opt-in.Suggested fix
- sandbox_kwargs.update(sandbox=shared_name, delete_on_exit=oscfg.delete_on_exit) + sandbox_kwargs.update(sandbox=shared_name, delete_on_exit=False)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py` at line 390, The shared-name attachment path in openshell.py is passing through delete_on_exit from oscfg, which can accidentally delete an existing debug sandbox that this job did not create. Update the sandbox_kwargs.update(...) call in the shared-name branch to disable deletion by default for attached sandboxes, and only allow cleanup when there is an explicit opt-in separate from the shared/debug attach flow.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py`:
- Line 390: The shared-name attachment path in openshell.py is passing through
delete_on_exit from oscfg, which can accidentally delete an existing debug
sandbox that this job did not create. Update the sandbox_kwargs.update(...) call
in the shared-name branch to disable deletion by default for attached sandboxes,
and only allow cleanup when there is an explicit opt-in separate from the
shared/debug attach flow.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 2ad4ae39-8901-4264-be74-600780d237d0
📒 Files selected for processing (9)
frontends/aiq_api/src/aiq_api/jobs/runner.pyscripts/smoke_openshell_isolation.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/jobs/test_runner.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (6)
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run ruff check and ruff format validation for Python code changes
**/*.py: Python code must be linted and formatted with Ruff using line length 120, target Python 3.11, rule sets E,F,W,I,PL,UP, and isort force-single-line configuration
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr instead, resolving API keys at runtime
Never print or log secret values, including in tool output or error messages
Missing-secret paths must degrade gracefully (stub/skip), not crash or leak
Do not hand-reformat unrelated code when making changes; match the existing import and formatting style
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pyscripts/smoke_openshell_isolation.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py
**
⚙️ CodeRabbit configuration file
**:AI-Q Agent Guidance
Repository-global instructions for coding agents and for humans reviewing
agent-authored changes. These rules apply to every task in this repository.
Task-specific runbooks live in.agents/skills/— load the
relevant skill before starting a workflow it covers.Project overview
AI-Q is an NVIDIA AI Blueprint: an enterprise research agent built on the
NeMo Agent Toolkit (NAT). The deployed product is a research blueprint, not
a general skill runtime. New retrieval sources and tools are NAT functions;
agent behavior is driven by workflow YAML, Jinja2 prompts, and a data-source
registry — not by hard-coded logic.Primary boundaries:
- Backend Python package:
src/aiq_agent/.- Data-source and tool packages:
sources/(each is its own package).- Frontends and tooling:
frontends/(web UI infrontends/ui/, eval harnesses
infrontends/benchmarks/).- Configs, deployment, docs:
configs/,deploy/,docs/.Stay inside this repository. If your workspace also contains adjacent repos
(for example a sibling NeMo-Relay checkout), do not edit them as part of an AI-Q
change. Treatsources/*as independent packages: prefer the smallest change
scoped to the package you are touching.Repository structure
Path Purpose src/aiq_agent/Backend agent, FastAPI extensions, auth, observability, knowledge sources/Data-source / tool packages (e.g. tavily_web_search,google_scholar_paper_search)configs/Workflow YAML configs (e.g. config_cli_default.yml)frontends/ui/Next.js / React / TypeScript / Tailwind / KUI web UI frontends/benchmarks/Eval harnesses: freshqa,deepsearch_qa,deepresearch_benchdeploy/Docker Compose and Helm/Kubernetes assets; deploy/.envfor secretsdocs/source/...
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pyscripts/smoke_openshell_isolation.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.py
**/*test*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run pytest for all behavior changes in Python code
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py
src/aiq_agent/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/aiq_agent/**/*.py: Respect authenticated data sources by honoring requires_auth, per-user token pass-through, and backend token validators; apply owner guardrails before loading protected report or artifact context into an agent
Do not weaken or bypass AuthMiddleware, validators, or auth gating without a prior design discussion
Files:
src/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py
🔇 Additional comments (9)
frontends/aiq_api/src/aiq_api/jobs/runner.py (1)
658-683: LGTM!src/aiq_agent/agents/deep_researcher/deepagents_runtime.py (1)
150-162: LGTM!Also applies to: 194-209, 279-322
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
36-46: LGTM!Also applies to: 271-295, 316-334, 407-430, 432-501, 528-600
src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
102-103: LGTM!Also applies to: 145-157, 210-216
tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py (1)
26-27: LGTM!Also applies to: 37-37, 82-83, 123-130, 400-438, 490-497, 500-529, 532-562, 565-620, 623-674, 814-841
tests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py (1)
20-21: LGTM!Also applies to: 308-330, 332-334, 340-359, 361-380, 381-429
tests/aiq_agent/jobs/test_runner.py (1)
1911-1919: LGTM!Also applies to: 1921-1930
tests/aiq_agent/agents/deep_researcher/test_agent.py (1)
292-336: LGTM!scripts/smoke_openshell_isolation.py (1)
29-52: LGTM!Also applies to: 55-72, 75-96, 98-142, 145-154, 156-188
KyleZheng1284
force-pushed
the
codex/aiq-openshell-isolation
branch
from
e5b69ab to
b05c93b
Compare
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
frontends/aiq_api/src/aiq_api/jobs/runner.py (1)
649-650: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick winUnguarded
event_store.flush()in terminalfinallyblock can mask the job result.Every other cleanup call in this
finallypath (_teardown_sandbox) is deliberately exception-safe, per its own docstring: cleanup must never replace the job result. This newevent_store.flush()call has no try/except — if_flush()raises, the exception propagates out of thefinallyblock and overrides whatever result/exception the job actually produced. It's also invoked synchronously on the event loop rather than viaasyncio.to_threadlike the sandbox teardown right above it, so if_flush()does any blocking I/O it will stall the worker.🛠️ Proposed fix: make flush best-effort and non-blocking
- if event_store is not None and hasattr(event_store, "flush"): - event_store.flush() + if event_store is not None and hasattr(event_store, "flush"): + try: + await asyncio.to_thread(event_store.flush) + except Exception: # noqa: BLE001 - flush must never replace the job result + logger.warning("Event store flush failed for job %s", job_id, exc_info=True)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontends/aiq_api/src/aiq_api/jobs/runner.py` around lines 649 - 650, The new event_store.flush() call in the runner’s terminal finally path should be made best-effort like _teardown_sandbox so it cannot override the job outcome. Update the cleanup in runner.py around the event_store handling to wrap flush in exception-safe handling, and if the flush implementation may block, invoke it off the event loop (consistent with the sandbox teardown pattern) instead of calling it synchronously. Use the existing runner cleanup block and event_store.flush as the locating symbols.src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
403-405: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winRespect
delete_on_exitfor job-owned sandboxes.The per-job path always passes
delete_on_exit=True, so a configureddelete_on_exit=Falseis ignored. Keep the shared-attachment override atFalse, but useoscfg.delete_on_exitfor owned sandboxes.Proposed fix
sandbox_kwargs.update( spec=_build_sandbox_spec(policy=policy, image=oscfg.image, job_id=self.job_id), - delete_on_exit=True, + delete_on_exit=oscfg.delete_on_exit, )As per path instructions, “Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts, HITL behavior, and failure handling.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py` around lines 403 - 405, The sandbox setup in openshell.py is hardcoding delete_on_exit=True for the job-owned path, which overrides the configured behavior. Update the sandbox_kwargs.update call in the owned-sandbox creation flow to use oscfg.delete_on_exit, while keeping the shared-attachment override path explicitly set to False. Make sure the change is applied in the same branch that builds the sandbox spec via _build_sandbox_spec and assigns self.job_id.Source: Path instructions
src/aiq_agent/agents/deep_researcher/deepagents_runtime.py (1)
96-106: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winEnforce the shared-sandbox debug opt-in in this config model.
Line 154 treats
existing_sandbox_name/sandbox_nameas a shared attachment, but the validator never rejectsallow_shared_sandbox=False. Add an explicit check here so the NAT/runtime config cannot accept a non-isolated sandbox unless the debug opt-in is present.Proposed fix
if self.provider == "openshell": shared_name = self.existing_sandbox_name or self.sandbox_name + if shared_name and not self.allow_shared_sandbox: + raise ValueError("existing_sandbox_name/sandbox_name requires allow_shared_sandbox=true") if not shared_name and not self.attest: raise ValueError("Per-job OpenShell creation requires attest=true")As per path instructions, “Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts, HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation, async cancellation, checkpointing, or data-source selection without focused tests and docs.”
Also applies to: 149-157
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/deepagents_runtime.py` around lines 96 - 106, The shared-sandbox debug opt-in is not enforced in this config model, so a non-isolated attachment can be accepted even when allow_shared_sandbox is false. Add an explicit validation check in the same config class that defines existing_sandbox_name, sandbox_name, and allow_shared_sandbox to reject any use of the shared sandbox fields unless the debug opt-in is enabled. Make sure the validator covers both the primary and deprecated alias fields consistently.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@frontends/aiq_api/src/aiq_api/jobs/runner.py`:
- Around line 649-650: The new event_store.flush() call in the runner’s terminal
finally path should be made best-effort like _teardown_sandbox so it cannot
override the job outcome. Update the cleanup in runner.py around the event_store
handling to wrap flush in exception-safe handling, and if the flush
implementation may block, invoke it off the event loop (consistent with the
sandbox teardown pattern) instead of calling it synchronously. Use the existing
runner cleanup block and event_store.flush as the locating symbols.
In `@src/aiq_agent/agents/deep_researcher/deepagents_runtime.py`:
- Around line 96-106: The shared-sandbox debug opt-in is not enforced in this
config model, so a non-isolated attachment can be accepted even when
allow_shared_sandbox is false. Add an explicit validation check in the same
config class that defines existing_sandbox_name, sandbox_name, and
allow_shared_sandbox to reject any use of the shared sandbox fields unless the
debug opt-in is enabled. Make sure the validator covers both the primary and
deprecated alias fields consistently.
In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py`:
- Around line 403-405: The sandbox setup in openshell.py is hardcoding
delete_on_exit=True for the job-owned path, which overrides the configured
behavior. Update the sandbox_kwargs.update call in the owned-sandbox creation
flow to use oscfg.delete_on_exit, while keeping the shared-attachment override
path explicitly set to False. Make sure the change is applied in the same branch
that builds the sandbox spec via _build_sandbox_spec and assigns self.job_id.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 1a4f355e-9777-4ffb-8f9e-c7419c6c4804
📒 Files selected for processing (9)
frontends/aiq_api/src/aiq_api/jobs/runner.pyscripts/smoke_openshell_isolation.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/jobs/test_runner.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (6)
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run ruff check and ruff format validation for Python code changes
**/*.py: Python code must be linted and formatted with Ruff using line length 120, target Python 3.11, rule sets E,F,W,I,PL,UP, and isort force-single-line configuration
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr instead, resolving API keys at runtime
Never print or log secret values, including in tool output or error messages
Missing-secret paths must degrade gracefully (stub/skip), not crash or leak
Do not hand-reformat unrelated code when making changes; match the existing import and formatting style
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pyscripts/smoke_openshell_isolation.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
**/*test*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run pytest for all behavior changes in Python code
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py
**
⚙️ CodeRabbit configuration file
**:AI-Q Agent Guidance
Repository-global instructions for coding agents and for humans reviewing
agent-authored changes. These rules apply to every task in this repository.
Task-specific runbooks live in.agents/skills/— load the
relevant skill before starting a workflow it covers.Project overview
AI-Q is an NVIDIA AI Blueprint: an enterprise research agent built on the
NeMo Agent Toolkit (NAT). The deployed product is a research blueprint, not
a general skill runtime. New retrieval sources and tools are NAT functions;
agent behavior is driven by workflow YAML, Jinja2 prompts, and a data-source
registry — not by hard-coded logic.Primary boundaries:
- Backend Python package:
src/aiq_agent/.- Data-source and tool packages:
sources/(each is its own package).- Frontends and tooling:
frontends/(web UI infrontends/ui/, eval harnesses
infrontends/benchmarks/).- Configs, deployment, docs:
configs/,deploy/,docs/.Stay inside this repository. If your workspace also contains adjacent repos
(for example a sibling NeMo-Relay checkout), do not edit them as part of an AI-Q
change. Treatsources/*as independent packages: prefer the smallest change
scoped to the package you are touching.Repository structure
Path Purpose src/aiq_agent/Backend agent, FastAPI extensions, auth, observability, knowledge sources/Data-source / tool packages (e.g. tavily_web_search,google_scholar_paper_search)configs/Workflow YAML configs (e.g. config_cli_default.yml)frontends/ui/Next.js / React / TypeScript / Tailwind / KUI web UI frontends/benchmarks/Eval harnesses: freshqa,deepsearch_qa,deepresearch_benchdeploy/Docker Compose and Helm/Kubernetes assets; deploy/.envfor secretsdocs/source/...
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pyscripts/smoke_openshell_isolation.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.py
src/aiq_agent/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/aiq_agent/**/*.py: Respect authenticated data sources by honoring requires_auth, per-user token pass-through, and backend token validators; apply owner guardrails before loading protected report or artifact context into an agent
Do not weaken or bypass AuthMiddleware, validators, or auth gating without a prior design discussion
Files:
src/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
🔇 Additional comments (9)
frontends/aiq_api/src/aiq_api/jobs/runner.py (1)
658-683: LGTM! Thefinalizefast-path now logs a warning on a falsycleanup_succeededresult, matching the previously requested fix and theTestTerminalTeardowntest suite (test_runtime_finalizer_false_result_is_logged).src/aiq_agent/agents/deep_researcher/deepagents_runtime.py (1)
36-36: LGTM!Also applies to: 92-95, 108-143, 160-162, 191-209, 279-322
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
36-36: LGTM!Also applies to: 46-46, 132-260, 272-273, 295-295, 316-316, 334-334, 390-392, 408-503, 530-602
src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
102-103: LGTM!Also applies to: 145-157, 204-216
tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py (1)
16-43: LGTM!Also applies to: 71-161, 163-270, 271-705, 828-870
tests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py (1)
20-21: LGTM!Also applies to: 37-37, 307-331, 332-335, 337-429
tests/aiq_agent/jobs/test_runner.py (1)
1911-1919: LGTM!Also applies to: 1921-1931
tests/aiq_agent/agents/deep_researcher/test_agent.py (1)
292-337: LGTM!scripts/smoke_openshell_isolation.py (1)
1-52: LGTM!Also applies to: 55-73, 75-96, 98-142, 145-155, 156-184, 185-188, 192-197
KyleZheng1284
force-pushed
the
codex/aiq-openshell-isolation
branch
from
b05c93b to
24264c4
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
frontends/aiq_api/src/aiq_api/jobs/runner.py (1)
666-672: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueDocstring doesn't mention the new
finalize()-first path.The docstring only describes the terminate()/close() routing, but the primary path (lines 675-682) now delegates to
finalize()when present, falling back to terminate/close only for runtimes without it. Worth a one-line update for future maintainers.📝 Suggested docstring update
"""Release sandbox resources on a terminal path (best-effort, never raises). + Prefers ``finalize(interrupted=...)`` when the runtime exposes it (logs a warning on a + falsy/failed result). Falls back to the legacy routing below for runtimes without a + ``finalize`` method: Interrupted jobs (cancel/timeout) call ``terminate()`` so a still-running ``execute`` is forcibly preempted; normal paths call ``close()`` gracefully. Both are idempotent. This runs off the event loop (``asyncio.to_thread``) so the SDK session close cannot block the worker. """🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@frontends/aiq_api/src/aiq_api/jobs/runner.py` around lines 666 - 672, Update the _teardown_sandbox docstring in runner.py to mention the new finalize()-first behavior before the existing terminate()/close() fallback description. Keep the note brief but explicit that sandbox_runtime.finalize() is used when available, with terminate() for interrupted jobs and close() for the remaining fallback path. Use the _teardown_sandbox symbol so maintainers can quickly find the routing logic.src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
155-156: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winSanitize base lifecycle exception logging.
Both event emission and session cleanup are best-effort paths; avoid
exc_info=Trueso secret-bearing callback/SDK details are not written to logs.Suggested fix
- except Exception: # noqa: BLE001 - event persistence is non-critical - logger.warning("Sandbox event emission failed for %s", self.sandbox_name, exc_info=True) + except Exception as exc: # noqa: BLE001 - event persistence is non-critical + logger.warning("Sandbox event emission failed for %s (%s)", self.sandbox_name, type(exc).__name__) @@ - except Exception: # noqa: BLE001 - cleanup must never raise on the terminal path + except Exception as exc: # noqa: BLE001 - cleanup must never raise on the terminal path self._cleanup_failed = True - logger.warning("Sandbox %s cleanup failed", self.sandbox_name, exc_info=True) + logger.warning("Sandbox %s cleanup failed (%s)", self.sandbox_name, type(exc).__name__)As per coding guidelines, “Never print or log secret values, including in tool output or error messages.”
Also applies to: 209-211
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/sandbox/base.py` around lines 155 - 156, The best-effort exception logging in the sandbox lifecycle currently includes exc_info=True, which can leak secret-bearing callback or SDK details into logs. Update the exception handlers in the base lifecycle methods (including the sandbox event emission path and the related session cleanup path) to log only a sanitized warning message without exception stack traces or raw exception objects, keeping the existing logger.warning calls and using the same identifiers like self.sandbox_name for context.Source: Coding guidelines
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
598-602: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winSanitize SDK cleanup exception logging.
The cleanup path should report failure without logging raw SDK exception text or traceback details.
Suggested fix
- except Exception: # noqa: BLE001 - cleanup must never raise on the terminal path + except Exception as exc: # noqa: BLE001 - cleanup must never raise on the terminal path self._cleanup_failed = True - logger.warning("OpenShell sandbox %s context cleanup failed", self.sandbox_name, exc_info=True) + logger.warning( + "OpenShell sandbox %s context cleanup failed (%s)", + self.sandbox_name, + type(exc).__name__, + )As per coding guidelines, “Never print or log secret values, including in tool output or error messages.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py` around lines 598 - 602, The OpenShell cleanup path in the ctx.__exit__ exception handler is logging raw SDK exception details via exc_info=True, which can expose sensitive data. Update the cleanup logging in openshell.py to report only that sandbox context cleanup failed, without including the exception text or traceback, while still preserving the _cleanup_failed flag and the existing warning in the cleanup flow.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/aiq_agent/agents/deep_researcher/deepagents_runtime.py`:
- Around line 294-295: The cleanup warning in the terminal exception handler is
logging raw exception details via exc_info=True, which can leak sensitive
SDK/event-sink messages. Update the exception handling in the sandbox cleanup
path around the cleanup logic in deepagents_runtime.py (including the related
block near the other cleanup handler mentioned in the comment) to log only the
exception type or a sanitized summary, and remove full traceback/exception
payload logging from logger.warning while keeping the failure signal.
- Around line 154-156: The shared-sandbox alias handling in DeepAgentsRuntime is
too permissive because self.existing_sandbox_name or self.sandbox_name silently
prefers one value when both are set. Update the validation around shared_name so
DeepAgentsRuntime explicitly detects when existing_sandbox_name and deprecated
sandbox_name are both provided with different values and raises a ValueError
instead of choosing one. Keep the existing allow_shared_sandbox guard, and use
the existing_sandbox_name/sandbox_name checks in the same block to enforce a
single unambiguous sandbox name.
In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py`:
- Around line 419-424: The attestation log in the OpenShell sandbox flow is
including an environment-specific gateway value that should not be emitted.
Update the logger.info call in the attestation path of the OpenShell provider to
remove oscfg.gateway from the message and its argument list, while keeping the
remaining identifiers like backend.id, sandbox_ref.name, policy_version, and
shared so the log stays useful without exposing deployment identifiers.
---
Outside diff comments:
In `@frontends/aiq_api/src/aiq_api/jobs/runner.py`:
- Around line 666-672: Update the _teardown_sandbox docstring in runner.py to
mention the new finalize()-first behavior before the existing
terminate()/close() fallback description. Keep the note brief but explicit that
sandbox_runtime.finalize() is used when available, with terminate() for
interrupted jobs and close() for the remaining fallback path. Use the
_teardown_sandbox symbol so maintainers can quickly find the routing logic.
In `@src/aiq_agent/agents/deep_researcher/sandbox/base.py`:
- Around line 155-156: The best-effort exception logging in the sandbox
lifecycle currently includes exc_info=True, which can leak secret-bearing
callback or SDK details into logs. Update the exception handlers in the base
lifecycle methods (including the sandbox event emission path and the related
session cleanup path) to log only a sanitized warning message without exception
stack traces or raw exception objects, keeping the existing logger.warning calls
and using the same identifiers like self.sandbox_name for context.
In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py`:
- Around line 598-602: The OpenShell cleanup path in the ctx.__exit__ exception
handler is logging raw SDK exception details via exc_info=True, which can expose
sensitive data. Update the cleanup logging in openshell.py to report only that
sandbox context cleanup failed, without including the exception text or
traceback, while still preserving the _cleanup_failed flag and the existing
warning in the cleanup flow.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 870175a1-89e7-4cd7-b6c1-709d76de1100
📒 Files selected for processing (9)
frontends/aiq_api/src/aiq_api/jobs/runner.pyscripts/smoke_openshell_isolation.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/jobs/test_runner.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (6)
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run ruff check and ruff format validation for Python code changes
**/*.py: Python code must be linted and formatted with Ruff using line length 120, target Python 3.11, rule sets E,F,W,I,PL,UP, and isort force-single-line configuration
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr instead, resolving API keys at runtime
Never print or log secret values, including in tool output or error messages
Missing-secret paths must degrade gracefully (stub/skip), not crash or leak
Do not hand-reformat unrelated code when making changes; match the existing import and formatting style
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pytests/aiq_agent/jobs/test_runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pyscripts/smoke_openshell_isolation.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py
**/*test*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run pytest for all behavior changes in Python code
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/jobs/test_runner.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py
**
⚙️ CodeRabbit configuration file
**:AI-Q Agent Guidance
Repository-global instructions for coding agents and for humans reviewing
agent-authored changes. These rules apply to every task in this repository.
Task-specific runbooks live in.agents/skills/— load the
relevant skill before starting a workflow it covers.Project overview
AI-Q is an NVIDIA AI Blueprint: an enterprise research agent built on the
NeMo Agent Toolkit (NAT). The deployed product is a research blueprint, not
a general skill runtime. New retrieval sources and tools are NAT functions;
agent behavior is driven by workflow YAML, Jinja2 prompts, and a data-source
registry — not by hard-coded logic.Primary boundaries:
- Backend Python package:
src/aiq_agent/.- Data-source and tool packages:
sources/(each is its own package).- Frontends and tooling:
frontends/(web UI infrontends/ui/, eval harnesses
infrontends/benchmarks/).- Configs, deployment, docs:
configs/,deploy/,docs/.Stay inside this repository. If your workspace also contains adjacent repos
(for example a sibling NeMo-Relay checkout), do not edit them as part of an AI-Q
change. Treatsources/*as independent packages: prefer the smallest change
scoped to the package you are touching.Repository structure
Path Purpose src/aiq_agent/Backend agent, FastAPI extensions, auth, observability, knowledge sources/Data-source / tool packages (e.g. tavily_web_search,google_scholar_paper_search)configs/Workflow YAML configs (e.g. config_cli_default.yml)frontends/ui/Next.js / React / TypeScript / Tailwind / KUI web UI frontends/benchmarks/Eval harnesses: freshqa,deepsearch_qa,deepresearch_benchdeploy/Docker Compose and Helm/Kubernetes assets; deploy/.envfor secretsdocs/source/...
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pytests/aiq_agent/jobs/test_runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pyscripts/smoke_openshell_isolation.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py
src/aiq_agent/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/aiq_agent/**/*.py: Respect authenticated data sources by honoring requires_auth, per-user token pass-through, and backend token validators; apply owner guardrails before loading protected report or artifact context into an agent
Do not weaken or bypass AuthMiddleware, validators, or auth gating without a prior design discussion
Files:
src/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.py
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.py
🔇 Additional comments (10)
frontends/aiq_api/src/aiq_api/jobs/runner.py (2)
675-682: 🩺 Stability & AvailabilityFalsy
finalize()return now logged — past review comment resolved.This correctly closes the gap flagged previously: a non-raising
finalize()failure is no longer silently dropped.
639-664: LGTM!src/aiq_agent/agents/deep_researcher/deepagents_runtime.py (1)
193-196: LGTM!Also applies to: 210-211, 281-293, 297-303
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
36-46: LGTM!Also applies to: 271-295, 316-334, 390-418, 427-503, 530-597
src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
102-103: LGTM!Also applies to: 145-154, 213-216
tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py (1)
26-27: LGTM!Also applies to: 37-37, 82-83, 123-130, 400-438, 458-498, 525-532, 535-709, 849-876
tests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py (1)
20-21: LGTM!Also applies to: 336-343, 388-437
tests/aiq_agent/agents/deep_researcher/test_agent.py (1)
292-337: LGTM!tests/aiq_agent/jobs/test_runner.py (1)
1911-1924: LGTM!Also applies to: 1935-1945
scripts/smoke_openshell_isolation.py (1)
75-95: LGTM!Also applies to: 166-171
Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>
KyleZheng1284
force-pushed
the
codex/aiq-openshell-isolation
branch
from
24264c4 to
8e2287b
Compare
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
597-601: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winSanitize OpenShell context cleanup failures.
exc_info=Truecan log raw SDK exception messages during terminal cleanup. Log only the exception type while preserving_cleanup_failed = True. As per coding guidelines, “Never print or log secret values, including in tool output or error messages.”Suggested fix
- except Exception: # noqa: BLE001 - cleanup must never raise on the terminal path + except Exception as exc: # noqa: BLE001 - cleanup must never raise on the terminal path self._cleanup_failed = True - logger.warning("OpenShell sandbox %s context cleanup failed", self.sandbox_name, exc_info=True) + logger.warning( + "OpenShell sandbox %s context cleanup failed (%s)", + self.sandbox_name, + type(exc).__name__, + )🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py` around lines 597 - 601, The OpenShell cleanup warning in the context exit path is leaking raw exception details via exc_info=True, which can expose sensitive SDK messages. In the ctx.__exit__ exception handler inside the OpenShell sandbox cleanup logic, keep setting _cleanup_failed = True but change the logger.warning call to report only the exception type (using the caught exception object) without full traceback or message details, and keep the context tied to the existing sandbox_name identifier.Source: Coding guidelines
src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
149-156: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winDo not log raw event-sink or cleanup exceptions.
Both handlers use
exc_info=True, which can emit secret-bearing exception messages from SDKs or persistence sinks. Keep these best-effort paths non-fatal but log onlytype(exc).__name__. As per coding guidelines, “Never print or log secret values, including in tool output or error messages.”Suggested fix
- except Exception: # noqa: BLE001 - event persistence is non-critical - logger.warning("Sandbox event emission failed for %s", self.sandbox_name, exc_info=True) + except Exception as exc: # noqa: BLE001 - event persistence is non-critical + logger.warning("Sandbox event emission failed for %s (%s)", self.sandbox_name, type(exc).__name__) @@ - except Exception: # noqa: BLE001 - cleanup must never raise on the terminal path + except Exception as exc: # noqa: BLE001 - cleanup must never raise on the terminal path self._cleanup_failed = True - logger.warning("Sandbox %s cleanup failed", self.sandbox_name, exc_info=True) + logger.warning("Sandbox %s cleanup failed (%s)", self.sandbox_name, type(exc).__name__)Also applies to: 204-211
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/aiq_agent/agents/deep_researcher/sandbox/base.py` around lines 149 - 156, The best-effort error handlers in _emit_event and the cleanup path are logging exception details with exc_info=True, which can leak secret-bearing messages from SDKs or persistence sinks. Update these handlers to keep failures non-fatal but log only the exception type name (for example via the caught Exception object), and remove traceback/raw exception output from the logger.warning calls while preserving sandbox_name context.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@frontends/aiq_api/src/aiq_api/jobs/runner.py`:
- Around line 680-681: The sandbox cleanup finalizer in `runner.py` still logs
full tracebacks via `logger.warning(..., exc_info=True)`, which can leak
secret-bearing details from unexpected SDK/event-sink exceptions. Update the
`finalize()` exception handler to log only the exception type/class name (using
the existing `job_id` context) and remove traceback emission so `Sandbox cleanup
failed for job %s` stays sanitized.
In `@tests/aiq_agent/jobs/test_runner.py`:
- Around line 1935-1945: The _teardown_sandbox path in aiq_api.jobs.runner still
logs finalize() exceptions with exc_info=True, which can leak raw exception
messages. Update the exception handling around runtime.finalize() to log only
the exception type or a sanitized summary, and add a regression test alongside
test_runtime_finalizer_false_result_is_logged that raises a RuntimeError with
sensitive text to confirm the warning does not expose the secret.
---
Outside diff comments:
In `@src/aiq_agent/agents/deep_researcher/sandbox/base.py`:
- Around line 149-156: The best-effort error handlers in _emit_event and the
cleanup path are logging exception details with exc_info=True, which can leak
secret-bearing messages from SDKs or persistence sinks. Update these handlers to
keep failures non-fatal but log only the exception type name (for example via
the caught Exception object), and remove traceback/raw exception output from the
logger.warning calls while preserving sandbox_name context.
In `@src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py`:
- Around line 597-601: The OpenShell cleanup warning in the context exit path is
leaking raw exception details via exc_info=True, which can expose sensitive SDK
messages. In the ctx.__exit__ exception handler inside the OpenShell sandbox
cleanup logic, keep setting _cleanup_failed = True but change the logger.warning
call to report only the exception type (using the caught exception object)
without full traceback or message details, and keep the context tied to the
existing sandbox_name identifier.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 30bb7492-02b8-46c4-87b0-b11bb1797bbf
📒 Files selected for processing (9)
frontends/aiq_api/src/aiq_api/jobs/runner.pyscripts/smoke_openshell_isolation.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pytests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/jobs/test_runner.py
📜 Review details
🧰 Additional context used
📓 Path-based instructions (6)
**/*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run ruff check and ruff format validation for Python code changes
**/*.py: Python code must be linted and formatted with Ruff using line length 120, target Python 3.11, rule sets E,F,W,I,PL,UP, and isort force-single-line configuration
Never commit secrets, tokens, or environment-specific hostnames; use environment variables and SecretStr instead, resolving API keys at runtime
Never print or log secret values, including in tool output or error messages
Missing-secret paths must degrade gracefully (stub/skip), not crash or leak
Do not hand-reformat unrelated code when making changes; match the existing import and formatting style
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pyscripts/smoke_openshell_isolation.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py
**/*test*.py
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run pytest for all behavior changes in Python code
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py
**
⚙️ CodeRabbit configuration file
**:AI-Q Agent Guidance
Repository-global instructions for coding agents and for humans reviewing
agent-authored changes. These rules apply to every task in this repository.
Task-specific runbooks live in.agents/skills/— load the
relevant skill before starting a workflow it covers.Project overview
AI-Q is an NVIDIA AI Blueprint: an enterprise research agent built on the
NeMo Agent Toolkit (NAT). The deployed product is a research blueprint, not
a general skill runtime. New retrieval sources and tools are NAT functions;
agent behavior is driven by workflow YAML, Jinja2 prompts, and a data-source
registry — not by hard-coded logic.Primary boundaries:
- Backend Python package:
src/aiq_agent/.- Data-source and tool packages:
sources/(each is its own package).- Frontends and tooling:
frontends/(web UI infrontends/ui/, eval harnesses
infrontends/benchmarks/).- Configs, deployment, docs:
configs/,deploy/,docs/.Stay inside this repository. If your workspace also contains adjacent repos
(for example a sibling NeMo-Relay checkout), do not edit them as part of an AI-Q
change. Treatsources/*as independent packages: prefer the smallest change
scoped to the package you are touching.Repository structure
Path Purpose src/aiq_agent/Backend agent, FastAPI extensions, auth, observability, knowledge sources/Data-source / tool packages (e.g. tavily_web_search,google_scholar_paper_search)configs/Workflow YAML configs (e.g. config_cli_default.yml)frontends/ui/Next.js / React / TypeScript / Tailwind / KUI web UI frontends/benchmarks/Eval harnesses: freshqa,deepsearch_qa,deepresearch_benchdeploy/Docker Compose and Helm/Kubernetes assets; deploy/.envfor secretsdocs/source/...
Files:
tests/aiq_agent/agents/deep_researcher/test_agent.pytests/aiq_agent/jobs/test_runner.pysrc/aiq_agent/agents/deep_researcher/sandbox/base.pyfrontends/aiq_api/src/aiq_api/jobs/runner.pytests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.pyscripts/smoke_openshell_isolation.pytests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py
src/aiq_agent/**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
src/aiq_agent/**/*.py: Respect authenticated data sources by honoring requires_auth, per-user token pass-through, and backend token validators; apply owner guardrails before loading protected report or artifact context into an agent
Do not weaken or bypass AuthMiddleware, validators, or auth gating without a prior design discussion
Files:
src/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py
src/aiq_agent/agents/**/*
⚙️ CodeRabbit configuration file
src/aiq_agent/agents/**/*: Review agent changes for research workflow correctness, graph state transitions, prompt/tool contracts,
HITL behavior, and failure handling. Flag changes that weaken source attribution, report generation,
async cancellation, checkpointing, or data-source selection without focused tests and docs.
Files:
src/aiq_agent/agents/deep_researcher/sandbox/base.pysrc/aiq_agent/agents/deep_researcher/deepagents_runtime.pysrc/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}
⚙️ CodeRabbit configuration file
{src/aiq_agent/fastapi_extensions/**,frontends/aiq_api/src/aiq_api/**}: Treat API, auth, and job-runner changes as externally visible contracts. Check authorization boundaries,
request tracing, async job lifecycle, websocket reconnect behavior, error responses, and cross-user data isolation.
Require tests for route behavior, access decisions, and job state transitions when those surfaces change.
Files:
frontends/aiq_api/src/aiq_api/jobs/runner.py
🔇 Additional comments (9)
src/aiq_agent/agents/deep_researcher/deepagents_runtime.py (1)
154-162: LGTM!Also applies to: 202-217, 287-309, 328-329
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.py (1)
36-46: LGTM!Also applies to: 272-295, 316-334, 390-502, 529-596
src/aiq_agent/agents/deep_researcher/sandbox/base.py (1)
102-103: LGTM!Also applies to: 213-216
frontends/aiq_api/src/aiq_api/jobs/runner.py (1)
641-663: LGTM!Also applies to: 678-679
scripts/smoke_openshell_isolation.py (1)
75-95: LGTM!Also applies to: 166-189
tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py (1)
26-27: LGTM!Also applies to: 37-37, 82-83, 123-130, 271-318, 411-449, 469-509, 536-720, 860-887
tests/aiq_agent/agents/deep_researcher/test_agent.py (1)
292-337: LGTM!tests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py (1)
20-22: LGTM!Also applies to: 337-352, 398-463
tests/aiq_agent/jobs/test_runner.py (1)
1911-1934: LGTM!
The generated policy listed /app under filesystem_policy.read_only, but the
aiq-openshell-demo image has no /app directory. Under the default
landlock.compatibility: hard_requirement this fails closed at sandbox prepare
("Landlock path unavailable in hard_requirement mode: /app"), so the container
reaches Ready then immediately errors. Removing the path fixes prepare on all
hosts; production keeps hard_requirement, best_effort stays opt-in for local demos.
Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>
Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>
Signed-off-by: Kyle Zheng <kyzheng@nvidia.com>
2 participants
Overview
Create and own one physical OpenShell sandbox per AI-Q job, with shared attachment retained only behind the explicit debug opt-in. Per-job creation now uses a secret-free
SandboxSpec, validates one policy snapshot against the OpenShell 0.0.72 schema and the production security floor, and requires READY state plus a positive (optionally pinned) loaded policy revision before execution.The lifecycle boundary covers context entry, attestation, adapter construction, workspace preparation, cancellation during creation, and terminal cleanup. Both API jobs and the registered NAT/CLI path finalize their runtime. Cleanup remains idempotent, reports failure truthfully, and emits sanitized attestation/cleanup events without policy contents, environment values, credentials, or raw exception text.
The review follow-up serializes concurrent finalizers around the real cleanup result, makes OpenShell context entry/exit race-safe, delays attestation success until the adapter is usable, and preserves cumulative fail-closed cleanup status across retry teardown. It also rejects conflicting shared-sandbox aliases, sanitizes terminal cleanup warnings, and keeps gateway identifiers out of attestation logs. OpenShell 0.0.72's authoritative loaded-policy RPC is used as a compatibility fallback when its generic sandbox status leaves
current_policy_versionat zero.This PR also updates the canonical config, policy, setup flow, and sandbox documentation. Artifact lifecycle and enforceable resource/watchdog controls remain in the two unopened follow-up branches.
Validation
How to test: per-job sandbox isolation + cleanup
Setup (once):
Build the sandbox image and generate the policy:
./scripts/setup_openshell.shStart the AI-Q backend + UI with the OpenShell config:
./scripts/start_e2e.sh --config_file configs/config_openshell.ymlRun the test:
In a terminal, watch sandboxes appear/disappear live (leave this running):
watch -n 1 'docker ps --filter name=openshell --format "{{.Names}}\t{{.Status}}"'
Open two browser tabs at http://localhost:3000, each with its own session (append a distinct ?session=... or start a new chat in each), and kick off a deep research job in both at the same time — use a prompt that forces computation/charts (e.g. "compare the last 4 quarters of capex for Cisco and AMD with a chart"), so each job hits the execute tool.
Watch the docker ps output: you should see two separate openshell-* containers spin up (one per job) when each job reaches its first execute, each showing Up/healthy (not Restarting).
Verify cleanup:
Another test to run is
uv run pytest -q tests/aiq_agent/agents/deep_researcher/sandbox tests/aiq_agent/agents/deep_researcher/test_deepagents_runtime.py tests/aiq_agent/agents/deep_researcher/test_agent.py tests/aiq_agent/fastapi_extensions/test_deep_research.py tests/aiq_agent/jobs/test_runner.py— 293 passed in 2.39s.With
openshell==0.0.72,langchain-nvidia-openshell==0.1.0, and lockeddeepagents==0.6.8:uv run pytest -q tests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.py— 35 passed in 0.89s.Deterministic remediation suite from the plan (
test_openshell_provider.py,test_sandbox_runtime.py,test_deepagents_runtime.py,test_agent.py, andtest_runner.py) — 239 passed in 2.24s.uv run ruff check .— All checks passed.uv run ruff format --check .— 253 files already formatted.uv run nat validate --config_file configs/config_openshell.yml— configuration valid.bash -n scripts/setup_openshell.sh— passed.cd docs && uv run make html— succeeded with 3 existing cross-reference warnings.Exact OpenShell 0.0.72 live smoke on macOS Docker using the documented explicit local-demo Landlock exception:
The production default remains
landlock.compatibility: hard_requirement; the best-effort flag was used only because the local macOS Docker host cannot enforce Landlock. A final gateway listing confirmed that both probe sandboxes and the temporary diagnostic sandbox were deleted; the pre-existing debug sandbox was preserved.I ran the relevant local checks or explained why they are not applicable.
I added or updated tests for behavior changes.
I updated documentation for user-facing or contributor-facing changes.
I confirmed this PR does not include secrets, credentials, or internal-only data.
I certify this contribution under the Developer Certificate of Origin (DCO) and signed my commits with
git commit -sor an equivalent sign-off.Where should reviewers start?
Start with
src/aiq_agent/agents/deep_researcher/sandbox/providers/openshell.pyfor policy/spec/attestation/context ownership, thentests/aiq_agent/agents/deep_researcher/sandbox/test_openshell_provider.pyfor the deterministic race and live-version compatibility cases. Concurrent cleanup ownership is indeepagents_runtime.py; registered NAT/CLI cancellation and API-runner observability are covered by their focused tests.Related Issues
Summary by CodeRabbit
New Features
Bug Fixes
Documentation