Skip to content

skills-eval: upload results as a scrubbed tarball (strip agent/ secrets)#301

Closed
sarathm-nvidia wants to merge 1 commit into
NVIDIA-AI-Blueprints:developfrom
sarathm-nvidia:skills-eval-tar-scrub-results
Closed

skills-eval: upload results as a scrubbed tarball (strip agent/ secrets)#301
sarathm-nvidia wants to merge 1 commit into
NVIDIA-AI-Blueprints:developfrom
sarathm-nvidia:skills-eval-tar-scrub-results

Conversation

@sarathm-nvidia

Copy link
Copy Markdown
Collaborator

Problem

.github/workflows/skills-eval.yml uploads Harbor eval results as a raw directory:

path: /tmp/aiq-skill-eval/results/${{ github.run_id }}-${{ github.run_attempt }}

Two issues:

  1. Secret leak. The raw dir ships the per-trial agent/ subtree intact, which can contain session credentials, tokens, and API keys. VSS deliberately strips it at CI time with tar --exclude='agent' after a real NGC key leak (VSS PR #516). AIQ leaks the same way today.
  2. Inconsistent artifact shape. VSS and RAG upload a single scrubbed .tar.gz; AIQ's raw dir forces the downstream skills-dashboard to re-tar it to persist evidence.

Fix

Add a Package Harbor results (scrub agent tree) step before the upload that tars the run's results dir with --exclude='agent' into /tmp/aiq-skill-eval/results.tar.gz, and point upload-artifact at that tarball. Same artifact name:, if-no-files-found: ignore, and retention-days: 7 are preserved. Mirrors VSS's --exclude='agent' convention (PR #516) so no secrets leave CI, and matches the RAG/VSS single-tarball artifact pattern.

Independence

This is independent of and stackable with #299 (nightly schedule). It touches only the results-upload steps and is separately reviewable.

Signed-off-by: Sarath Maddala <sarathm@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Jul 2, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 16a06ad8-2f84-45b9-a261-309697747933

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant