Sync eng/common directory with azure-sdk-tools for PR 16052#49752

Merged

helen229 merged 27 commits into

mainfrom

sync-eng-common-feat/vally-eval-ci-phase1-16052

Jul 8, 2026

Commits on Jul 7, 2026

Port Vally eval CI scripts from PowerShell to JavaScript

helen229
authored and
azure-sdk
committed
Split Vally eval pipeline into reusable templates

helen229
authored and
azure-sdk
committed
Generalize Vally eval templates for skill + live verticals; fix missing lib modules

helen229
authored and
azure-sdk
committed
polish and rename

helen229
authored and
azure-sdk
committed
Switch Vally eval CI to 1ES; quiet teardown-noise warnings

helen229
authored and
azure-sdk
committed
Fix npm ci EPERM under 1ES by restoring via azure-sdk npm mirror

helen229
authored and
azure-sdk
committed
Simplify eval archetype: build Mock+Cli MCP by default, drop never-overridden params and mcpBuilds

helen229
authored and
azure-sdk
committed
Rework MCP build to useCustomMcp + runtime repo detection per review

helen229
authored and
azure-sdk
committed
Copy Vally CLI pin to eng/common/vally-eval and repoint eval pipeline

helen229
authored and
azure-sdk
committed
Move eval-mcp-setup.yml step template into eng/common

helen229
authored and
azure-sdk
committed
Repoint eval-mcp-setup.yml references to eng/common path

helen229
authored and
azure-sdk
committed
Eval CI: move entrypoints/templates to eng/common, rename vally-* templates to eval-*, extract generate-eval-matrix + eval-shard jobs, merge eval package.json, drop variable group, add maxParallel

helen229
authored and
azure-sdk
committed
Eval CI: make eng/common templates MCP- and auth-agnostic

helen229
authored and
azure-sdk
committed
Simplify eval pipeline templates: inline non-varying parameters

helen229
authored and
azure-sdk
committed
Flatten eval pipeline to stages -> jobs (steps inline)

helen229
authored and
azure-sdk
committed
Trim verbose comments across eval pipeline and scripts

helen229
authored and
azure-sdk
committed
Reject eval paths with whitespace; note JUnit parse coupling

helen229
authored and
azure-sdk
committed
Make shard timeout a compile-time parameter

helen229
authored and
azure-sdk
committed
Map copilot PAT into GITHUB_TOKEN env for eval invoke steps

helen229
authored and
azure-sdk
committed
Attach eval variable group to entrypoint pipelines so copilot PAT resolves

helen229
authored and
azure-sdk
committed
Land eval cache on a local branch so worktree fixtures resolve the ref

helen229
authored and
azure-sdk
committed
Consolidate eval invoke into one auth-conditional template and tidy shard/MCP seams

helen229
authored and
azure-sdk
committed
Add eval hooks, parameterize gate threshold, require mcpSetupTemplate

helen229
authored and
azure-sdk
committed
Consolidate MCP templates and add opt-in cross-repo checkout

helen229
authored and
azure-sdk
committed
Fix eval archetype: declare toolsRepo param and default mcpServer

helen229
authored and
azure-sdk
committed
Address review feedback: portable lib/ gitignore, rerun-safe shard artifacts, TestType rename, exec.js note

helen229
authored and
azure-sdk
committed
Always stage debug dir so failed()-gated transcript publish can't error on missing path

helen229
authored and
azure-sdk
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Sync eng/common directory with azure-sdk-tools for PR 16052#49752

Sync eng/common directory with azure-sdk-tools for PR 16052#49752
helen229 merged 27 commits into
mainfrom
sync-eng-common-feat/vally-eval-ci-phase1-16052

Commits on Jul 7, 2026

Port Vally eval CI scripts from PowerShell to JavaScript

Split Vally eval pipeline into reusable templates

Generalize Vally eval templates for skill + live verticals; fix missing lib modules

polish and rename

Switch Vally eval CI to 1ES; quiet teardown-noise warnings

Fix npm ci EPERM under 1ES by restoring via azure-sdk npm mirror

Simplify eval archetype: build Mock+Cli MCP by default, drop never-overridden params and mcpBuilds

Rework MCP build to useCustomMcp + runtime repo detection per review

Copy Vally CLI pin to eng/common/vally-eval and repoint eval pipeline

Move eval-mcp-setup.yml step template into eng/common

Repoint eval-mcp-setup.yml references to eng/common path

Eval CI: move entrypoints/templates to eng/common, rename vally-* templates to eval-*, extract generate-eval-matrix + eval-shard jobs, merge eval package.json, drop variable group, add maxParallel

Eval CI: make eng/common templates MCP- and auth-agnostic

Simplify eval pipeline templates: inline non-varying parameters

Flatten eval pipeline to stages -> jobs (steps inline)

Trim verbose comments across eval pipeline and scripts

Reject eval paths with whitespace; note JUnit parse coupling

Make shard timeout a compile-time parameter

Map copilot PAT into GITHUB_TOKEN env for eval invoke steps

Attach eval variable group to entrypoint pipelines so copilot PAT resolves

Land eval cache on a local branch so worktree fixtures resolve the ref

Consolidate eval invoke into one auth-conditional template and tidy shard/MCP seams

Add eval hooks, parameterize gate threshold, require mcpSetupTemplate

Consolidate MCP templates and add opt-in cross-repo checkout

Fix eval archetype: declare toolsRepo param and default mcpServer

Address review feedback: portable lib/ gitignore, rerun-safe shard artifacts, TestType rename, exec.js note

Always stage debug dir so failed()-gated transcript publish can't error on missing path