Init E2E Integration Test#189
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
Introduces an initial end-to-end (E2E) GitHub Actions workflow to spin up multiple agent samples (Python/Node.js/.NET), provision runtime configuration, and validate basic health/message endpoints using a shared set of PowerShell helper scripts.
Changes:
- Adds a new GitHub Actions workflow to run E2E integration checks for agent samples.
- Adds PowerShell utilities to acquire MCP bearer tokens, generate runtime config files, start/stop agents, and capture logs.
- Adds documentation for the E2E scripts and required CI secrets.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
.github/workflows/e2e-agent-samples.yml |
New E2E workflow that installs dependencies, starts each sample agent, validates endpoints, and collects logs/cleanup. |
scripts/e2e/Acquire-BearerToken.ps1 |
Acquires an MCP bearer token via ROPC for CI use. |
scripts/e2e/Generate-EnvConfig.ps1 |
Generates .env files for Python/Node agents from secret mappings. |
scripts/e2e/Generate-AppSettings.ps1 |
Updates .NET appsettings.json with mapped config values. |
scripts/e2e/Start-Agent.ps1 |
Starts an agent process and waits until it is reachable via health/messages endpoints. |
scripts/e2e/Stop-AgentProcess.ps1 |
Stops an agent process and kills any listeners on the test port. |
scripts/e2e/Capture-AgentLogs.ps1 |
Prints agent logs and shows redacted .env/wrapper script content for debugging. |
scripts/e2e/Copy-ToolingManifest.ps1 |
Writes a standardized ToolingManifest.json into the sample directory for E2E runs. |
scripts/e2e/README.md |
Documents scripts, MCP auth approach, and required secrets. |
| try { | ||
| $response = Invoke-WebRequest -Uri $messagesUrl -Method GET -UseBasicParsing -TimeoutSec 5 -ErrorAction Stop | ||
| } | ||
| catch { | ||
| if ($_.Exception.Response.StatusCode.value__ -eq 405) { | ||
| $ready = $true | ||
| Write-Host "Agent is ready! (messages endpoint returned 405)" -ForegroundColor Green | ||
| break | ||
| } |
There was a problem hiding this comment.
In the messages-endpoint fallback, the catch block dereferences $_.Exception.Response.StatusCode.value__ without checking whether Response is present. For connection failures/timeouts, Response can be null and this will throw inside the catch, aborting the readiness wait loop. Handle non-HTTP errors by null-checking Response (or using the exception type) before reading StatusCode, and only treat 405 as ready when an HTTP response exists.
| $wrapperScript = Join-Path $AgentPath "run-agent.ps1" | ||
| $escapedToken = $BearerToken -replace "'", "''" | ||
|
|
||
| $scriptLines = @( | ||
| "`$env:PORT = '$Port'" | ||
| "`$env:ASPNETCORE_URLS = 'http://localhost:$Port'" | ||
| "`$env:BEARER_TOKEN = '$escapedToken'" | ||
| "`$env:PYTHONIOENCODING = 'utf-8'" |
There was a problem hiding this comment.
This script writes the bearer token into run-agent.ps1 on disk ($env:BEARER_TOKEN = '...'). That leaves a plaintext secret in the repo working directory, which is easy to leak if the workspace is reused or artifacts are collected. Prefer setting env vars in the current process before Start-Process (child inherits them) or write the wrapper into a temp path and delete it once the process is started/after cleanup.
| # ============================================================================= | ||
| # E2E Tests: Agent Samples | ||
| # ============================================================================= | ||
| # Runs E2E integration tests for all agent samples | ||
| # Uses external PowerShell scripts for maintainability | ||
| # ============================================================================= | ||
|
|
||
| name: E2E Agent Samples |
There was a problem hiding this comment.
This workflow is missing the standard Microsoft copyright header and a minimal permissions: block. Other workflows in this repo consistently include both (e.g., .github/workflows/ci-orchestrator.yml:1-14). Add the header and set least-privilege permissions such as contents: read at the workflow or job level.
| MCP_ENVIRONMENT: 'Development' | ||
|
|
||
| # Common settings | ||
| TEST_TIMEOUT: 120 |
There was a problem hiding this comment.
TEST_TIMEOUT is defined as a workflow env var but is never used. Either wire it into Start-Agent.ps1 via -TimeoutSeconds ${{ env.TEST_TIMEOUT }} (or the E2E test step timeouts) or remove it to avoid configuration drift.
| TEST_TIMEOUT: 120 |
Restored E2E test workflow configuration for agent samples, including job definitions and environment variables.
Restored E2E tests for agent samples with detailed job configurations for Python, Node.js, and .NET agents. Added summary generation for test results.
Updated PowerShell script to enhance error handling and diagnostics.
Updated the Start-Agent.ps1 script to include additional runtime checks for Python, Node.js, and .NET environments. Enhanced logging and error handling for better diagnostics during agent startup.
- Add e2e-agent-samples.yml workflow for all 4 sample types - Add PowerShell scripts for agent management (Start-Agent, token acquisition, etc.) - Add E2E test project with HTTP integration tests - Add MockBotFrameworkServer to capture agent responses - Tests are self-contained - no external repo dependencies Merge conflict resolve
rahuldevikar761
dismissed stale reviews from tmlsousa and abdulanu0
via
f3d964e
- Add Microsoft copyright header to workflow file - Remove unused TEST_TIMEOUT env var from workflow - Move wrapper script to temp directory (not repo working dir) - Pass BEARER_TOKEN via env var inheritance instead of writing to script - Delete wrapper script after process starts for additional security
No description provided.