Skip to content

Commit a54ffdc

Browse files
LangChain sample to E2E workflow (#196)
* Add Claude and LangChain samples to E2E workflow - Add nodejs-claude, python-claude, and nodejs-langchain jobs - Add SDK version logging for all samples (Get-SDKVersions.ps1) - Add prettified test conversation output (Emit-TestConversations.ps1) - Add PR comment with test summary (skips manual/scheduled runs) - Update summary job to include all 7 samples with status icons - Add pull-requests: write permission for PR comments * fix(e2e): Fix PowerShell variable parsing in Emit-TestConversations.ps1 Use \ instead of \ * fix(e2e): Add agentic_scopes to nodejs-claude and nodejs-langchain jobs Required for AgenticAuthorization handler * Address PR review comments - Fix pipe character escaping in Emit-TestConversations.ps1 (use \\| instead of \|) - Fix Windows-specific path construction in Get-SDKVersions.ps1 (use nested Join-Path) - Fix package filtering to exclude A365 packages from Agents SDK list - Fix bash status icon syntax (use if-else instead of command substitution) * Remove Claude samples from E2E workflow * Use Node.js OpenAI secrets for LangChain sample * Fix LangChain sample: add health endpoint and read PORT from env * Update LangChain sample to support both Azure OpenAI and OpenAI - Add flexible model configuration that checks for Azure OpenAI first - Fall back to regular OpenAI if Azure credentials not present - Update .env.example with both options documented - Provides clear error message if no credentials found * Split E2E in its own workflow for better management (#198) * fix: Correct YAML syntax in update-e2e-status workflow * fix: Apply Copilot review suggestions - proper URL encoding, empty string handling, textwrap.dedent, and job count tracking * Split into multiple workflows * refactor: Use workflow_call instead of workflow_dispatch for orchestrator - removes need for actions:write permission * fix: Correct PowerShell parameter names - use -AgentPID instead of -ProcessId, remove invalid -OutputPath and -TestName parameters * fix: Remove invalid test filter - test names BasicConversation and Notification don't exist * chore: Remove deprecated monolithic E2E workflow * chore: Remove deprecated monolithic E2E workflow * Add E2E workflow enhancements - Add LangChain E2E workflow (e2e-nodejs-langchain.yml) - Add SDK version logging step to all E2E workflows - Add test conversation emission step to all E2E workflows - Add PR test summary comment to orchestrator (only on PR events) - Update orchestrator to include LangChain and add pull-requests: write permission * Fix YAML syntax error in e2e-orchestrator.yml - use array join for PR comment body * Add debug logging to PR comment step * Fix bot comment detection - use login name instead of type * Remove debug logging from PR comment step * Add logging to diagnose PR comment update issue --------- Co-authored-by: Rahul Devikar <rahuldevikar@gmail.com>
1 parent 4573f48 commit a54ffdc

11 files changed

Lines changed: 881 additions & 11 deletions

.github/workflows/e2e-dotnet-agent-framework.yml

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,13 @@ jobs:
4949
working-directory: ${{ env.SAMPLE_PATH }}
5050
run: dotnet build --no-restore
5151

52+
- name: Log SDK Versions
53+
shell: pwsh
54+
run: |
55+
& "${{ env.SCRIPTS_PATH }}/Get-SDKVersions.ps1" `
56+
-Runtime "dotnet" `
57+
-WorkingDirectory "${{ env.SAMPLE_PATH }}"
58+
5259
- name: Acquire Bearer Token (ROPC)
5360
id: token
5461
shell: pwsh
@@ -125,6 +132,15 @@ jobs:
125132
--verbosity normal `
126133
--logger "console;verbosity=detailed" `
127134
--logger "trx;LogFileName=test-results-dotnet-af.trx"
135+
env:
136+
TEST_RESULTS_DIR: ${{ runner.temp }}/TestConversations
137+
138+
- name: Emit Test Conversations
139+
if: always()
140+
shell: pwsh
141+
run: |
142+
& "${{ env.SCRIPTS_PATH }}/Emit-TestConversations.ps1" `
143+
-TestResultsDir "${{ runner.temp }}/TestConversations"
128144
129145
- name: Capture Agent Logs
130146
if: always()

.github/workflows/e2e-dotnet-semantic-kernel.yml

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,13 @@ jobs:
4949
working-directory: ${{ env.SAMPLE_PATH }}
5050
run: dotnet build --no-restore
5151

52+
- name: Log SDK Versions
53+
shell: pwsh
54+
run: |
55+
& "${{ env.SCRIPTS_PATH }}/Get-SDKVersions.ps1" `
56+
-Runtime "dotnet" `
57+
-WorkingDirectory "${{ env.SAMPLE_PATH }}"
58+
5259
- name: Acquire Bearer Token (ROPC)
5360
id: token
5461
shell: pwsh
@@ -125,6 +132,15 @@ jobs:
125132
--verbosity normal `
126133
--logger "console;verbosity=detailed" `
127134
--logger "trx;LogFileName=test-results-dotnet-sk.trx"
135+
env:
136+
TEST_RESULTS_DIR: ${{ runner.temp }}/TestConversations
137+
138+
- name: Emit Test Conversations
139+
if: always()
140+
shell: pwsh
141+
run: |
142+
& "${{ env.SCRIPTS_PATH }}/Emit-TestConversations.ps1" `
143+
-TestResultsDir "${{ runner.temp }}/TestConversations"
128144
129145
- name: Capture Agent Logs
130146
if: always()
Lines changed: 185 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,185 @@
1+
# Copyright (c) Microsoft Corporation.
2+
# Licensed under the MIT License.
3+
4+
name: E2E - Node.js LangChain
5+
6+
on:
7+
workflow_call: # Allow orchestrator to call this workflow
8+
workflow_dispatch: # Allow manual triggering
9+
push:
10+
branches: [main]
11+
paths:
12+
- 'nodejs/langchain/**'
13+
- 'scripts/e2e/**'
14+
- '.github/workflows/e2e-nodejs-langchain.yml'
15+
pull_request:
16+
branches: [main]
17+
paths:
18+
- 'nodejs/langchain/**'
19+
- 'scripts/e2e/**'
20+
21+
permissions:
22+
contents: read
23+
24+
env:
25+
SAMPLE_PATH: nodejs/langchain/sample-agent
26+
AGENT_PORT: 3979
27+
SCRIPTS_PATH: scripts/e2e
28+
E2E_TESTS_PATH: tests/e2e
29+
30+
jobs:
31+
nodejs-langchain:
32+
name: Node.js LangChain Agent
33+
runs-on: windows-latest
34+
35+
steps:
36+
- name: Checkout repository
37+
uses: actions/checkout@v4
38+
39+
- name: Setup .NET SDK
40+
uses: actions/setup-dotnet@v4
41+
with:
42+
dotnet-version: '8.0.x'
43+
44+
- name: Setup Node.js
45+
uses: actions/setup-node@v4
46+
with:
47+
node-version: '20'
48+
49+
- name: Install dependencies
50+
working-directory: ${{ env.SAMPLE_PATH }}
51+
run: |
52+
if (Test-Path "package-lock.json") {
53+
npm ci
54+
} else {
55+
npm install
56+
}
57+
if (Test-Path "tsconfig.json") {
58+
npm run build
59+
}
60+
shell: pwsh
61+
62+
- name: Log SDK Versions
63+
shell: pwsh
64+
run: |
65+
& "${{ env.SCRIPTS_PATH }}/Get-SDKVersions.ps1" `
66+
-Runtime "nodejs" `
67+
-WorkingDirectory "${{ env.SAMPLE_PATH }}"
68+
69+
- name: Acquire Bearer Token (ROPC)
70+
id: token
71+
shell: pwsh
72+
run: |
73+
$token = & "${{ env.SCRIPTS_PATH }}/Acquire-BearerToken.ps1" `
74+
-ClientId "${{ secrets.MCP_CLIENT_ID }}" `
75+
-TenantId "${{ secrets.MCP_TENANT_ID }}" `
76+
-Username "${{ secrets.MCP_TEST_USERNAME }}" `
77+
-Password "${{ secrets.MCP_TEST_PASSWORD }}"
78+
echo "BEARER_TOKEN=$token" >> $env:GITHUB_OUTPUT
79+
echo "::add-mask::$token"
80+
81+
- name: Copy ToolingManifest.json
82+
shell: pwsh
83+
run: |
84+
& "${{ env.SCRIPTS_PATH }}/Copy-ToolingManifest.ps1" -TargetPath "${{ env.SAMPLE_PATH }}"
85+
86+
- name: Generate .env configuration
87+
shell: pwsh
88+
run: |
89+
$configMappings = @{
90+
"NODE_ENV" = "development"
91+
"AZURE_OPENAI_API_KEY" = "${{ secrets.NODEJS_OPENAI_AZURE_OPENAI_API_KEY }}"
92+
"AZURE_OPENAI_ENDPOINT" = "${{ secrets.NODEJS_OPENAI_AZURE_OPENAI_ENDPOINT }}"
93+
"AZURE_OPENAI_DEPLOYMENT" = "${{ secrets.NODEJS_OPENAI_AZURE_OPENAI_DEPLOYMENT }}"
94+
"AZURE_OPENAI_API_VERSION" = "2024-12-01-preview"
95+
"connections__service_connection__settings__authType" = "ClientSecret"
96+
"connections__service_connection__settings__clientId" = "${{ secrets.NODEJS_OPENAI_AGENT_ID }}"
97+
"connections__service_connection__settings__clientSecret" = "${{ secrets.NODEJS_OPENAI_CLIENT_SECRET }}"
98+
"connections__service_connection__settings__tenantId" = "${{ secrets.TENANT_ID }}"
99+
"connectionsMap__0__serviceUrl" = "*"
100+
"connectionsMap__0__connection" = "service_connection"
101+
"agentic_scopes" = "ea9ffc3e-8a23-4a7d-836d-234d7c7565c1/.default"
102+
}
103+
& "${{ env.SCRIPTS_PATH }}/Generate-EnvConfig.ps1" `
104+
-OutputPath "${{ env.SAMPLE_PATH }}/.env" `
105+
-BearerToken "${{ steps.token.outputs.BEARER_TOKEN }}" `
106+
-Port ${{ env.AGENT_PORT }} `
107+
-ConfigMappings $configMappings
108+
109+
- name: Start Agent
110+
id: start-agent
111+
shell: pwsh
112+
run: |
113+
$agentPid = & "${{ env.SCRIPTS_PATH }}/Start-Agent.ps1" `
114+
-AgentPath "${{ env.SAMPLE_PATH }}" `
115+
-StartCommand "node dist/index.js" `
116+
-Port ${{ env.AGENT_PORT }} `
117+
-BearerToken "${{ steps.token.outputs.BEARER_TOKEN }}" `
118+
-Environment "Development" `
119+
-Runtime "nodejs"
120+
echo "AGENT_PID=$agentPid" >> $env:GITHUB_OUTPUT
121+
122+
- name: Verify Agent Running
123+
shell: pwsh
124+
run: |
125+
$agentPid = "${{ steps.start-agent.outputs.AGENT_PID }}"
126+
if ($agentPid) {
127+
try {
128+
$proc = Get-Process -Id $agentPid -ErrorAction Stop
129+
Write-Host "Agent process (PID: $agentPid) is running: $($proc.ProcessName)" -ForegroundColor Green
130+
} catch {
131+
Write-Host "ERROR: Agent process (PID: $agentPid) is NOT running!" -ForegroundColor Red
132+
throw "Agent process has stopped"
133+
}
134+
}
135+
$agentUrl = "http://localhost:${{ env.AGENT_PORT }}"
136+
$healthResponse = Invoke-WebRequest -Uri "$agentUrl/api/health" -Method GET -UseBasicParsing -ErrorAction SilentlyContinue
137+
Write-Host "Health check: $($healthResponse.StatusCode)" -ForegroundColor Green
138+
139+
- name: Restore E2E Test Dependencies
140+
run: dotnet restore "${{ env.E2E_TESTS_PATH }}/Agent365.E2E.Tests.csproj"
141+
142+
- name: Run .NET E2E Tests
143+
shell: pwsh
144+
run: |
145+
dotnet test "${{ env.E2E_TESTS_PATH }}/Agent365.E2E.Tests.csproj" `
146+
--no-restore `
147+
--verbosity normal `
148+
--logger "console;verbosity=detailed" `
149+
--logger "trx;LogFileName=test-results-nodejs-langchain.trx" `
150+
--filter "FullyQualifiedName~BasicConversation|FullyQualifiedName~Notification"
151+
env:
152+
TEST_RESULTS_DIR: ${{ runner.temp }}/TestConversations
153+
154+
- name: Emit Test Conversations
155+
if: always()
156+
shell: pwsh
157+
run: |
158+
& "${{ env.SCRIPTS_PATH }}/Emit-TestConversations.ps1" `
159+
-TestResultsDir "${{ runner.temp }}/TestConversations"
160+
161+
- name: Capture Agent Logs
162+
if: always()
163+
shell: pwsh
164+
run: |
165+
& "${{ env.SCRIPTS_PATH }}/Capture-AgentLogs.ps1" `
166+
-AgentPath "${{ env.SAMPLE_PATH }}"
167+
168+
- name: Stop Agent Process
169+
if: always()
170+
shell: pwsh
171+
run: |
172+
$agentPid = "${{ steps.start-agent.outputs.AGENT_PID }}"
173+
if ($agentPid) {
174+
& "${{ env.SCRIPTS_PATH }}/Stop-AgentProcess.ps1" -AgentPID $agentPid
175+
}
176+
177+
- name: Upload Test Results
178+
if: always()
179+
uses: actions/upload-artifact@v4
180+
with:
181+
name: test-results-nodejs-langchain
182+
path: |
183+
${{ env.E2E_TESTS_PATH }}/TestResults/**/*.trx
184+
${{ env.E2E_TESTS_PATH }}/TestResults/**/*-logs.txt
185+
retention-days: 7

.github/workflows/e2e-nodejs-openai.yml

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,13 @@ jobs:
5959
}
6060
shell: pwsh
6161

62+
- name: Log SDK Versions
63+
shell: pwsh
64+
run: |
65+
& "${{ env.SCRIPTS_PATH }}/Get-SDKVersions.ps1" `
66+
-Runtime "nodejs" `
67+
-WorkingDirectory "${{ env.SAMPLE_PATH }}"
68+
6269
- name: Acquire Bearer Token (ROPC)
6370
id: token
6471
shell: pwsh
@@ -141,6 +148,15 @@ jobs:
141148
--logger "console;verbosity=detailed" `
142149
--logger "trx;LogFileName=test-results-nodejs-openai.trx" `
143150
--filter "FullyQualifiedName~BasicConversation|FullyQualifiedName~Notification"
151+
env:
152+
TEST_RESULTS_DIR: ${{ runner.temp }}/TestConversations
153+
154+
- name: Emit Test Conversations
155+
if: always()
156+
shell: pwsh
157+
run: |
158+
& "${{ env.SCRIPTS_PATH }}/Emit-TestConversations.ps1" `
159+
-TestResultsDir "${{ runner.temp }}/TestConversations"
144160
145161
- name: Capture Agent Logs
146162
if: always()

0 commit comments

Comments
 (0)