Add W365 Computer Use sample agent#305
Merged
Merged
Conversation
- Add ConversationSession class to track per-conversation W365 sessions - Refactor ComputerUseOrchestrator to use ConcurrentDictionary keyed by conversationId - Parse sessionId from QuickStartSession response and pass to all MCP tool calls - Pass conversationId from turnContext to orchestrator - Add deployment artifacts to .gitignore (a365 configs, app.zip, publish/)
…loop Instead of sending the full conversation history (including all base64 screenshots) on every model call, use the OpenAI Responses API's previous_response_id to let the server reconstruct prior context. Only new items (computer_call_output, function_call_output) are sent per iteration, reducing API payload by ~15x. Between user messages, computer actions and screenshots are pruned from history while text context is preserved for conversational continuity. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
feat: use previous_response_id to avoid resending screenshots in CUA …
…r screenshots Two fixes: 1. Keep the last computer_call + computer_call_output pair when pruning history between user messages. The API requires a matching computer_call for every computer_call_output (linked by call_id) — dropping one causes BadRequest: "No tool call found for computer call with call_id". This also gives the model visual context for simple follow-ups. 2. Add session recovery to the screenshot capture path, matching the existing pattern used by action tools (click, type, scroll). If CaptureScreenshot returns "no active session", recover the session and retry once — instead of failing outright. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use the W365 session ID instead of truncated conversation ID for the OneDrive screenshot subfolder. Updated in both initial session start and session recovery paths. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Address review feedback — rename cryptic loop variables (i, j, t, ct, cid, ccid) to descriptive names (histIdx, searchIdx, entryType, earlierType, outputCallId, etc.). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Support multiple MCP server URLs (W365 + MailTools) with per-server HttpClient - Fix _cachedTools overwrite bug that dropped mail tools on second message - Add function tool instructions to system prompt so model prefers them over CUA - Per-server error handling so one server failure doesn't block others - Fix CancellationTokenSource disposal races in typing indicator - Make CUA session start message ephemeral (informative update) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix CUA orchestrator session recovery and screenshot handling
- ToolingManifest.json: add mcp_MailTools entry so Mail MCP is discovered in prod - ComputerUseOrchestrator: restore onFolderLinkReady callback, session-ID-based screenshot subfolder, ShareConversationFolderAsync, FolderShared flag, and have UploadScreenshotToOneDriveAsync return the share URL - ComputerUseOrchestrator: log function_call args and returned output for troubleshooting MCP tool invocations - ComputerUseOrchestrator: update system prompt so model tells the user "I can't" when no matching tool exists (instead of silently calling OnTaskComplete) - MyAgent.cs: pass onFolderLinkReady callback that posts a View-folder link Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Additive port of 407b43c features that don't conflict with our multi-mcp work: - System prompt: mention EndSession and casual-chat behavior - Add EndSession as a model-callable function tool - Handle EndSession in function_call: tear down W365 session, drop state, return "Session ended..." text to the user - EndSessionAsync: add catch for HttpRequestException 404 (MCP transport already expired — no need to warn) - Transparent session recovery during CUA actions: detect session-not-found tool responses, end the stale session, start a fresh one, and retry - Pin A365 SDK package versions to 0.1.72-beta (was beta.*) - Add nuget.config (clear + nuget.org only) Skipped: HasActiveSession / EndConversationSessionAsync public methods (not called anywhere on the target branch). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two targeted changes to eliminate a post-response crash on the `computer`
tool path (gpt-5.4-mini):
fail: Kestrel[13] ConnectionAbortedException
→ InvalidOperationException: Reading is already in progress
Unhandled: ObjectDisposedException (HttpRequestPipeReader)
1. MyAgent.cs: drop the background typing-indicator Task.Run loop. The
loop fired SendActivityAsync every ~4s concurrently with the main
reply path, and the resulting race against StreamingResponse.EndStream
triggered the crash. Keep a single initial typing activity; informative
updates via onStatusUpdate/onCuaStarting already cover visual feedback.
2. Program.cs: call request.EnableBuffering() at the /api/messages
endpoint so observability/tracing middleware can re-read the body
without hitting "Reading is not allowed after reader was completed".
Validated against a long CUA session (10+ iterations), a mixed
CUA/email/chat exchange, and parallel requests — no crashes observed in
the test window that previously reproduced the bug within minutes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Register the full stock-MCP set so the A365 SDK gateway surfaces tools from each server at runtime: mcp_W365ComputerUse, mcp_MailTools, mcp_MeServer, mcp_CalendarTools, mcp_TeamsServer, mcp_ODSPRemoteServer, mcp_SharepointListsTools, mcp_AdminTools, mcp_WordServer, mcp_m365copilot Loads ~149 function tools in prod when the blueprint has the matching McpServers.*.All inheritable scopes consented. Note: mcp_SharepointListsTools currently fails to load with an ObjectDisposedException inside the A365 SDK's MCP client factory (CancellationTokenSource). Appears to be an SDK-side issue — the other nine servers load cleanly. Not addressed here. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Multi-MCP tools, mail + EndSession, OneDrive folder link restored
… noise Rename V2 references to the canonical single W365 server name now that ATG merged its V1/V2 implementations into one file-based remote MCP server: - ComputerUseOrchestrator: V2CuaToolNames -> W365CuaToolNames, IsV2CuaTool -> IsW365CuaTool, and the ATG-local EndSession tool becomes mcp_W365ComputerUse_EndSession. - MyAgent: follow-up call-site rename. - README: replace V1 QuickStartSession/W365_Click2/W365_WriteText prose with the transparent-checkout + dynamic-remote-tool model. Log-volume cleanup (dev ergonomics, no runtime change): - Drop the OpenTelemetry AddConsoleExporter in development — it dumped a full Activity block per HTTP request and swamped the console. - Raise System.Net.Http to Warning in appsettings.json. - Demote the take_screenshot raw-JSON preview + Model request/response 2KB dumps in ComputerUseOrchestrator to LogDebug.
Keeps the prod settings file (ClientSecret auth config) out of git. Mirrors the existing pattern for appsettings.Development.json.
…quiring status Before: every user message eagerly loaded W365 tools via tools/list, which triggers ATG's hostname-discovery handler and acquires a Cloud PC session (10-30s cold start). Even a trivial "hi" paid the full cost. Now: - Add ClassifyNeedsCuaAsync: a cheap tool-less LLM call that decides whether the message needs desktop control or session management. Biased toward YES when uncertain (safer to pay the session cost than miss a CUA request). - Add RunAsync(includeCuaTool: bool): when false, strip the computer tool, OnTaskComplete, and EndSession from the model's tools — only caller- provided function tools (mail/calendar/etc.) remain visible. - Split _mcpServerUrls in MyAgent into _w365McpServerUrls and _otherMcpServerUrls (by URL substring match on /mcp_W365ComputerUse). Non-CUA path now loads only the non-W365 servers, so ATG never fires its W365 hostname-discovery handler for chit-chat or mail-only flows. - Add GetOrCreateNonW365McpConnectionAsync in the orchestrator with its own cache (non-W365 tools don't need the Error-tool retry gating that the W365 cache uses). - On the CUA path, surface "Acquiring a Windows 365 Cloud PC session…" via QueueInformativeUpdateAsync right before the W365 tools/list call, so the user sees progress during the 10-30s session acquisition instead of apparent hang. Closes the "getting session" UX gap. Prod note: A365 SDK loads all registered servers in one call; per-server gating isn't exposed today, so in production the session is still acquired eagerly. The CUA compute gate still saves iteration cost on the non-CUA branch. A per-server prod API would be the full fix. appsettings.Development.json (gitignored) updated locally to add mcp_MailTools alongside mcp_W365ComputerUse — mail tools now flow through the non-CUA path without touching W365. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…nse begins Previously 'Got it — working on it…' was sent via SendActivityAsync from inside the CUA loop's onCuaStarting callback, after the streaming response had already started. The chat UI then ordered the 'Got it' activity AFTER the streaming activity's final text (because the streaming activity was created earlier in the turn), so users saw the result before the acknowledgment — confusing. Move the SendActivity to the top of the CUA branch in OnMessageAsync, before the first QueueInformativeUpdateAsync. The 'Got it' activity now has an earlier ID than the streaming activity and renders in the expected visual order. Side effect: 'Got it' now fires on every CUA-routed message rather than only when the model emits a computer_call. That's the right semantics — the classifier routed us to CUA, so 'working on it' is accurate regardless of whether the model ends up actually using the desktop. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously the 'Acquiring a Windows 365 Cloud PC session…' informative update fired on every CUA-routed message, including warm reuses where tools were already cached and ATG had a live session. Misleading — the user saw an "acquiring" status for operations that resolved instantly. Add HasCachedW365Tools on ComputerUseOrchestrator (true iff the agent has already loaded a W365 tools/list in this process), and gate the streaming update on !HasCachedW365Tools. On warm reuse the user sees only "Got it — working on it…" and the final result. Note: this doesn't cover server-side idle reap (~30 min on ATG). If ATG's cached session is reaped while the agent still has tools cached, the first failing tool call triggers RecoverSessionAsync in the CUA loop which emits its own 'Session lost — recovering...' status. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When W365 tools/list fails (empty w365Tools), we were sending the error via a separate SendActivityAsync. The still-open streaming response then finalized empty and the framework emitted a 'No text was streamed' bubble in addition to the real error — two confusing messages for one failure. Write the error text into the streaming response via QueueTextChunk so EndStreamAsync sees a real payload. The 'Got it' acknowledgment stays as a separate earlier activity; only the final error message changes surface from SendActivity to streaming text. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Azure OpenAI rejects requests with this 400 when the conversation history contains a `computer_call_output` item but neither `computer` nor `computer_use_preview` tool is declared in the current turn's tool set: "You provided computer_output, but neither 'computer' nor 'computer_use_preview' tool is enabled." The two-phase router can route turn N to the CUA path (history accrues computer_call/computer_call_output items) and then turn N+1 to the non-CUA path (computer tool stripped). The leftover history makes turn N+1 invalid. Filter computer_call and computer_call_output items out of the conversation passed to the model when includeCuaTool is false. Session state (session.ConversationHistory) is left untouched so future CUA turns still see the full record. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds a new .NET sample agent (dotnet/w365-computer-use/) that demonstrates running a “computer use” loop against a Windows 365 Cloud PC via the W365 Computer Use MCP server, including optional function-tool routing and OneDrive screenshot upload/sharing.
Changes:
- Introduces a new Agent Framework sample project/solution with configuration, local/production setup docs, and deployment guidance.
- Implements a CUA orchestrator that translates
computer_callactions into MCP tool invocations, captures screenshots, and iterates until completion/end-session. - Adds basic OpenTelemetry wiring + custom Activity/Meter helpers and an ASP.NET JWT token validation helper.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| dotnet/w365-computer-use/W365ComputerUseSample.sln | New solution file wiring up the sample-agent project. |
| dotnet/w365-computer-use/sample-agent/W365ComputerUseSample.csproj | New web project with Agent Framework + MCP + OTEL dependencies. |
| dotnet/w365-computer-use/sample-agent/ToolingManifest.json | Declares the W365 MCP server metadata (reference material for tooling). |
| dotnet/w365-computer-use/sample-agent/telemetry/AgentMetrics.cs | Adds custom Activity/Meter helpers for agent/http operations (contains a critical async/finalization bug). |
| dotnet/w365-computer-use/sample-agent/telemetry/A365OtelWrapper.cs | Wraps observed operations with baggage + observability token cache registration (has agentId fallback bug). |
| dotnet/w365-computer-use/sample-agent/ServiceExtensions.cs | Adds OpenTelemetry resource/tracing/metrics configuration for the sample. |
| dotnet/w365-computer-use/sample-agent/README.md | End-to-end documentation (setup, Playground testing, production deployment, troubleshooting). |
| dotnet/w365-computer-use/sample-agent/Properties/launchSettings.json | Local dev profile for http://localhost:3978. |
| dotnet/w365-computer-use/sample-agent/Program.cs | ASP.NET host setup, /api/messages + /api/health endpoints, and agent registration. |
| dotnet/w365-computer-use/sample-agent/nuget.config | Pins package source configuration to nuget.org. |
| dotnet/w365-computer-use/sample-agent/ComputerUse/Models/ComputerUseModels.cs | DTOs for Azure OpenAI Responses “computer use” request/response/tool definitions. |
| dotnet/w365-computer-use/sample-agent/ComputerUse/ICuaModelProvider.cs | Abstraction for sending serialized Responses API requests. |
| dotnet/w365-computer-use/sample-agent/ComputerUse/ComputerUseOrchestrator.cs | Core CUA loop, action→MCP mapping, screenshot capture, OneDrive upload/share (has a couple concrete runtime/URL/cancellation issues). |
| dotnet/w365-computer-use/sample-agent/ComputerUse/AzureOpenAIModelProvider.cs | Azure OpenAI Responses API client using API key auth. |
| dotnet/w365-computer-use/sample-agent/AspNetExtensions.cs | Adds JWT bearer validation helper (contains token parsing robustness issues). |
| dotnet/w365-computer-use/sample-agent/appsettings.json | Default configuration with placeholders for AOAI + connection + screenshot settings. |
| dotnet/w365-computer-use/sample-agent/Agent/MyAgent.cs | Agent turn handling: intent classification fast-path, tool loading, CUA orchestration, streaming updates (has async-callback misuse). |
| dotnet/w365-computer-use/sample-agent/a365.config.example.json | Example a365 CLI config template with placeholders. |
| dotnet/w365-computer-use/sample-agent/.gitignore | Ignores local overrides, screenshots, and deploy artifacts for this sample. |
desmarest
force-pushed
the
users/bertd/w365-computer-use-sample
branch
from
ffe7c23 to
8725549
Compare
…oning items
- EndSession prompt + tool description: scope explicitly to releasing the
Cloud PC / VM, with negative examples ("close all apps", "close window")
so the model stops conflating in-VM closes with session termination.
- ClassifierInstructions: rewritten as explicit YES/NO bullet list adding
URL navigation, web-form fill, and "tell me what's on this page" as YES,
fixing the non-CUA refusal path for prompts like "Go to <url>".
- CUA loop: persist 'reasoning' output items in conversation history; the
Responses API rejects subsequent turns echoing a computer_call without
its paired reasoning rs_… (gpt-5.4 / gpt-5.4-pro requirement). Mirror in
the non-CUA history filter so reasoning items are stripped alongside the
computer_calls they pair with.
- OnTaskComplete: capture the latest 'message' text into finalAnswerText
during the per-turn foreach instead of returning early on the first
message. The OnTaskComplete branch now returns finalAnswerText (or a
diagnostic fallback pointing at the screenshots folder) instead of the
hardcoded "Task completed successfully." string. System prompt updated
to require an answer message before OnTaskComplete is called.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a `narrate(reason)` function tool the model calls to push
short, present-tense status sentences to Playground's live
informative-update banner. Replaces the per-action
"Performing: click..." spam with model-driven narration that
explains the plan in human terms — "Clicking Search to run the
query", "Opening the first trial to read its details", etc.
- New FunctionToolDefinition registered alongside OnTaskComplete /
EndSession with a JSON schema for the single `reason` string arg.
- Dispatch arm in the function_call branch parses arguments,
forwards the reason via onStatusUpdate (already wired to
QueueInformativeUpdateAsync in MyAgent.cs), acks the call, and
continues the loop. Non-terminal: doesn't end the run.
- System prompt section telling the model when to narrate (every
2-3 actions, before major clicks/fills/scrolls, at phase
transitions) and when not to (trivially repetitive sequences).
Explicitly states narrate is NOT a substitute for the final
answer message before OnTaskComplete.
- Suppress the per-action "Performing: {actionType}..." banner in
HandleComputerCallAsync so the live banner is reserved for
model-driven narrations. Session-lifecycle banners (acquire /
end / recover) still fire.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The W365 press_keys tool was rejecting "ArrowDown" with "Unknown key name: 'arrowdown'" because the OpenAI computer-use model emits W3C-flavored key names (ArrowDown, Control, Escape, Meta, …) while W365 expects the bare canonical names (down, control, escape, win, …). OpenAI's CUA docs explicitly recommend a client-side normalization helper for this: https://developers.openai.com/api/docs/guides/tools-computer-use#3-run-every-returned-action Adds CuaActionNormalization with: - NormalizeKey: map W3C aliases (Arrow*/Ctrl/Esc/Cmd/Meta/PgUp/PgDn/etc.) to W365 canonical names. Unknown keys pass through lowercased so single- character keys (letters, digits, punctuation) keep working. Covers F1-F24, modifiers, navigation, locks, print screen / pause / context menu. - NormalizeMouseButton: lowercase->PascalCase, with wheel->Middle. Wires the helper into MapActionToMcpTool for click, double_click, triple_click (new — previously threw NotSupportedException), drag, and keypress. double_click and drag now respect the model-supplied button instead of hardcoding Left. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- start W365 sessions explicitly and pass sessionId to remote tools - track multiple W365 sessions per conversation and support explicit selection - pass sessionId to EndSession and clear stale local state - normalize CUA key names for W365 press_keys - update OpenTelemetry package versions to remove build warning
desmarest
force-pushed
the
users/bertd/w365-computer-use-sample
branch
from
8725549 to
1a6cd42
Compare
- Add ComputerUse/W365McpSessionClient.cs to issue tools/list with _meta.sessionId, matching the latest A365 MCP-Platform W365 partner extension behavior (explicit-session model). Without this, tools/list against the W365 server only returns lifecycle tools and the CUA loop has no click/type/screenshot tools to invoke. - Orchestrator: in production, bypass the SDK Tooling Gateway for W365 and connect directly to the W365 gateway via SSE with the w365-scoped agentic token; cache and dispose the per-session IMcpClient; thread prestartedW365SessionId through RunAsync. - MyAgent: add a separate w365 agentic auth handler (scope da81128c-e5b5-4f9e-8d89-50d906f107c5/.default), build a per-(conversation,user) session key, and start the W365 session at the tool-discovery step in production. - Trim ToolingManifest.json to just mcp_W365ComputerUse (only server the sample uses). - Remove all references to the OBO auth handler from MyAgent and appsettings. - README: production now uses the direct W365 gateway, document W365:GatewayUrl, and note screenshots land under per-session subfolders. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
CustomEndpointProvider targets Microsoft's internal model resource proxy (BICEvaluationService) for pre-GA CUA model builds — hard-coded URL shape, MS-internal headers, MSAL cert auth against a 1P scope. Adds no value for external customers and confuses the public sample. AzureOpenAIModelProvider already covers the supported customer path (both computer-use-preview and gpt-5.4 family). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Substantive bug fixes (Copilot reviewer): - AgentMetrics.InvokeObservedAgentOperation: now properly async, awaits func(), tracks success based on exceptions, and only finalizes the Activity after the operation completes. - AspNetExtensions.OnMessageReceived: case-insensitive Bearer scheme check; wrap JwtSecurityToken parsing in try/catch so malformed/opaque tokens produce 401 instead of 500; remove redundant null-conditional after IsNullOrEmpty guard; drop the undisposed HttpClient passed to ConfigurationManager (use the overload without it); refactor audience validation to LINQ Where. - A365OtelWrapper.ResolveTenantAndAgentId: null-check turnContext up front; use string.IsNullOrEmpty(agentId) instead of the no-op '?? Guid.Empty.ToString()'; narrow generic catch around observability registration. - ComputerUseOrchestrator: guard conversationId prefix length (avoid ArgumentOutOfRangeException on short conv ids); URL-encode _oneDriveUserId in Graph paths so UPNs produce valid URLs; thread CancellationToken through Upload/Share OneDrive flows; narrow generic catches to filtered (HttpRequestException/JsonException/FormatException) with OperationCanceledException re-thrown. - onStatusUpdate callback: change from Action<string> to Func<string, Task> and await it at all call sites in MyAgent and orchestrator — exceptions no longer get swallowed and ConfigureAwait(false) on an unawaited Task is no longer a no-op. Style cleanups (github-code-quality bot): - MyAgent.WelcomeMessageAsync and ExtractW365ToolListError: replace implicit-filter foreach with explicit LINQ Where/OfType/FirstOrDefault. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Shorter, more generic prompt: - Drop domain-specific narration examples (ClinicalTrials/GLP-1/Obesity) that don't belong in a public reference sample - Stop telling the model to call StartSession itself — the agent pre-starts the session and attaches sessionId, so this guidance contradicted reality - Make finalAnswer the single path for user-visible answers in OnTaskComplete (the previous two-path 'emit message OR pass finalAnswer' guidance produced inconsistent runs) - Trim narration section from rule-heavy to 3 lines on when/what/what-it-isn't - Drop the misleading 'log me out' EndSession example (ambiguous with in-VM sign-out) - Remove the duplicated 'greetings → text' line at top and bottom Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Convert implicit-filter foreach loops to explicit LINQ (Where/FirstOrDefault) per github-code-quality bot: - MyAgent.WelcomeMessageAsync: use discard '_' in foreach since the element value is unused - Orchestrator ClassifyNeedsCuaAsync: pre-filter response.Output to message items - Orchestrator RunAsync cuaOnlyCallIds collection: extract HashSet of names and filter ConversationHistory in one LINQ pass - Orchestrator TryExtractToolError: filter content array to text blocks - Orchestrator and W365McpSessionClient TryExtractStringProperty: introduce TryExtractStringPropertyTuple helper so the recursive object/array traversals can use Select+FirstOrDefault instead of foreach with implicit early return - Orchestrator and W365McpSessionClient TryGetProperty: use FirstOrDefault on EnumerateObject Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Substantive: - SaveScreenshotToDisk: normalize subfolder/name through Path.GetFileName so a rooted or parent-traversal segment can't make Path.Combine drop _screenshotPath - ExtractScreenshot: log JsonException at Debug instead of swallowing silently (still falls through to raw-base64 last resort) - ExtractBase64FromText: comment the intentional swallow so the empty catch isn't ambiguous - StartDirectW365SessionAndListToolsAsync: collapse nested ifs into a single guard Style: - ClassifyNeedsCuaAsync: convert message-item foreach to LINQ Where+Select+FirstOrDefault - ExtractText: same pattern - ExtractPngBase64FromJson Walk: convert object/array recursion to Select+FirstOrDefault - cuaOnlyCallIds collection: build via Where+Select+ToHashSet Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
MohamedAbdekader
approved these changes
Jun 3, 2026
ajmfehr
approved these changes
Jun 9, 2026
gwharris7
approved these changes
Jun 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
dotnet/w365-computer-use/as a new sample alongsideagent-framework,autonomous, andsemantic-kernel.The sample demonstrates an Agent 365 agent that controls a Windows 365 Cloud PC via the W365 Computer Use MCP server using Azure OpenAI's computer-use model (
computer-use-previewandgpt-5.4/gpt-5.4-miniare both supported).Highlights
computer_callactions into MCP tool invocations, captures screenshots, and feeds them back to the model until it emitsOnTaskCompleteorEndSession.{yyyy-MM-dd}/{HHmmss}_{slug}) with org-scoped sharing links surfaced to the user inline.Errorsentinel tool out of the LLM tool list to avoid Azure OpenAIinvalid_function_parametersrejections, while still reading its description for user-facing error messaging.appsettings.Development.jsonClientSecret override) and full Azure App Service production deployment (resource creation, App Service config withAuthType=ClientSecret+AuthorityEndpoint,dotnet publish+az webapp deploy, Bot Channels Registration in Foundry, blueprint linking, anda365 publishfor Teams app upload).a365.config.example.jsoncommitted as a placeholder template since the CLI no longer ships aconfig initcommand.