Add ApplyGuardrailScope for security guardrail evaluation tracing#252
Merged
Conversation
Introduce a new OpenTelemetry tracing scope (ApplyGuardrailScope) that captures security guardrail evaluations as spans. This enables observability into content safety, policy enforcement, and risk assessment decisions made during agent operations. New components: - ApplyGuardrailScope: Disposable tracing scope for guardrail spans - GuardrailDetails: Contract for guardian evaluation metadata (target, decision, policy, provider info) - GuardrailFinding: Individual risk finding with severity and category - GuardrailDecisionType, GuardrailRiskSeverity, GuardrailTargetType: Enums for well-known guardrail values - ApplyGuardrailData/Builder: DTO and builder for ETW event logging - OpenTelemetryConstants: New attribute keys for guardrail spans Also includes unit tests and integration test coverage for the new scope. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add AddSdkAttributes method to BaseDataBuilder that includes telemetry.sdk.name, telemetry.sdk.version, and telemetry.sdk.language in ETW event payloads, ensuring parity with the OpenTelemetry span pipeline where ActivityProcessor sets these same attributes. Updated all concrete builders (InvokeAgent, ExecuteInference, ExecuteTool, OutputMessages, ApplyGuardrail) to call AddSdkAttributes at the start of BuildAttributes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Scanned FilesNone |
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds a new ApplyGuardrailScope OpenTelemetry tracing scope (plus ETW DTOs/logging) to emit guardrail evaluation spans and finding events for security/guardrail observability.
Changes:
- Introduces
ApplyGuardrailScopeand new guardrail contracts (GuardrailDetails,GuardrailFinding, decision/target/severity types) with newmicrosoft.security.*/microsoft.guardian.*attributes. - Adds ETW support via
ApplyGuardrailData+ builder and extendsIA365EtwLogger/A365EtwLoggerwithLogApplyGuardrail. - Adds unit + integration tests covering span creation and finding event emission, and adds SDK attributes to multiple DTO builders.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Tests/Microsoft.Agents.A365.Observability.Runtime.Tests/Tracing/Scopes/ApplyGuardrailScopeTest.cs | New unit tests validating guardrail span tags, naming, decision updates, findings, and errors. |
| src/Tests/Microsoft.Agents.A365.Observability.Runtime.IntegrationTests/Agent365ExporterE2ETests.cs | Adds an integration-style test that captures an Activity for ApplyGuardrailScope and asserts emitted tags/events. |
| src/Observability/Runtime/Tracing/Scopes/OpenTelemetryScope.cs | Adds a protected helper to add ActivityEvent to the current activity. |
| src/Observability/Runtime/Tracing/Scopes/OpenTelemetryConstants.cs | Adds operation name constant and guardrail-related semantic attribute keys. |
| src/Observability/Runtime/Tracing/Scopes/ApplyGuardrailScope.cs | New scope implementation that sets guardrail/guardian/policy/content tags and emits finding events. |
| src/Observability/Runtime/Tracing/Contracts/GuardrailTargetType.cs | New extensible target type wrapper with well-known values + implicit string conversions. |
| src/Observability/Runtime/Tracing/Contracts/GuardrailRiskSeverity.cs | Adds well-known risk severity string constants. |
| src/Observability/Runtime/Tracing/Contracts/GuardrailFinding.cs | New finding contract used to emit per-finding events/tags. |
| src/Observability/Runtime/Tracing/Contracts/GuardrailDetails.cs | New guardrail evaluation details contract used to populate scope attributes. |
| src/Observability/Runtime/Tracing/Contracts/GuardrailDecisionType.cs | New enum of guardrail decisions (allow/audit/deny/modify/warn). |
| src/Observability/Runtime/Etw/IA365EtwLogger.cs | Adds LogApplyGuardrail API to the ETW logger interface. |
| src/Observability/Runtime/Etw/A365EtwLogger.cs | Implements LogApplyGuardrail and assigns event id/name. |
| src/Observability/Runtime/DTOs/Builders/OutputDataBuilder.cs | Adds SDK attributes to Output DTO attributes. |
| src/Observability/Runtime/DTOs/Builders/InvokeAgentDataBuilder.cs | Adds SDK attributes to InvokeAgent DTO attributes. |
| src/Observability/Runtime/DTOs/Builders/ExecuteToolDataBuilder.cs | Adds SDK attributes to ExecuteTool DTO attributes. |
| src/Observability/Runtime/DTOs/Builders/ExecuteInferenceDataBuilder.cs | Adds SDK attributes to ExecuteInference DTO attributes. |
| src/Observability/Runtime/DTOs/Builders/BaseDataBuilder.cs | Adds shared AddSdkAttributes helper. |
| src/Observability/Runtime/DTOs/Builders/ApplyGuardrailDataBuilder.cs | New builder for ETW DTO attribute population for apply_guardrail. |
| src/Observability/Runtime/DTOs/ApplyGuardrailData.cs | New ETW DTO representing apply_guardrail telemetry. |
- Throw ArgumentNullException instead of silently no-oping on null, matching the non-nullable parameter signature. - Remove unused System.Runtime.Serialization import. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
juliomenendez
approved these changes
May 19, 2026
fpfp100
approved these changes
May 20, 2026
nikhilNava
added a commit
to microsoft/opentelemetry-distro-dotnet
that referenced
this pull request
May 27, 2026
* Add ApplyGuardrailScope for security guardrail evaluation tracing Introduces a new OpenTelemetry tracing scope (ApplyGuardrailScope) that captures security guardrail evaluations as spans. This enables observability into content safety, policy enforcement, and risk assessment decisions made during agent operations. New contracts: - GuardrailDetails: Immutable contract for guardian evaluation metadata - GuardrailFinding: Individual risk finding with severity and score - GuardrailDecisionType: Enum for guardian decisions (Allow, Deny, etc.) - GuardrailRiskSeverity: String constants for risk severity levels - GuardrailTargetType: Extensible type for guardrail targets New tracing scope: - ApplyGuardrailScope: Disposable OTel scope with RecordDecision, RecordContentOutput, and RecordFinding methods Also adds: - microsoft.security.* and microsoft.guardian.* OTel attribute keys - AddEvent helper on OpenTelemetryScope base class - AddSdkAttributes helper on BaseDataBuilder - ETW logging support via LogApplyGuardrail - ApplyGuardrailData DTO and builder - 12 unit tests covering scope creation and attribute emission Ref: microsoft/Agent365-dotnet#252 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add apply_guardrail to exporter allow-list and add test The ApplyGuardrailOperationName was missing from the GenAiOperationNames set in Agent365ExporterCore, which would cause guardrail spans to be filtered out as NonGenAI in PartitionByIdentity. Also adds a test verifying apply_guardrail spans are included in exported batches. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Add AddSdkAttributes call to ApplyGuardrailDataBuilder Aligns with the other DTO builders (Output, InvokeAgent, ExecuteTool, ExecuteInference) which all call AddSdkAttributes as the first step in BuildAttributes to include telemetry.sdk.name/version/language. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Nikhil Chitlur Navakiran (from Dev Box) <nikhilc@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces a new OpenTelemetry tracing scope (\ApplyGuardrailScope) that captures security guardrail evaluations as spans. This enables observability into content safety, policy enforcement, and risk assessment decisions made during agent operations.
Changes
New Contracts
New Tracing Scope
ETW Support
OpenTelemetry Constants
Tests
Design Notes
Manual/Integration test
below is the generated span