Description
Add stream-aware device overlap validation to AIU Trace Analyzer / Acelyzer verification mode.
The existing overlap infrastructure detects and resolves overlaps primarily for trace processing and visualization. This work extends that infrastructure so relevant accelerator execution overlaps can be reported through the verification framework as PASS/FAIL results without modifying the original trace events.
PR #124 (Kernel overlap verification stage, initial version) has been merged and provides the stream-aware accelerator overlap verification stage.
Follow-up validation confirmed that the existing verifier can handle kernel and memory overlap semantics through stream-aware accelerator event grouping without requiring a separate memory-specific overlap algorithm or memory super-events.
PR #126 (Add stream-aware device overlap verification tests) has also been merged and adds regression coverage for the required same-stream and cross-stream device overlap behavior.
The corresponding Torch-Spyre integration work is tracked separately in torch-spyre/torch-spyre#3541.
Expected Behavior
Same stream
- Kernel-kernel overlap → FAIL
- Kernel-memory overlap → FAIL
- Memory-memory overlap → FAIL
- Back-to-back kernel or memory events → PASS
- Nested same-stream execution intervals → FAIL in strict verification mode
Different streams
- Kernel-kernel overlap → PASS
- Kernel-memory overlap → PASS
- Memory-memory overlap → PASS
Events executing on different accelerator streams should not be incorrectly reported simply because their timestamps overlap globally.
Missing stream information
Events without args.stream are currently attributed to stream 0.
Memory events do not yet always carry the correct logical accelerator stream information from Torch-Spyre. Once torch-spyre/torch-spyre#3699 enables the correct stream IDs for memory events, the analyzer can use the emitted stream ID through the existing stream-aware grouping without requiring additional overlap-verification logic.
The analyzer should not rely on thread IDs such as 400 as the logical accelerator stream.
Device Event Handling
Relevant accelerator execution events participate in the existing stream-aware overlap verification.
For Torch traces, accelerator events are grouped by process and args.stream. This allows the existing overlap verifier introduced in #124 to validate kernel and memory events using the same stream-aware overlap logic.
This means separate memory super-events, memory coalescing, or a separate memory-memory overlap detector are not required.
Once memory events contain the correct stream IDs, kernel-kernel, kernel-memory, and memory-memory overlaps can all be evaluated through the existing verifier.
Original trace events should remain unchanged by verification. When stream information is missing, the agreed default-stream behavior is applied for verification and the temporary default should not remain on the original event afterward.
Implementation
The verification implementation should:
- reuse the existing
OverlapDetectionContext infrastructure rather than introduce a separate general overlap algorithm
- perform overlap detection using stream-aware grouping
- include relevant accelerator events such as kernels and device memory operations
- support strict overlap detection so nested same-stream execution intervals are reported
- report invalid overlaps through the verification framework with error severity
- leave original trace events unchanged
- include useful overlap details in the verification report
- handle events without stream information using the agreed default-stream behavior
- use the Trace Analyzer event/dialect infrastructure when determining which accelerator events participate in overlap verification
For Torch traces, the existing verifier groups accelerator events by process and stream. This allows kernel-kernel, kernel-memory, and memory-memory overlap behavior to be handled by the same verification path.
No additional memory-specific overlap pipeline is required once the profiler provides the correct stream information.
Event Filtering
The overlap verification event filter needs to distinguish actual accelerator execution events from other complete profiler events.
Torch-Spyre profiler traces can contain complete events from categories such as:
kernelgpu_memcpygpu_memsetcpu_opuser_annotationprivateuse1_runtimeprivateuse1_driverTrace
Not every complete event represents accelerator execution.
For Torch traces, overlap verification should include the relevant accelerator execution events, including:
kernel events for compute executiongpu_memcpy events for device memory transfersgpu_memset events for device memory operations
Runtime, driver, annotation, or CPU events should not automatically participate in device overlap verification simply because they are represented as complete (ph == "X") events.
The implementation should continue using the analyzer's dialect/event classification infrastructure rather than relying on broad category assumptions where possible.
Validation
The analyzer verification should cover at least:
- Same-stream non-overlapping kernels → PASS
- Same-stream kernel-kernel overlap → FAIL
- Cross-stream kernel-kernel overlap → PASS
- Same-stream kernel-memory overlap → FAIL
- Cross-stream kernel-memory overlap → PASS
- Same-stream memory-memory overlap → FAIL
- Cross-stream memory-memory overlap → PASS
- Back-to-back same-stream memory events → PASS
- Nested same-stream execution intervals → FAIL
- Missing stream ID → use stream
0
- Temporary default stream information should not modify the original event
The Trace Analyzer event filter should also be validated against representative Torch-Spyre profiler events so unrelated runtime, CPU, driver, annotation, or profiler events are not incorrectly treated as accelerator execution intervals.
Validation should include synthetic traces and representative real Spyre profiler traces where appropriate.
The verification report should provide:
- correct PASS/FAIL results
- correct overlap counts
- useful event instances/details
- correct stream information
- unchanged original trace events
Validation Completed
PR #126 adds regression coverage for the existing stream-aware overlap verifier, including:
- same-stream kernel-kernel overlap
- cross-stream kernel-kernel overlap
- same-stream kernel-memory overlap
- cross-stream kernel-memory overlap
- same-stream memory-memory overlap
- cross-stream memory-memory overlap
- nested same-stream memory overlap
- back-to-back same-stream memory events
- missing-stream default behavior
- cleanup of temporarily added default stream information
The tests confirmed that the verifier introduced in #124 already handles the required device overlap semantics through stream-aware accelerator event grouping.
PR #126 has been merged into main.
Representative real Spyre profiler traces have also been processed through the verification path as part of the corresponding Torch-Spyre integration work.
Implementation Status
Merged PR #124 provides the stream-aware accelerator overlap verification stage and verification reporting.
During follow-up validation, it was confirmed that the existing verifier can handle kernel and memory overlap semantics without additional memory super-event processing, provided events carry the correct stream information.
Merged PR #126 adds regression coverage for the required stream-aware device overlap behavior.
Correct memory stream IDs from Torch-Spyre are tracked separately in torch-spyre/torch-spyre#3699.
The corresponding Torch-Spyre integration is tracked separately in torch-spyre/torch-spyre#3541.
Related Work
Current Status
PR #124 has been merged and provides the stream-aware accelerator overlap verification stage.
PR #126 has also been merged and provides regression coverage for the required same-stream and cross-stream kernel/memory overlap behavior.
The follow-up investigation confirmed that additional memory super-event handling is not required. The existing verifier already provides the required stream-aware overlap behavior for accelerator events.
For memory events, correct stream attribution depends on Torch-Spyre emitting the appropriate stream IDs. That work is tracked separately in torch-spyre/torch-spyre#3699. Once those stream IDs are available, the existing analyzer verifier can consume them directly without additional memory-specific overlap-verification changes.
The corresponding Torch-Spyre integration is tracked in torch-spyre/torch-spyre#3541.
Description
Add stream-aware device overlap validation to AIU Trace Analyzer / Acelyzer verification mode.
The existing overlap infrastructure detects and resolves overlaps primarily for trace processing and visualization. This work extends that infrastructure so relevant accelerator execution overlaps can be reported through the verification framework as PASS/FAIL results without modifying the original trace events.
PR #124 (
Kernel overlap verification stage, initial version) has been merged and provides the stream-aware accelerator overlap verification stage.Follow-up validation confirmed that the existing verifier can handle kernel and memory overlap semantics through stream-aware accelerator event grouping without requiring a separate memory-specific overlap algorithm or memory super-events.
PR #126 (
Add stream-aware device overlap verification tests) has also been merged and adds regression coverage for the required same-stream and cross-stream device overlap behavior.The corresponding Torch-Spyre integration work is tracked separately in torch-spyre/torch-spyre#3541.
Expected Behavior
Same stream
Different streams
Events executing on different accelerator streams should not be incorrectly reported simply because their timestamps overlap globally.
Missing stream information
Events without
args.streamare currently attributed to stream0.Memory events do not yet always carry the correct logical accelerator stream information from Torch-Spyre. Once torch-spyre/torch-spyre#3699 enables the correct stream IDs for memory events, the analyzer can use the emitted stream ID through the existing stream-aware grouping without requiring additional overlap-verification logic.
The analyzer should not rely on thread IDs such as
400as the logical accelerator stream.Device Event Handling
Relevant accelerator execution events participate in the existing stream-aware overlap verification.
For Torch traces, accelerator events are grouped by process and
args.stream. This allows the existing overlap verifier introduced in #124 to validate kernel and memory events using the same stream-aware overlap logic.This means separate memory super-events, memory coalescing, or a separate memory-memory overlap detector are not required.
Once memory events contain the correct stream IDs, kernel-kernel, kernel-memory, and memory-memory overlaps can all be evaluated through the existing verifier.
Original trace events should remain unchanged by verification. When stream information is missing, the agreed default-stream behavior is applied for verification and the temporary default should not remain on the original event afterward.
Implementation
The verification implementation should:
OverlapDetectionContextinfrastructure rather than introduce a separate general overlap algorithmFor Torch traces, the existing verifier groups accelerator events by process and stream. This allows kernel-kernel, kernel-memory, and memory-memory overlap behavior to be handled by the same verification path.
No additional memory-specific overlap pipeline is required once the profiler provides the correct stream information.
Event Filtering
The overlap verification event filter needs to distinguish actual accelerator execution events from other complete profiler events.
Torch-Spyre profiler traces can contain complete events from categories such as:
kernelgpu_memcpygpu_memsetcpu_opuser_annotationprivateuse1_runtimeprivateuse1_driverTraceNot every complete event represents accelerator execution.
For Torch traces, overlap verification should include the relevant accelerator execution events, including:
kernelevents for compute executiongpu_memcpyevents for device memory transfersgpu_memsetevents for device memory operationsRuntime, driver, annotation, or CPU events should not automatically participate in device overlap verification simply because they are represented as complete (
ph == "X") events.The implementation should continue using the analyzer's dialect/event classification infrastructure rather than relying on broad category assumptions where possible.
Validation
The analyzer verification should cover at least:
0The Trace Analyzer event filter should also be validated against representative Torch-Spyre profiler events so unrelated runtime, CPU, driver, annotation, or profiler events are not incorrectly treated as accelerator execution intervals.
Validation should include synthetic traces and representative real Spyre profiler traces where appropriate.
The verification report should provide:
Validation Completed
PR #126 adds regression coverage for the existing stream-aware overlap verifier, including:
The tests confirmed that the verifier introduced in #124 already handles the required device overlap semantics through stream-aware accelerator event grouping.
PR #126 has been merged into
main.Representative real Spyre profiler traces have also been processed through the verification path as part of the corresponding Torch-Spyre integration work.
Implementation Status
Merged PR #124 provides the stream-aware accelerator overlap verification stage and verification reporting.
During follow-up validation, it was confirmed that the existing verifier can handle kernel and memory overlap semantics without additional memory super-event processing, provided events carry the correct stream information.
Merged PR #126 adds regression coverage for the required stream-aware device overlap behavior.
Correct memory stream IDs from Torch-Spyre are tracked separately in torch-spyre/torch-spyre#3699.
The corresponding Torch-Spyre integration is tracked separately in torch-spyre/torch-spyre#3541.
Related Work
devicePropertiesduring ingestion #125Current Status
PR #124 has been merged and provides the stream-aware accelerator overlap verification stage.
PR #126 has also been merged and provides regression coverage for the required same-stream and cross-stream kernel/memory overlap behavior.
The follow-up investigation confirmed that additional memory super-event handling is not required. The existing verifier already provides the required stream-aware overlap behavior for accelerator events.
For memory events, correct stream attribution depends on Torch-Spyre emitting the appropriate stream IDs. That work is tracked separately in torch-spyre/torch-spyre#3699. Once those stream IDs are available, the existing analyzer verifier can consume them directly without additional memory-specific overlap-verification changes.
The corresponding Torch-Spyre integration is tracked in torch-spyre/torch-spyre#3541.