Skip to content

pipeline: make pipeline_barrier context-aware, give each stage its own barrier - #101

Closed
aryanputta wants to merge 1 commit into
IBM:mainfrom
aryanputta:pipeline-context-aware-barriers
Closed

aryanputta wants to merge 1 commit into
IBM:mainfrom
aryanputta:pipeline-context-aware-barriers

Conversation

@aryanputta

Copy link
Copy Markdown
Contributor

What this does

First concrete step toward the pipeline restructuring tracked in #100.

pipeline_barrier ignored its context parameter and always wrote events to the module-level _main_barrier_context singleton. All four barrier registrations in register_processing_functions shared one queue. That made independent per-stage barriers impossible and made the context parameter misleading — every call site passed event_pipe._main_barrier_context but the function never used it.

Root cause

# before
def pipeline_barrier(event: TraceEvent, _: AbstractContext) -> list[TraceEvent]:
    bctx = _main_barrier_context   # always the global, ignoring the argument
    bctx.collect(event)
    return []

Changes

src/aiu_trace_analyzer/pipeline/barrier.py

  • _BarrierContext renamed to BarrierContext (public) so call sites can create instances without importing a private name.
  • pipeline_barrier now uses ctx.collect(event) — the context it is passed — instead of the hardwired global.
  • _main_barrier_context is kept as a BarrierContext() singleton for backwards compatibility; any call site that still passes it gets the same shared-barrier semantics as before.

src/aiu_trace_analyzer/pipeline/__init__.py

  • BarrierContext exported so acelyzer.py and future PipelineStage code can create instances via the normal event_pipe namespace.

src/aiu_trace_analyzer/core/acelyzer.py

  • Each of the four logical barrier groups in register_processing_functions now owns a named BarrierContext instance:
Name Logical stage
normalize_barrier_ctx frequency_align / normalize phase
overlap_barrier_ctx overlap_tid detection (conditional)
comm_barrier_ctx comm_summarize / rcu_util
categorize_barrier_ctx event categorizer

No change to registration order or conditional logic.

tests/aiu_trace_analyzer/pipeline/test_barrier.py

  • Updated existing test to pass a BarrierContext instance (previously passed None, which only worked because the function ignored its argument).
  • test_independent_barrier_contexts_do_not_share_events: two barriers with separate contexts hold and drain their own events independently.
  • test_barrier_drain_clears_hold: drain empties the hold list.

Validation

PYTHONPATH=src python3 -m pytest
# 157 passed, 5 xfailed

What this unblocks

With barriers context-aware and BarrierContext public, a PipelineStage class (next step in #100) can own its barrier instance and call pipeline_barrier correctly without any global state. The _main_barrier_context singleton is still available for the shared-barrier case but is no longer required.

…n barrier

Closes one of the blockers identified in IBM#100.

Root cause: pipeline_barrier ignored its context parameter and always
wrote events to the module-level _main_barrier_context singleton. All four
barrier registrations in register_processing_functions shared one queue,
making it impossible to scope a barrier to a specific logical stage group
and making the context parameter misleading.

What changed:

barrier.py
- Rename _BarrierContext to BarrierContext (public) so call sites can
  create independent instances without importing a private name.
- pipeline_barrier now uses the context it is passed (ctx.collect(event))
  instead of the global singleton. _main_barrier_context is kept as a
  convenience singleton for any call site that wants shared-barrier
  semantics, but it is no longer hardwired into the function.

pipeline/__init__.py
- Export BarrierContext alongside _main_barrier_context so acelyzer.py
  and future PipelineStage code can create instances via the normal
  event_pipe namespace.

acelyzer.py
- Each of the four logical barrier groups in register_processing_functions
  now owns a named BarrierContext instance:
    normalize_barrier_ctx  -- frequency_align stage
    overlap_barrier_ctx    -- overlap_tid detection (conditional)
    comm_barrier_ctx       -- comm_summarize / rcu_util stage
    categorize_barrier_ctx -- event categorizer stage
- No change to registration order or conditional logic; the only
  difference is which context each barrier call receives.

tests/aiu_trace_analyzer/pipeline/test_barrier.py
- Update existing test to pass a BarrierContext instance explicitly
  (previously passed None, which only worked because the function ignored
  its argument).
- Add test_independent_barrier_contexts_do_not_share_events: two barriers
  with separate contexts hold and drain their events independently.
- Add test_barrier_drain_clears_hold: drain returns events and leaves the
  context empty.

Signed-off-by: Aryan Putta <aryansputta@gmail.com>
@aryanputta

Copy link
Copy Markdown
Contributor Author

Step 2 building on this is in #102 (PipelineStage abstraction). Both are ready for review when you have a chance, @lasch.

@aryanputta

Copy link
Copy Markdown
Contributor Author

Superseded by #102 which combines both steps into one PR.

@aryanputta aryanputta closed this Apr 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant