Skip to content

[Story] Decide whether three tile-wide ops priced zero are really free #211

Description

@WarningRan

User Story

As someone reading a latency estimate for a kernel that fills, broadcasts or
converts a tile
,
I want those operations either priced or documented as free with a reason,
so that the reported cost is not quietly lower than the hardware's for a reason
nobody decided
.

Context / Background

@register() defaults latency_category to "zero", so "this operation is free" and
"nobody priced this operation" are the same state in the registry. #209 split that
state in two: ktir_cpu/kernelentry/pricing.py requires every zero-priced op to be
listed either in ZERO_COST_OPS, with the reason it costs nothing, or in
UNJUDGED_ZERO_OPS, with what is unresolved about it. Twenty-five ops are currently
in the second list, and the gate accepts that — it only fails on an op in neither.

Three of those twenty-five are worth separating from the rest, because they do work
proportional to a whole tile:

op what it does charged
linalg.fill writes a scalar across the outs tile nothing
linalg.broadcast expands a tile along new dimensions nothing
arith.sitofp integer to float across the tile, via _unary nothing

None of the three carries a docstring reason; all are bare @register(...) and take
the default. All three appear in kernels already on main:

  • linalg.fill — nine of them, including examples/sdsc/sdpa_pv_ksplit.mlir,
    examples/ktir/ffn_swiglu_4core.mlir, examples/ktir/ring_reduce.mlir and
    examples/latency/ring_reduce_multi_group.mlir
  • linalg.broadcastexamples/ktir/scalar_broadcast.mlir,
    examples/triton-ktir/sdpa_2d.mlir
  • arith.sitofpexamples/latency/rmsnorm_4core_2x2.mlir,
    examples/latency/rmsnorm_4core_4x1.mlir,
    examples/triton-ktir/layernorm_fwd_ktir.mlir

arith.sitofp is tile-wide by way of _unary in ktir_cpu/dialects/_helpers.py,
which applies its function to the whole Tile when given one.

Whether Spyre does measurable work for them cannot be settled from RFC 0682. Each has
a plausible free reading and a plausible non-free one — linalg.broadcast is free if
the consumer reads it strided and not if it is materialised. That is a hardware
question, which is why this is a decision to be made rather than a fix to be applied.

Two neighbours are deliberately not in scope, and both look like they should be:

  • arith.index_cast / arith.index_castui are in UNJUDGED_ZERO_OPS too, and
    zero is defensible for them: the handler returns a Python int, so it is one scalar
    conversion rather than a tile's worth.
  • linalg.reduce is priced zero on purpose and is correct. It executes its
    combiner region, and the ops inside it are charged individually. Measured on the
    RMSNorm generator, one core's trace carries 256 arith.addf entries totalling
    1,280 cycles from inside a reduce. The orchestrator is free; the arithmetic is not.
    linalg.generic runs its region the same way but is registered COMPUTE_FLOAT, so
    it is not a zero-priced op and is not in this audit's population at all.

One limit of this check, recorded here so it is not lost: a zero-versus-nonzero audit
cannot detect a wrong category. arith.cmpi was priced COMPUTE_FLOAT while being an
integer compare, and no amount of auditing against zero would have found it; #207
corrected that line.

Related: #209 (the audit that surfaced this), #88 (ktdp.coreid — a different
category of entry in the same list: a non-spec op to remove rather than to price).

Acceptance Criteria

  • Given linalg.fill, when its cost is looked up, then it is either a non-zero
    latency_category or an entry in ZERO_COST_OPS stating why the hardware does
    no measurable work for it.
  • Same for linalg.broadcast.
  • Same for arith.sitofp.
  • Given a decision that any of the three is not free, when a kernel using it is
    run, then its reported cycles change — and the change is visible as a diff in
    the committed derivations of the affected kernels, not only in a test.
  • ktir_cpu/kernelentry/pricing.py's UNJUDGED_ZERO_OPS shrinks by three, and
    audit() still returns no findings.
  • Tests added/updated and uv run pytest tests/ -v passes.

Out of Scope

  • The other twenty-two entries in UNJUDGED_ZERO_OPS. Shape metadata
    (tensor.reshape, tensor.collapse_shape, …) and the remaining casts are the same
    kind of question but a cheaper one to be wrong about; they can follow the precedent
    this story sets.
  • arith.cmpi's wrong category — a different defect, already addressed in Add multi-core decode SDPA (P@V) to the latency demo notebook #207.
  • linalg.reduce — free by design, measured, not a gap. linalg.generic is registered
    COMPUTE_FLOAT and is not a zero-priced op at all.
  • Changing how @register() defaults. Making the default explicit would be a much
    wider change and would touch every dialect module; the audit in [Feature] Make "the simulator supports this kernel" a checkable statement #209 exists so that
    the default's consequences are visible without it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions