You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As someone reading a latency estimate for a kernel that fills, broadcasts or
converts a tile,
I want those operations either priced or documented as free with a reason,
so that the reported cost is not quietly lower than the hardware's for a reason
nobody decided.
Context / Background
@register() defaults latency_category to "zero", so "this operation is free" and
"nobody priced this operation" are the same state in the registry. #209 split that
state in two: ktir_cpu/kernelentry/pricing.py requires every zero-priced op to be
listed either in ZERO_COST_OPS, with the reason it costs nothing, or in UNJUDGED_ZERO_OPS, with what is unresolved about it. Twenty-five ops are currently
in the second list, and the gate accepts that — it only fails on an op in neither.
Three of those twenty-five are worth separating from the rest, because they do work
proportional to a whole tile:
op
what it does
charged
linalg.fill
writes a scalar across the outs tile
nothing
linalg.broadcast
expands a tile along new dimensions
nothing
arith.sitofp
integer to float across the tile, via _unary
nothing
None of the three carries a docstring reason; all are bare @register(...) and take
the default. All three appear in kernels already on main:
linalg.fill — nine of them, including examples/sdsc/sdpa_pv_ksplit.mlir, examples/ktir/ffn_swiglu_4core.mlir, examples/ktir/ring_reduce.mlir and examples/latency/ring_reduce_multi_group.mlir
arith.sitofp is tile-wide by way of _unary in ktir_cpu/dialects/_helpers.py,
which applies its function to the whole Tile when given one.
Whether Spyre does measurable work for them cannot be settled from RFC 0682. Each has
a plausible free reading and a plausible non-free one — linalg.broadcast is free if
the consumer reads it strided and not if it is materialised. That is a hardware
question, which is why this is a decision to be made rather than a fix to be applied.
Two neighbours are deliberately not in scope, and both look like they should be:
arith.index_cast / arith.index_castui are in UNJUDGED_ZERO_OPS too, and
zero is defensible for them: the handler returns a Python int, so it is one scalar
conversion rather than a tile's worth.
linalg.reduce is priced zero on purpose and is correct. It executes its
combiner region, and the ops inside it are charged individually. Measured on the
RMSNorm generator, one core's trace carries 256 arith.addf entries totalling
1,280 cycles from inside a reduce. The orchestrator is free; the arithmetic is not. linalg.generic runs its region the same way but is registered COMPUTE_FLOAT, so
it is not a zero-priced op and is not in this audit's population at all.
One limit of this check, recorded here so it is not lost: a zero-versus-nonzero audit
cannot detect a wrong category. arith.cmpi was priced COMPUTE_FLOAT while being an
integer compare, and no amount of auditing against zero would have found it; #207
corrected that line.
Related: #209 (the audit that surfaced this), #88 (ktdp.coreid — a different
category of entry in the same list: a non-spec op to remove rather than to price).
Acceptance Criteria
Given linalg.fill, when its cost is looked up, then it is either a non-zero latency_category or an entry in ZERO_COST_OPS stating why the hardware does
no measurable work for it.
Same for linalg.broadcast.
Same for arith.sitofp.
Given a decision that any of the three is not free, when a kernel using it is
run, then its reported cycles change — and the change is visible as a diff in
the committed derivations of the affected kernels, not only in a test.
ktir_cpu/kernelentry/pricing.py's UNJUDGED_ZERO_OPS shrinks by three, and audit() still returns no findings.
Tests added/updated and uv run pytest tests/ -v passes.
Out of Scope
The other twenty-two entries in UNJUDGED_ZERO_OPS. Shape metadata
(tensor.reshape, tensor.collapse_shape, …) and the remaining casts are the same
kind of question but a cheaper one to be wrong about; they can follow the precedent
this story sets.
User Story
As someone reading a latency estimate for a kernel that fills, broadcasts or
converts a tile,
I want those operations either priced or documented as free with a reason,
so that the reported cost is not quietly lower than the hardware's for a reason
nobody decided.
Context / Background
@register()defaultslatency_categoryto"zero", so "this operation is free" and"nobody priced this operation" are the same state in the registry. #209 split that
state in two:
ktir_cpu/kernelentry/pricing.pyrequires every zero-priced op to belisted either in
ZERO_COST_OPS, with the reason it costs nothing, or inUNJUDGED_ZERO_OPS, with what is unresolved about it. Twenty-five ops are currentlyin the second list, and the gate accepts that — it only fails on an op in neither.
Three of those twenty-five are worth separating from the rest, because they do work
proportional to a whole tile:
linalg.filloutstilelinalg.broadcastarith.sitofp_unaryNone of the three carries a docstring reason; all are bare
@register(...)and takethe default. All three appear in kernels already on
main:linalg.fill— nine of them, includingexamples/sdsc/sdpa_pv_ksplit.mlir,examples/ktir/ffn_swiglu_4core.mlir,examples/ktir/ring_reduce.mlirandexamples/latency/ring_reduce_multi_group.mlirlinalg.broadcast—examples/ktir/scalar_broadcast.mlir,examples/triton-ktir/sdpa_2d.mlirarith.sitofp—examples/latency/rmsnorm_4core_2x2.mlir,examples/latency/rmsnorm_4core_4x1.mlir,examples/triton-ktir/layernorm_fwd_ktir.mlirarith.sitofpis tile-wide by way of_unaryinktir_cpu/dialects/_helpers.py,which applies its function to the whole
Tilewhen given one.Whether Spyre does measurable work for them cannot be settled from RFC 0682. Each has
a plausible free reading and a plausible non-free one —
linalg.broadcastis free ifthe consumer reads it strided and not if it is materialised. That is a hardware
question, which is why this is a decision to be made rather than a fix to be applied.
Two neighbours are deliberately not in scope, and both look like they should be:
arith.index_cast/arith.index_castuiare inUNJUDGED_ZERO_OPStoo, andzero is defensible for them: the handler returns a Python
int, so it is one scalarconversion rather than a tile's worth.
linalg.reduceis pricedzeroon purpose and is correct. It executes itscombiner region, and the ops inside it are charged individually. Measured on the
RMSNorm generator, one core's trace carries 256
arith.addfentries totalling1,280 cycles from inside a reduce. The orchestrator is free; the arithmetic is not.
linalg.genericruns its region the same way but is registeredCOMPUTE_FLOAT, soit is not a zero-priced op and is not in this audit's population at all.
One limit of this check, recorded here so it is not lost: a zero-versus-nonzero audit
cannot detect a wrong category.
arith.cmpiwas pricedCOMPUTE_FLOATwhile being aninteger compare, and no amount of auditing against
zerowould have found it; #207corrected that line.
Related: #209 (the audit that surfaced this), #88 (
ktdp.coreid— a differentcategory of entry in the same list: a non-spec op to remove rather than to price).
Acceptance Criteria
linalg.fill, when its cost is looked up, then it is either a non-zerolatency_categoryor an entry inZERO_COST_OPSstating why the hardware doesno measurable work for it.
linalg.broadcast.arith.sitofp.run, then its reported cycles change — and the change is visible as a diff in
the committed derivations of the affected kernels, not only in a test.
ktir_cpu/kernelentry/pricing.py'sUNJUDGED_ZERO_OPSshrinks by three, andaudit()still returns no findings.uv run pytest tests/ -vpasses.Out of Scope
UNJUDGED_ZERO_OPS. Shape metadata(
tensor.reshape,tensor.collapse_shape, …) and the remaining casts are the samekind of question but a cheaper one to be wrong about; they can follow the precedent
this story sets.
arith.cmpi's wrong category — a different defect, already addressed in Add multi-core decode SDPA (P@V) to the latency demo notebook #207.linalg.reduce— free by design, measured, not a gap.linalg.genericis registeredCOMPUTE_FLOATand is not a zero-priced op at all.@register()defaults. Making the default explicit would be a muchwider change and would touch every dialect module; the audit in [Feature] Make "the simulator supports this kernel" a checkable statement #209 exists so that
the default's consequences are visible without it.