docs(inter-tile): add gather and scatter as first-class ops

[tnakaike]

This PR adds ktdp.inter_tile_gather and ktdp.inter_tile_scatter as delivery ops in docs/inter-tile-communication.md, bringing the specification to six ops covering all five inter-tile communication patterns: broadcast, all-reduce, reduce-scatter, gather, and scatter.

Closes #52.

What this PR does

Two commits, one per op:

• ktdp.inter_tile_gather (§6) — assembles the producers' partials into one tensor by ordered concatenation along gather_dimension (no fold). General consumer set (one tile per group is a plain gather, all tiles is an all-gather), optional producer_dependency_per_consumer for a partial / segmented gather over a declared producer subset, and a full-IR multi-group example (§10.5.1, 128×8×12×64).
• ktdp.inter_tile_scatter (§7) — splits a single producer tile's tensor into ordered slices, one per consumer, along scatter_dimension (the dual of gather). Single producer tile per group, no producer_dependency_per_consumer (with one producer the full-barrier and per-tile modes collapse), and a full-IR multi-group example (§10.6.1, 128×8×64).

Both ops carry no combiner region and no identity operand: unlike reduce / reduce_scatter they place data by position rather than folding it, so once the axis is known the placement is deterministic. On this point they align with consume.

The rest of each op — the !ktdp.tile_future<..., #groups> operand, producer_tiles_per_group / consumer_tiles_per_group, groups carried in the future type, and the def-use synchronization model — reuses the existing delivery-op scaffolding unchanged.

Supporting edits

• Scope / motivation / coverage tables and count-prose updated (four ops → six, three patterns → five).
• Relationship-to-existing-ops note extended (gather and scatter have no pre-existing counterparts).
• Resolved open questions folded out of the open-questions list (the gather / ordered-concatenation question, and the consumer-need-not-be-producer question for scatter).
• Sections after each new op renumbered, with internal cross-references fixed. The final layout is §1 Motivation → §2 produce → §3–§7 the five delivery ops → §8 Synchronization → §9 Coverage → §10 Pattern instantiation → §11 Relationship → §12 Open questions → §13 Extensions.

Scope

Documentation only — this PR updates the specification. No dialect / verifier / lowering implementation is included.

[fabianlim]

Proposal: restructure inter-tile-communication.md

cc: @tnakaike @mudhakar @moriohara I had a discussion with @AdnanHoque and reviewed the PRs on torch-spyre on LX-reshuffle, and drafted a discussion for the new delivery ops. My suggestion is to restructure this document as it is becoming very long and can be better refocused with the use-cases

Status: discussion notes. Nothing here is implemented.

Problem. The document grew by accretion — gather (6ae0ced) and scatter
(ff61eeb) were each appended as self-contained sections on top of the
original three ops. Each addition restated the shared machinery with a local
amendment instead of extending a common definition. The result is five
near-duplicate rule blocks whose deltas are where the inconsistencies live,
and a reader has to hold five separate rule sets rather than one grid.

Concretely, producer_dependency_per_consumer is specified in §3.1, then
"identical in form to §3.1" in §4.1, §5.1, and §6.1 (each with an amendment),
declared absent in §7.1, and then restated a third time in §8. The
subset/coverage invariants appear in both §3.1 and §8. That three-way
duplication is what let §9's coverage table drift out of sync with §6.1.

This proposal reorganizes the same content along its actual axes, folds in
all_to_all as a first-class op (promoted before scatter), and collects the
verification rules into one owned list.

---

1. Semantics matrix

The six delivery ops differ along exactly three independent axes:

• combine — none | fold (combiner region + identity operand)
• placement — how producer contributions map onto consumer results
• cardinality — producers per group × consumers per group

Op	combine	placement	producers/grp	consumers/grp	dim attrs	region	identity
consume	none	replicate	1	free	—	—	—
reduce	fold	replicate	all	free	—	combiner	yes
reduce_scatter	fold	split	all	free	scatter_dimension	combiner	yes
gather	none	concat	all	free	gather_dimension	—	—
all_to_all	none	permute	all	all	split_dimension, concat_dimension	—	—
scatter	none	split	1	free	scatter_dimension	—	—

all_to_all is promoted before scatter because it shares the all-producers
cardinality cell with gather and reduce_scatter; its relationship to
scatter is structural (permute = split + concat in one step) and should be
visible in the table rather than deferred to a section.

Three things this matrix makes visible:

• placement takes only four values. The per-op "type rules" subsections are
  four formulas wearing six hats.
• The empty cells are principled. none × replicate with all producers is
  undefined (which producer's value wins?), and fold × concat / fold ×
  permute is meaningless (folding then shuffling what?).
• all_to_all is the fourth placement value, not a special case: gather + scatter in one step.

1.1 Type rules, one per placement value

placement	result type derived from T_p
replicate	tile axes collapsed (fold) / unchanged (none)
concat	extent along gather_dimension multiplied by K
permute	split_dimension extent divided by M; concat_dimension extent multiplied by K
split	extent along scatter_dimension divided by M

For the square all_to_all case (M == K == |group|) the permute result type
equals T_p — total tensor size is conserved. This is the uniform one-to-one
shuffle that the SDSC backend emits today.

1.2 Normative definition of within-group local index

l is the rank of a tile, by ascending tile id, among the relevant set
(producers for concat and permute's concat axis; consumers for split and
permute's split axis) within its group.

Without this definition, concatenation and split orders are pinned down only by
contiguous-tile-id coincidence and silently break under non-monotone tile
assignments (see §8.4 of the backend findings).

For all_to_all specifically: producer rank determines which split_dimension
slice each producer contributes; consumer rank determines which
concat_dimension position each consumer receives.

---

2. Verification matrix

Principle: each rule has exactly one owner and one statement; applicability
is a column, not a restatement.

Rule	Owner	consume	reduce	red_scat	gather	all_to_all	scatter
R1 group disjointness	produce	y	y	y	y	y	y
R2 single-use future	produce	y	y	y	y	y	y
R3 dep set subset of producers	delivery	y	y	y	y	y	n/a
R4 every producer covered by some consumer	delivery	y	y	y	y	y	n/a
R5 dep sets pairwise disjoint	delivery	—	—	—	y	y	n/a
R6 uniform dep-set cardinality	delivery	—	—	—	y	y	n/a
R7 uniform producer cardinality across groups	delivery	—	—	—	y	y	n/a
R8 producers per group = 1	delivery	y	—	—	—	—	y
R9 scatter_dimension divisible by consumer count	delivery	—	—	y	—	—	y
R9b split_dimension divisible by group size	delivery	—	—	—	—	y	—
R10 combiner purity	delivery	—	y	y	—	—	—
R11 identity shape matches T_p	delivery	—	y	y	—	—	—
R12 concat_dimension extent × K well-defined	delivery	—	—	—	y	y	—

New rules introduced by all_to_all:

• R9b — split_dimension extent must be divisible by M (the group size).
  Analogous to R9 for reduce_scatter/scatter. For the square case M == K
  so R9b and R12 reduce to the same divisibility.
• R12 — The result extent along concat_dimension is K × T_p[concat_dim].
  This requires that all producers contribute the same post-split size (follows
  from R7 + R9b for the square case, but should be stated independently for the
  non-square case).

Rules implemented in KTIRCheckLegality.cpp:

Rule	Op	Check	Location
R2 (single-use future)	inter_tile_produce	future.hasOneUse()	KTIRCheckLegality.cpp:80–85
R3 (dep set ⊆ producers)	inter_tile_reduce	C⊆P check per group	KTIRCheckLegality.cpp:107–117
R4 (every producer covered)	inter_tile_reduce	coverage check per group	KTIRCheckLegality.cpp:163–174
R3 on dep attr	inter_tile_reduce	dep tile ∈ P(g)	KTIRCheckLegality.cpp:151–160
R4 on dep attr	inter_tile_reduce	every p covered by dep	KTIRCheckLegality.cpp:163–174

Not yet in KTIRCheckLegality.cpp — R1, R5, R6, R7, R8, R9, R9b, R10,
R11, R12, and R3/R4 for gather, scatter, consume, reduce_scatter, all_to_all.
The current legality pass only walks InterTileProduceOp and InterTileReduceOp.

R7 and R5 are enforced in the Torch-Spyre SDSC planner (_compatible_partitions)
but absent from the KTIR verifier entirely — the gap exists at both the spec and
implementation level.

2.1 Rules that do not exist yet (the actual defects)

R7 — uniform producer cardinality across groups. producer_tiles_per_group
is a parameterized affine set over g, and nothing requires equal cardinality
per group. The op result is a single static tensor type, so unequal groups yield
no expressible result type. Applies to both gather and all_to_all.

R5 — dependency sets pairwise disjoint. §6.1's coverage check requires only
that each producer be claimed by at least one consumer — overlap is permitted.
Uniform cardinality combined with at-least-one coverage admits declared sets
that double-count producers. Either add R5, or retract the uniform-result claim.

R9b — stated above, not yet in the document.

2.2 An asymmetry the matrix forces into the open

R8 is verifier-enforced for scatter but merely conventional for consume.
Whether that difference is intentional should be decided rather than inherited.

---

3. All-gather and all-to-all

3.1 All-gather is gather with wider consumer set

All-gather is ktdp.inter_tile_gather with
consumer_tiles_per_group = <all tiles in group>. It is not a new op — the
existing attribute already accepts this value and §6.4 already states the case.
What all-gather does need (inherited from gather):

• R7 (uniform producer cardinality, §2.1). Without it K in the result type
  is undefined when groups differ.
• §1.2 (normative local index). Without it concatenation order depends on
  contiguous-tile-id coincidence.

3.2 All-to-all is not decomposable into existing ops

All-to-all requires every tile to be simultaneously a producer of M distinct
slices and a consumer of K distinct slices:

tile 0: A[0][0..3]     tile 0: A[0][0] A[1][0] A[2][0] A[3][0]
tile 1: A[1][0..3]     tile 1: A[0][1] A[1][1] A[2][1] A[3][1]
tile 2: A[2][0..3] --> tile 2: A[0][2] A[1][2] A[2][2] A[3][2]
tile 3: A[3][0..3]     tile 3: A[0][3] A[1][3] A[2][3] A[3][3]

Neither existing op admits this:

• gather delivers the same assembled tensor to every consumer (§6.4). It
  cannot give consumers different content.
• scatter permits exactly one producer per group. It cannot have every tile
  contribute.

Composing them materializes the full concatenation on every tile — wrong data
volume and wrong communication pattern.

3.3 all_to_all — op definition

One new op, ktdp.inter_tile_all_to_all, with placement permute.

Attributes: split_dimension, concat_dimension.

Type rule: T_p with split_dimension extent divided by M and
concat_dimension extent multiplied by K. For the square case
(M == K == |group|) the result type equals T_p.

Verification rules: reuses R3, R4, R5, R6, R7 unchanged from gather; adds
R9b (split_dimension divisible by M) and R12 (well-defined concat extent).

MLIR sketch:

// all-to-all — 4 tiles, each splits along dim 0 and collects along dim 1
%r = ktdp.inter_tile_all_to_all(%future)
    producer_tiles_per_group = <all tiles>,
    consumer_tiles_per_group = <all tiles>,
    split_dimension          = 0,
    concat_dimension         = 1

3.4 Relationship to the SDSC backend shuffle

The SDSC backend emits a single opfunc = "shuffle" for all relayout patterns.
The entire payload is two coreIdToWkSlice_ tables — one per tensor in
coordinates_ — describing the per-core ownership before and after the movement.
all_to_all with explicit split_dimension/concat_dimension attributes lowers
directly to this table pair.

The axis-transpose case ([4,8] → [8,4], same dimensions swapped) is the one
pattern the current KTIR dim-attribute design cannot express — the ownership
tables differ but no single split_dimension or concat_dimension captures the
transformation. An explicit source→destination affine map (Option B below) would
handle it; for now, mark it as out of scope for all_to_all and add a verifier
that rejects non-decomposable transpositions.

3.5 Option B — explicit coordinate map (recorded, not recommended now)

Replace the split_dimension/concat_dimension attribute pair with a single
source-to-destination affine map, subsuming all four placement values. This is
the shape that interface-specs PR 14 already uses (SHUFFLE as source/destination
coordinate sets). The backend already speaks per-core partitionings
(coreIdToWkSlice_ tables), not dim-attributes, so Option B is arguably closer
to the existing contract.

Recommendation: implement all_to_all with dim-attributes first (additive,
reviewable on its own), and record Option B as the long-term direction. If none
of the five delivery ops is built yet, the cost argument for Option A over B is
weaker than usual.

3.6 Fused relayout — deferred

Relayout is a separate preceding op; fusion is a lowering concern. The backend
structurally cannot fuse them: restickify is a separate pass (runs before LX
planning), and restickified weights are explicitly barred as shuffle sources.

---

4. All- naming convention

Ops whose consumers/grp is free already subsume the all-tiles case by
widening the consumer set. The "all-" prefix names the pattern, not a new op:

Pattern name	KTIR op	Consumer set	Notes
all-gather	inter_tile_gather	all tiles in group	existing op, wider consumer set
all-to-all	inter_tile_all_to_all	all tiles in group	new op, §3.3
all-reduce	inter_tile_reduce	all tiles in group	existing op
all-reduce-scatter	inter_tile_reduce_scatter	all tiles in group	existing op

inter_tile_scatter and inter_tile_consume do not have natural "all-" variants
under the current cardinality constraints (R8 limits them to 1 producer per group).

---

5. Proposed outline

1  Motivation + three-axis decomposition     ← matrix from §1 (six ops, four placements)
2  produce                                    (unchanged)
3  Shared delivery semantics                 ← NEW. consumer sets,
     consumer_tiles_per_group                   producer_dependency_per_consumer,
     producer_dependency_per_consumer           local-index definition (§1.2), and
     within-group local index                   sync model stated ONCE.
4  Placement algebra + type rules            ← four formulas (§1.1), not five
5  Verification rules R1-R12                 ← matrix from §2
6  The delivery ops                          ← per op: only its own cells;
                                                all_to_all placed before scatter
7  Pattern instantiation / examples          ← current §10, annotated per §1.2
8  Backend pattern catalogue                 ← new, §7 of this proposal
9  Open questions and extensions

---

6. Smaller corrections to fold in

• §9 coverage table contradicts §6.1. The table lists gather as "all tiles
  per group" for inter_tile_produce and omits the consumer-set axis. Add
  all_to_all row; show consumer set as a column since that distinguishes
  gather from all-gather and all_to_all from scatter.
• §1 line 51 cross-reference omits §7.1.
• §12 renumbering dropped a still-open question. Q1 (must a consumer also be
  a producer?) is resolved for scatter only; mark open for consume/reduce/reduce_scatter.
• §10.6.1 example style diverges. Align with other full-IR examples or note
  why it differs.
• all_to_all belongs in the §9 coverage table as a new row with
  placement = permute, producers/grp = all, consumers/grp = all.

---

7. Backend pattern catalogue (Torch-Spyre)

The following table records patterns observed in the Torch-Spyre backend
(scratchpad/lx_relayout.py) where all classification fields are known. Rows
with unknown split axes are omitted. Patterns are classified using the derivation
in this proposal: gathered_dims = src_syms − dst_syms,
scattered_dims = dst_syms − src_syms, factor = num_cores // prod(dst splits).

All implemented patterns emit a single SDSC opfunc = "shuffle" whose payload
is two coreIdToWkSlice_ tables on the source and destination tensors.

ID	SenDNN op	Src work_div	Dst view	gathered	scattered	factor	Proposed KTIR op	Backend status
P01	all-gather	{H:8, Lk:4}	replicated (all cores)	H, Lk	—	N/A	inter_tile_gather (all consumers)	missing — replication not supported
P03	grouped all-gather	{H:8, Lk:4}	{H:8}	Lk	—	4	inter_tile_gather	#3440 (open PR)
P04	grouped all-gather	{Lk:32}	{H:8}	Lk	H	4	inter_tile_gather	#3440 (open PR)
P06	all-to-all	{H:8, Lk:4}	{Lk:32}	H	—	1	inter_tile_all_to_all	main (#3439)
P08	all-to-all (axis transpose)	{A:4, B:8}	{A:8, B:4}	—	—	1	inter_tile_all_to_all + explicit coord map	missing — axis swap inexpressible
P14	all-to-all	{H:8, Lq:4}	{Lq:32}	H	—	1	inter_tile_all_to_all	main (#3439)

Notes:

• P03 / P04 demonstrate that gathered_dims ≠ ∅ with factor > 1 maps to
  inter_tile_gather. For P04, scattered_dims = {H} (H is introduced at the
  destination) — this is a combined gather+scatter in one shuffle step; the KTIR
  op must express both axes.
• P06 / P14 demonstrate inter_tile_all_to_all at factor=1: gathered_dims = {H},
  scattered_dims = ∅. H is contracted into the 1D destination axis.
• P08 is the case all_to_all with dim-attributes cannot express: both sides
  split the same two dims with swapped counts ([4,8] → [8,4]). The
  coreIdToWkSlice_ tables differ because the mixed-radix odometer ordering
  changes, but no single split_dimension / concat_dimension captures the
  transformation. This is the §8.4 miscompile class — making the coordinate map
  explicit in KTIR would catch it statically.
• P01 (replication / full all-gather) is blocked because
  _compatible_partitions requires distinct slices per destination core. Maps to
  inter_tile_gather with consumer_tiles_per_group = all, but requires new
  backend support for non-bijective shuffle.

Classification decision rule (derived from TensorArg.work_division on a
relayout identity OpSpec):

gathered_dims	scattered_dims	factor	slot_exprs_differ	KTIR op
∅	∅	1	false	no-op
∅	∅	1	true	inter_tile_all_to_all + explicit coord map
non-∅	∅ or non-∅	> 1	—	inter_tile_gather
non-∅	∅	1	—	inter_tile_all_to_all
∅	non-∅	1	—	inter_tile_scatter
∅	∅	1	— (replication)	inter_tile_gather (all consumers)

---

[tnakaike]

@fabianlim Thank you so much for your review. I reflected your comments.

[fabianlim]

@ani300 @AdnanHoque would appreciate if you guys can review the Section 7 in the comment above regarding the use cases in torch-spyre

[AdnanHoque]

The overall direction looks right: KTIR should describe the communication meaning, while torch-spyre can lower several of these forms to the same SDSC shuffle.

Some notes on Section 7:

1. Classification cannot use only src_dims - dst_dims. Split counts matter too. For example, P06 changes {mb:8, head:4} to {mb:32}. It gathers the four head shards and also splits mb four ways. This is a valid all-to-all, but the current rule misses the split because mb exists on both sides.

2. P08 keeps the same logical 4×8 partition and changes only which core owns each piece. It is not {A:4,B:8} → {A:8,B:4}. Since this is a one-to-one movement of complete pieces, it appears to fit the document’s existing consume plus bijective dependency case, rather than requiring all_to_all.

3. P14 first selects the last-token region and then redistributes it. It is not a full-tensor all-to-all. KTIR should represent the selection separately from the communication.

4. The status column should separate “the generic mechanism exists” from “this Granite boundary is actually emitted.” #3439 provides the base one-to-one relayout mechanism, but it does not by itself reproduce the Granite work divisions. #3440 adds repeated-destination grouped gathers. Exact P01 and P08 remain unrealized; P14 currently depends on prototype-specific policy.

I suggest classifying from:

• the split-count change on every dimension;
• the source and destination core-to-slice maps;
• the number of physical owners per logical destination;
• whether the transfer covers the full tensor or only a selected region.

With that adjustment, the proposed mappings are roughly: P01/P03/P04 → gather, P06 → all-to-all, P08 → one-to-one consume/permutation, and P14 → selection followed by permutation.

[fabianlim]

Thanks @AdnanHoque I did one more round to update Section 7 and came up now with a pretty detailed design. cc: @tnakaike

---

7. Backend pattern catalogue

Every relayout in the backend compiles to one SDSC entry with opfunc = "shuffle",
whose whole payload is a pair of per-core ownership tables — what each core owns
before the movement and after. Classifying a pattern means deciding, from those two
tables, which delivery op of §1 expresses the same movement.

Everything that classifies factorizes onto axes. A region is a multi-dimensional
product and hard to picture; an axis is one-dimensional. So the centre of this section
is one row per axis (§7.2) and a four-row decision read off it (§7.3). Per-axis and
region-level formulations were checked equivalent on 59,095 division pairs, and the
classification on 51 measured relayouts (§7.5).

7.1 The input

A table maps core_id → {axis: slice_index}. An axis cut into n slices has indices
0..n-1; an axis absent from an entry is uncut, so the core spans it whole. A core's
region is the intersection of its per-axis ranges.

Axis versus region. An axis is a direction that gets cut; a region is what cutting
produces. Cutting H two ways and Lk four ways gives eight regions:

                        Lk  — cut 4 ways →
                  |   0   |   1   |   2   |   3   |
            H   0 |  R0   |  R1   |  R2   |  R3   |
       cut 2     -+-------+-------+-------+-------+
          ways  1 |  R4   |  R5   |  R6   |  R7   |

{H:1, Lk:2} names R6. So the region count is the product of the per-axis slice
counts. The quantities that classify count regions; the ones that mislead count
cores, because several cores may hold one region and a region count need not
divide the core count.

Axes must be aligned by physical axis, not by label. Labels differ between the two
sides; in the backend PerCoreView is keyed by device-dimension index and
TensorWorkDivision.remap_symbols re-keys to loop symbols, so identical physical
slicings compare equal regardless of naming. This is the only place a wrong answer can
enter. Per §1.0, an axis named by a dimension attribute is at or outside the stick
level, never a lane axis.

Shorthand and its limits. A work division {axis: n} abbreviates the full table:
it gives the region count but not the core-to-region map, which is exactly what matters
when the two differ. A grid [A,B,C] is weaker still. Decoding it from the measured
relayouts: the slice count of each cut axis in layoutDimOrder_ order with uncut axes
dropped, then the replication factor (cores per region) if above 1, right-padded to
three — with cut counts coalesced by multiplication, not necessarily adjacently, when
more than three axes are cut. The invariant, exact on all 102 measured sides:

prod(grid) == number of cores actually holding data on that side

so prod(grid) is the active core count, not always 32. Three things a grid still
cannot say: which slot a lone replication factor occupies ([32,1,1] and [1,28,1]
are the same content differently spelled); whether an entry is one axis or several
multiplied; and which entry is coarsened — [8,4,1] → [32,1,1] admits both
assignments, and they give different group structures.

Precondition. Per-axis reasoning assumes each side's region set is the full product
of its cuts. An arbitrary table need not comply: diagonal-only source regions
{(0,0),(1,1)} against a full 2×2 destination give per-region fanin [1,0,0,1], where
reading it as {X:2, Y:2} sees four regions and fanin 1. For such a table, enumerate
regions instead.

7.2 The axis table

One row per axis, with Ns(a) and Nd(a) the source and destination slice counts
(1 if absent). The relation is coarsened when Ns(a) > Nd(a), refined when
Nd(a) > Ns(a), unchanged when equal. The two overlap counts are exact 1-D scans:

fanin(a)  = max(ceil((kd+1)*Ns(a)/Nd(a)) - floor(kd*Ns(a)/Nd(a)) for kd in range(Nd(a)))
fanout(a) = max(ceil((ks+1)*Nd(a)/Ns(a)) - floor(ks*Nd(a)/Ns(a)) for ks in range(Ns(a)))

max(1, Ns // Nd) is not the fanin — only its divisible-case shortcut.
Ns=6, Nd=4 gives per-slice [2,2,2,2] while 6 // 4 = 1; Ns=2, Nd=3 gives
[1,2,1], non-uniform and a refined axis carrying fanin 2. The shortcut disagrees
with region enumeration on 4223 of 14161 two-axis pairs.

Everything the decision needs is a product or conjunction down the rows:

max_fanin  = prod(fanin(a))        uniform_fanin  = all(ufi(a))
max_fanout = prod(fanout(a))       uniform_fanout = all(ufo(a))
len(src_regions) = prod(Ns(a))     len(dst_regions) = prod(Nd(a))
components       = prod(gcd(Ns(a), Nd(a)))
C = {a : Ns(a) > Nd(a)}    # coarsened
R = {a : Nd(a) > Ns(a)}    # refined

A component is a maximal set of source and destination regions exchanging only
among themselves. For all_to_all a component is a permute group, so, following
§3.2 — each tile contributes M slices and receives K:

M = len(dst_regions) / components      # contributed = consumers per group
K = len(src_regions) / components      # received    = producers per group

Note the sides: M counts destination regions. Inverting them is invisible on a
square exchange and wrong on every other — on 4 cores, {B:2} → {A:4} needs A ÷ 4
and B × 2, which M=4, K=2 gives and the swap does not. For gather the groups are
finer, one per destination region, since one source region may feed several.

What the axis table cannot say. num_cores is given, not derived. The
core-to-region maps cores_at_src / cores_at_dst are not a function of per-axis
data — identical divisions occur with contiguous [0,1,2,3] and strided
[0,8,16,24] groupings — so producer and consumer tile sets must come from the tables.
self_sufficient (a core's destination region being its own source region) needs
per-core equality. coverage is unreachable from a division pair, which always tiles
the tensor, so it survives as a guard on inputs from outside the shorthand.

7.3 The decision table

#	condition	result
—	prod(Ns(a)) != len(src_regions) or prod(Nd(a)) != len(dst_regions)	not a work-division pair — the axis table is invalid; enumerate regions. Check first
—	any axis non-uniform	insufficient information — ragged; no single op has uniform dependency-set cardinality
1	C = ∅ and R = ∅	region sets identical: no op needed if every core's region is its own, else inter_tile_consume — a relocation, or a broadcast where destination regions are shared
2	C = ∅, R ≠ ∅	inter_tile_scatter, scatter_dimensions = R
3	C ≠ ∅, R ≠ ∅, prod(Nd(a)) == num_cores	inter_tile_all_to_all, split_dimensions = R, concat_dimensions = C; one group per component
4	C ≠ ∅ otherwise	inter_tile_gather, gather_dimensions = C; consumers per group = cores_at_dst of its region

The dimension attributes are the axis sets themselves, in the order §1.2 fixes, so
a multi-axis split or concat needs no special case. Rows 2, 3 and 4 return
insufficient information when a source region has several holders, since R8 admits
one producer per group and the tables do not say which transmits (open question 9).

Two things worth stating plainly. Row 3's prod(Nd(a)) == num_cores is the one
irreducibly global test — holding the division fixed and varying the core count
changes the op, so no per-axis quantity can see it. And it is weaker than asking
whether any destination region is shared: the two part company on every row-4 output
under an idle-core reading, which is open question 8's territory and why the
membership step must run first.

7.4 Worked example

A real all-to-all — {mb:8, out:4} → {mb:32} on 32 cores.

axis	Ns(a)	Nd(a)	relation	fanin	fanout	uniform	gcd
mb	8	32	refined	1	4	yes	8
out	4	1	coarsened	4	1	yes	1

max_fanin = 4, max_fanout = 4, 32 regions a side, components = 8. Both sets
non-empty and prod(Nd(a)) = 32 = num_cores, so row 3: inter_tile_all_to_all,
split_dimensions = [mb], concat_dimensions = [out], with 8 groups and
M = K = 4 — eight independent 4-way exchanges, not one 32-way.

Two shapes are instructive by contrast. Source and destination cutting different
axes, {Lk:32} → {H:8}, makes both sets non-empty yet prod(Nd(a)) = 8 ≠ 32, so row 4
gathers — the only shape where the global comparison does the work, and unattested in
the measurements. And a selection — 32 destination regions all at one mb index —
manufactures Nd = {mb:512, out:32} if slice counts are taken as extent // size,
giving prod(Nd) = 16384 against 32 real regions; the validity guard catches it, and
counting slices from distinct start coordinates avoids inventing the cut at all. A
selection is not a delivery: it needs a select op before one.

7.5 Measured patterns

51 measured relayouts, each opfunc = "shuffle" on 32 cores, carrying explicit regions
and per-region core sets — so replication versus idleness, which a work division can
never settle, is read directly. The Pnn catalogue IDs were assigned from these files;
the decoded grid (§7.1) matches 10 of 14 to a measured shape.

pattern	files	n	src	dst	C	R	|dst_regions|	cpr	max_fanin	comps	row	KTIR op
P01, P02	BatchMatMulV2_QC_{3,12,21}_inpLds_1	3	{in:2, out:8, x:2}	{}	in,out,x	—	1	32	32	1	4	inter_tile_gather, gather_dimensions = [in, out, x] (all-gather)
P03	BatchMatMulV2_QC_{6,7,15,16,24,25}_inpLds_0	6	{mb:8, in:4}	{mb:8}	in	—	8	4	4	8	4	inter_tile_gather, gather_dimensions = [in]
P04	BatchMatMulV2_QC_{5,14,23}_inpLds_0, LayerNormNorm_QC_{1..6}_inpLds_{1,2}	15	{mb:32}	{mb:8}	mb	—	8	4	4	8	4	inter_tile_gather, gather_dimensions = [mb]
P05	none	—	[8,4,1]	[8,1,1]	unknown	unknown	unknown	unknown	unknown	unknown	—	no file: [8,1,1] needs a side with 8 active cores
P06 or P14	Exx2_QC_{1..6}_inpLds_0	6	{mb:8, out:4}	{mb:32}	out	mb	32	1	4	8	3	inter_tile_all_to_all, split [mb] / concat [out]; 8 groups, M=K=4
P06 or P14	Stcdp_QC_{5,18,30}_inpLds_0	3	{x:8, mb:4}	{x:32}	mb	x	32	1	4	8	3	inter_tile_all_to_all, split [x] / concat [mb]; 8 groups, M=K=4
P07	Add_QC_3_inpLds_1	1	{mb:16}	{mb:8, out:4}	mb	out	32	1	2	8	3	inter_tile_all_to_all, split [out] / concat [mb]; 8 groups, M=4, K=2
P08	none	—	{A:4, B:8}	{A:8, B:4}	B	A	32	1	2 lit.	16	3 or 1	no file; unresolved — see below
P09, P12	BatchMatMulV2_QC_{0,1,2}_inpLds_0	3	{mb:16}	{mb:8}	mb	—	8	4	2	8	4	inter_tile_gather, gather_dimensions = [mb]
P10, P11	PR #4061	—	[8,1,1] = {h:8} on 8 cores	[8,4,1] = {h:8} × 4 cores	—	—	8	4	1	8	1	inter_tile_consume, consumer set widened to the 4 holders — broadcast
P13	BatchMatMulV2_QC_27_inpLds_0	1	{in:32}	{}	in	—	1	28	32	1	4	inter_tile_gather, gather_dimensions = [in] (all-gather, 4 cores idle)
no ID	Mul_QC_{1,2,3,4,11,12,13,14,21,22,23,24}_inpLds_1	12	{y:16}	{y:32}	—	y	32	1	1	16	2	inter_tile_scatter, scatter_dimensions = [y]
no ID	Stcdp_QC_38_inpLds_0	1	{mb:8, out:4}	selection	—	—	32	1	—	—	guard	not a work-division pair — see below

Divisions are the measured ones, in layoutDimOrder_ order; cpr is cores per
destination region, comps is components.

What the measurements establish.

• A non-square all-to-all exists — 8 components with 2 source and 4 destination
  regions each, so M = 4, K = 2. Every catalogued pattern is square, which is why
  the M/K sides could not be checked from the IDs alone.
• A three-axis concat exists — C = [in, out, x], so §1.2's flattening order must
  be fixed over three axes. It is P01/P02, whose [8,4,1] source is three cuts
  coalesced; list-valued attributes are a requirement of a named pattern, not a corner.
• inter_tile_scatter is real — 12 files with single holders on both sides, so the
  producer-election escape does not fire and row 2 emits a definite op.
• Broadcast is real — P10/P11 keep every cut and only replicate the destination
  four ways, which is row 1 with a widened consumer set.
• Replicated sources are rare and idleness is the norm — every source region in the
  51 has one holder; 16 files have fewer source regions than cores and all resolve to
  single holders plus idle cores. The sparse-source cases come from the broadcast work.
• Idle cores occur on both sides, including one destination region held by 28 cores
  with 4 idle — which is P13, whose grid [1,28,1] records that holder count and not
  28 cuts, so no axis is cut into 28 and all 28 consumers hold the same region.
• §1.0 holds — 20 files coarsen or refine the stick axis and every piece size on it
  is an exact stick multiple.
• The contiguity assumption is false — four-core destination groups are contiguous
  in 9 files and strided in 15, so the core map is not a function of the division.
• One file is a selection, not a partition — 1/512 coverage, both uniformity flags
  true while the real fanout is {0, 8}. This is what the §7.3 validity guard is for.

Where the records and the measurements disagree. In each case the measurement is
the checkable one.

• P04 is recorded as {Lk:32} → {H:8}, cutting different axes, which gives a
  refined H and one component. The 15 files at that grid cut the same axis both
  sides, so R = ∅ and there are 8 components — a pure gather, matching the recorded
  label. Open question 4 exists only because of the other reading.
• P06/P14 are recorded with the coarsened and refined roles the opposite way round
  from the files, giving 4 groups of 8 rather than 8 of 4. A grid cannot say which entry
  is coarsened; the measurement settles it.
• P07 is a leading hypothesis only. Two structurally different shapes produce
  [16,1,1] → [8,4,1] — one with 32 distinct destination regions (all-to-all), one with
  8 regions of 4 cores (gather) — so the recorded "grouped all-gather" fits the second,
  leaving the first for P07 and making its label wrong. That is also what distinguishes
  P07 from P09/P12, invisible in the grid.
• P05 and P08 match no file. P05's destination needs a side with 8 active cores, and
  measured counts are 1, 16, 28 and 32; P08's source needs cut counts ascending in
  layout order, and the only non-descending case is P01/P02's. Both are recorded as
  unimplemented, so their absence is coherent.

7.6 What would still need measuring

1. Physical axis identities for the sides that match no file — P05's destination and
   P08's — which is what fixes their dimension attributes and slice counts.
2. P08's divisions keyed by physical axis, the only way to choose between row 1 and
   row 3 for it.
3. Whether the P07 label or the P07 assignment is wrong, which the grid cannot say.
4. Per-region counts for any pattern believed ragged. Uniformity holds on all 51
   measured files; the unmatched IDs are untested, and R6/R7 of §2 need it to be true
   rather than assumed.

[fabianlim]

@tnakaike incorporated the torch-spyre relayout analysis from @AdnanHoque in e196353

[tnakaike]

@AdnanHoque @fabianlim Thank you for your reviews and updates. I made a few additional minor changes to improve the readability of Section 9.

Pushed bbcc830 — a follow-up to your §9 rewrite, plus a rebase note.

Rebase note first. The branch was force-pushed. Your e196353 is now a01c23c — same content, same authorship, replayed onto current main (d450d8e, so including #66); the previous head sat on ecfb9ed. git diff e196353:docs/inter-tile-communication.md a01c23c:docs/inter-tile-communication.md is empty. Nothing of yours changed.

What bbcc830 does. Two clarifications. It changes no rule — §9 already decides the awkward case; this makes the decision reproducible by a reader.

(1) A strict definition for Ns(a) and Nd(a), so they cannot be read as extents. They are counts of the distinct slices an ownership table contains. On the one measured selection that distinction bites: Nd(mb) = 1, because all 32 cores name the same mb[511:512] — not 512, which is the extent. The preamble now says so, and carries that file's two ownership tables so the count can be read off real data.

(2) The row-0 condition, with a description and an example. Its verdict is right for a selection — "not a work-division pair — enumerate regions instead". The following is an example of a case where the classification fails: both region-count clauses match, so the row lets it through.

• prod(Ns(a)) = 8 × 4 = 32 = len(src_regions)
• prod(Nd(a)) = 1 × 32 = 32 = len(dst_regions)

What rejects it is coverage — the 1/512 §9.3 already reports: 32 distinct destination regions of one stick each, 32 × 64 = 2048 elements against 512 × 32 × 64 = 1048576. So the row now states that condition too, or either side's distinct regions do not cover the tensor, with a description of its two qualifications — sum a side's distinct regions, and measure against the value being delivered rather than the original tensor — and the tables above as the worked example. Same row, same verdict, nothing reclassified.

One thing I did not change, flagging in case you want to: the preamble still says "Everything else the classification needs is a product down the axes." With the coverage condition spelled out that is now slightly narrow — it reads slice sizes and the tensor shape, the only test in §9 that does.

For completeness, the other new commit on the branch, d5f87c3: names every table per the doc's **Bold name.** One row per … convention (11 tables), and corrects the §4 claim that the equal-type split_dimensions == concat_dimensions distributed transpose is what the SDSC backend emits — every measured all-to-all splits and concats different axes, so P == C conserves the element count while the type still changes.

[AdnanHoque]

@tnakaike @fabianlim Thanks for incorporating the earlier feedback. I would like us to use the real Torch-Spyre relayout edges as the acceptance contract for this design, rather than deciding coverage from operation names alone.

I put the full workload inventory in torch-spyre #4300. It covers all 55 prefill and 75 decode records in the pinned Granite catalog, the Gemma E0–E10 expert-body boundaries, and the known paged-attention cache/K/V/state edges. Missing current application captures are listed as gaps, so this does not claim that KTIR already passes them.

For each edge, I propose that we record the original source owners, the destination owners, the local value order, and whether the inputs are independent pieces or an already-completed reduction. Then we require three separate checks: the KTIR form verifies, the lowering moves the right values without an intermediate HBM spill, and the real consumer produces the right result.

The four-way paged-attention V case is a useful first example. Cores 0–3 each start with a different piece; cores 0, 8, 16, and 24 each need all four pieces in order. That is gather + broadcast. Loading a complete V page from HBM on one leader is not equivalent because it changes the starting ownership.

I suggest we settle these cases first in this PR:

• source pieces shared by several receivers;
• receivers that are not themselves producers;
• select followed by movement, such as Granite's last-token edge;
• moving an already-completed sum without reducing it again;
• equal split counts with different core ownership or local ordering.

Once we agree on the smallest KTIR examples for those cases, we can turn them into verifier and execution tests and then run the full inventory. Any equivalent on-chip lowering is fine; the requirement is the ownership relation and the observed result, not one prescribed spelling.

[tnakaike]

@AdnanHoque Thank you so much for summarizing real use cases. I created KTIR examples that cover the use cases which you suggested under docs/inter-tile-examples/. The following is the summary of the examples.

• source pieces shared by several receivers — kt01-shared-sources.mlir
• receivers that are not themselves producers — kt02-receiving-only-cores.mlir
• local ordering against core-ID order — kt03-logical-order.mlir
• multi-axis assembly — kt04-multi-axis-assembly.mlir
• select followed by movement, Granite's last-token edge — kt05-select-then-scatter.mlir
• equal split counts with different core ownership — kt06-owner-permutation.mlir
• moving an already-completed sum without reducing it again — kt07b-completed-sum.mlir, paired with kt07a-raw-contributions.mlir as the fold it must not be confused with

docs/inter-tile-examples/README.md describes how each example is derived and what each one is for, including which catalog record it comes from and what it establishes.

Some notes that are not in the README:

The spec commit settles what your earlier notes touched. §10.1 is resolved as n for every delivery op except reduce — receiving-only cores are the measured norm, 64 of the 130 records have destination owners that are not source owners. R5 is scoped so it cannot be read as forbidding shared sources, and §3.3 now states normatively that an assembling consumer's positions come from its own declared dependency set, which is what actually makes those assemblies well-defined.

Two files reach a verdict today. kt07a parses and passes --ktir-check-legality, and so does kt06's variant A — for the opposite reason, since that is the branch §9.1 row 1 resolves to "no op needed", so it has no delivery op to be missing. Everything else parses up to its delivery op and stops, because consume, gather and scatter are specified in §6 but absent from KTDP.td. Each file names the op it waits on. No execution or application verdict is claimed for any of them.

KT-06 turned out to be a clean 4x8 transpose, and its five records are one KTIR form. GR-PF-052 (relayouts[51]) plus [52], [53], [118], [119] all share the geometry and the owner map: 32 pieces to 32 pieces of the same size, P = 1, a bijection. With J = j/2 and M = mb/64, the source owner is 8J + M and the destination owner is J + 4M, which is exactly the row-major-versus-column-major core order you describe. It decomposes into 8 groups of 4: group g has consumers {4g .. 4g+3} drawing from producers {g, g+8, g+16, g+24}, the cores congruent to g mod 8. That grouping is not cosmetic — R1, R4 and R5 are obligations within a group, so 8 groups of 4 make all of R1, R3, R4 and R5 bite, one group of 32 leaves R1 and R3 nearly trivial, and 32 singleton groups make R4 and R5 vacuous. The residual hole is a permutation inside one group, which every rule accepts.

KT-04 needs no new op — it is gather with two entries in gather_dimensions — and its verdict splits. §4's normative flattening, R9's ascending-order rule and R12's per-axis check together close both spelling errors, a reversed list and a single axis where two are needed. They do not close the case where the producers' tile order and the required cell order are a transpose of each other, which is what your 2x2 toy encodes once its coordinates are read as (row, column): every rule is satisfied — ascending list, both axes, uniform extents, P = 4 = 2 x 2, the declared result type — and the assembly still comes out [[0,2],[1,3]]. That is the same finding as KT-03 in multi-axis form.

KT-03 needs no new capability, but its correctness is not something a verifier can reach. §3.3 orders an assembly by ascending tile id and no dependency set can change that, so the reorder belongs at the store: two stores at swapped base coordinates carry it, with identity order inside each and no extra data movement. A single access tile cannot, being a base plus a region relative to that base. Coverage §5 allows exactly this form — "local selection/reordering is acceptable if fully expressed and stays on-chip" — so the requirement is satisfied as written. What it costs is verifiability: no rule relates the store bases to §3.3's assembly order, so a verifier sees two well-formed stores that between them cover the region exactly once, while storing the assembly verbatim has the same shape, the same element count, the same coverage and the wrong answer. The correctness of the reorder therefore falls outside inter-tile verification, and the numerical check is its only guarantee. I did consider an ordering attribute on the delivery op redefining §3.3's l, and decided against proposing it: an attribute can be checked for well-formedness but never for intent, so it would not move this case into the verifier — it would only add a second statement of the ordering that can disagree with the stores. One aside for whoever owns KTDP.td, independent of this case: RFC 0682 defines access_tile_order both as "the rightmost dimension in the output space corresponds to the innermost iteration dimension" and as "the enumeration of points in the intermediate variable space", and only the second could reorder within a dimension. kt03 deliberately does not depend on the generous reading, and the dialect's op description carries neither sentence.

Two things about reading the catalog, in case they save anyone else the detour. relayouts[*].source_pieces is ordered lexicographically by its key string (p0, p1, p10, p11, ... p2, p20), so piece index is not owner order, and source_core_patterns is a separate summary that carries no piece key — the authoritative owner is the owners field on each piece. Deriving KT-06's map from index position instead gives a bijection that is not a transpose, which cost me a rewrite. And the byte fields are logical rather than physical: prod(extents) * word_length == logical_tensor_bytes exactly, including for GR-PF-055, where an SDSC run of the same logical shape shows a y extent of 1 carried as 64 padded lanes. So padding is invisible in the catalog by construction and no record can settle a layout question on its own — which is why kt06 and kt04 state their layout assumption explicitly instead of deriving one, and why KT-08 (stick indivisibility) is not written yet: it needs an SDSC run for one of the y-changing shapes first.

Still to write, for the record: KT-08 as above, KT-09 (reuse and lifetime, which your text notes does not require multiple delivery users of one future, so it does not need R2 relaxed), and KT-10's deliberate bad variants. Of those, only negatives against kt07a and kt06's variant A can actually be run today.