Consider replacing regex parser with xDSL's MLIR text parser

[fabianlim]

Motivation

Thanks to @KFAFSP for letting me know about xDSL, something to keep in mind as a tool that can serve as a use case between full regex and full MLIR. cc: @tardieu @dgrove-oss @tnakaike

We maintain ~1900 lines of hand-rolled regex parsing (parser.py, parser_utils.py, parser_ast.py) plus ~200 lines of per-dialect custom parsers. This is fragile — we've hit bugs with indexing map parsing (#174, #180), attribute extraction, and region handling. Each new MLIR syntax pattern requires adding more regex.

Proposal

Replace the regex parser with xDSL's pure-Python MLIR text parser. xDSL is a community-maintained Python MLIR compiler framework with a proper recursive-descent parser that handles the full MLIR text format.

Example: today vs xDSL

Today — parsing linalg.generic requires a bespoke @register_parser with fragile regex:

@register_parser("linalg.generic")
def parse_linalg_generic(op_text, parse_ctx):
    # indexing_maps = [affine_map<(d0, d1) -> (d0)>, ...]
    maps = []
    maps_match = re.search(r'indexing_maps\s*=\s*', op_text)
    if maps_match:
        raw_maps = parse_attr_list(op_text[maps_match.end() - 1:])
        for raw in raw_maps:
            maps.append(parse_affine_map(raw))

    # iterator_types = ["parallel", "reduction", ...]
    iter_match = re.search(r'iterator_types\s*=\s*\[([^\]]*)\]', op_text)
    ...

    ins_match = re.search(r'\bins\s*\(([^)]+)\)', op_text)
    ins_operands = find_ssa_names(ins_match.group(1).split(':')[0])
    ...

Every new attribute or syntax variant requires adding another regex. The bracket-matching for nested affine_map<...> inside [...] needed its own parse_attr_list helper. Bugs are silent — if a regex doesn't match, the attribute is quietly missing.

With xDSL — standard dialects (linalg, arith, scf, tensor) are parsed correctly out of the box:

from xdsl.parser import Parser
from xdsl.ir import MLContext

ctx = MLContext(allow_unregistered=True)
module = Parser(ctx, mlir_text).parse_module()

# All standard-dialect ops are parsed correctly — attributes, regions,
# types, affine maps are structured objects, not regex captures.
for op in module.walk():
    # op.attributes["indexing_maps"] → ArrayAttr of AffineMapAttr
    # op.attributes["iterator_types"] → ArrayAttr of StringAttr
    # op.regions[0].blocks[0].args → block arguments with types
    ...

No per-dialect parser registration needed for standard MLIR dialects. Nested attributes, affine maps, regions all parsed correctly by the grammar.

ktdp dialect handling

Our KTIR uses pretty-printed (custom) syntax for ktdp ops:

%x_view = ktdp.construct_memory_view %ptr, sizes: [1, 64], strides: [64, 1] {
  memory_space = #ktdp.spyre_memory_space<HBM>
}

xDSL's allow_unregistered=True only handles generic form ("ktdp.op"(%args) {attrs} : types), not custom pretty-print syntax. Two options:

1. Compiler emits generic form — use --mlir-print-op-generic flag. Zero code on our side, but the KTIR text becomes harder to read.
2. Define ktdp ops in xDSL's dialect framework — declarative Python classes that describe the op syntax. xDSL generates parser/printer automatically. This is more structured than regex but still code we write. Example:

from xdsl.irdl import irdl_op_definition, operand_def, result_def, attr_def

@irdl_op_definition
class ConstructMemoryView(IRDLOperation):
    name = "ktdp.construct_memory_view"
    ptr = operand_def()
    result = result_def()
    sizes = attr_def(DenseArrayAttr)
    strides = attr_def(DenseArrayAttr)
    memory_space = attr_def(SpyreMemorySpaceAttr)

Option 2 is more work upfront but gives us type-checked op definitions and auto-generated parsing. The key win is that standard dialects — where our regex keeps breaking (linalg, scf, tensor, arith) — come for free.

Integration approach

1. Add xdsl as a dependency (pure Python; deps: immutabledict, ordered-set, typing-extensions)
2. Define ktdp ops in xDSL's IRDL framework (or use generic form)
3. Write a translation layer (~200-300 lines) converting xDSL's Operation IR → our Operation dataclass
4. Gradually deprecate the regex parser, keeping it as fallback initially

Trade-offs

	Regex parser (today)	xDSL	mlir_frontend (C++ bindings)
Correctness	Fragile, silent failures	Proper grammar, errors on invalid input	Correct by construction
Dependencies	None	3 pure-Python packages	Full MLIR/LLVM build
Maintenance	We own all parser bugs	Community-maintained for standard dialects	Upstream MLIR
ktdp support	Custom regex (done)	IRDL definitions or generic form	Native (C++ dialect)
Unregistered ops	Only what we wrote regex for	Yes (generic form)	Yes

Priority

Low — the regex parser works for our current subset, and the mlir_frontend path exists for full correctness. This becomes higher priority if we keep hitting parser edge cases in standard dialects.