feat: enrich AST with structured word spans and assignment detection

[mpecan]

Summary

Moves expansion boundary tracking into the lexer and eliminates duplicate re-parsing in the sexp/format layers. The lexer now records WordSpans for all 14 bash expansion types with full quoting context, and all downstream code uses spans instead of re-scanning word value strings.

Key changes

• WordBuilder (word_builder.rs): New lexer type bundling word value + spans + quoting context stack. All 14 lexer word-reading functions converted from &mut String to &mut WordBuilder
• 14 span kinds recorded: CommandSub, ArithmeticSub, ParamExpansion, SimpleVar, AnsiCQuote, LocaleString, ProcessSub, SingleQuoted, DoubleQuoted, Backtick, BracketSubscript, Extglob, DeprecatedArith, Escape
• QuotingContext tracked per span: None, DoubleQuote, ParamExpansion, CommandSub, Backtick — enables context-sensitive ANSI-C handling ($'...' inside ${...} vs "...")
• segments_from_spans() replaces parse_word_segments() — converts spans to segments without re-parsing, with top-level span filtering and context-aware formatting
• Assignment detection: Lexer emits AssignmentWord tokens; parser populates Command.assignments
• Array parsing at lex time: read_array_elements() parses array content as words directly, replacing post-hoc normalize_array_content string manipulation
• Token spans threaded to all nodes: word_node_from_token() and cond_term_from_token() preserve lexer spans; write_redirect() uses spans instead of string searches

Dead code removed (~540 lines)

Function	File	Lines
parse_word_segments() + helpers	sexp/word.rs	~195
extract_paren_content()	sexp/mod.rs	~107
try_normalize_array() + normalize_array_content()	sexp/word.rs	~107
needs_word_processing(), write_word_value(), needs_value_path()	sexp/	~25
should_format_from_value(), parts_to_segments()	sexp/word.rs	~40
skip_single/double/backtick, is_backslash_escaped, read_balanced_delim	context.rs	~110

Numbers

Metric	Value
Files changed	17 (+ 2 new)
Lines added	~960
Lines removed	~880
Net	~+80 (includes 22 new span tests + 6 assignment tests)
Oracle	167/181 (was 165 at baseline — gained 2 from eliminating buggy re-parsing)

Test plan

• 130 tests pass (87 unit + 37 integration + 6 doc)
• Oracle: 167/181 (+2 improvement)
• cargo clippy --all-targets -- -D warnings clean
• S-expression output identical to before (Parable compatibility)
• 16 span recording tests verify byte offsets for all expansion types
• 6 assignment tests verify lexer + parser behavior
• 7 word decomposition tests use real lexer spans (not re-parsing fallback)

🤖 Generated with Claude Code