Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 25 additions & 12 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,19 +21,30 @@
| Module | Responsibility |
|---|---|
| `src/lib.rs` | Library re-exports, public `parse()` entry point |
| `src/error.rs` | `ParseError`, `MatchedPairError` via thiserror |
| `src/token.rs` | `TokenType` enum, `Token` struct |
| `src/lexer.rs` | Hand-written context-sensitive tokenizer |
| `src/lexer_word.rs` | Word/expansion parsing (split for complexity) |
| `src/lexer_matched.rs` | Matched pair parsing for nested constructs |
| `src/ast.rs` | `Node` enum with all 50+ variants |
| `src/sexp.rs` | S-expression output via Display trait |
| `src/parser.rs` | Recursive descent parser (top-level) |
| `src/parser_compound.rs` | Compound commands: if/while/for/case/select/coproc |
| `src/parser_arith.rs` | Arithmetic expression parser |
| `src/parser_cond.rs` | Conditional expression parser `[[ ]]` |
| `src/ast.rs` | `Node` struct, `NodeKind` enum with 50+ variants, `Span` |
| `src/token.rs` | `TokenType` enum (62 variants), `Token` struct |
| `src/error.rs` | `RableError` (Parse / MatchedPair) via thiserror |
| `src/context.rs` | Parsing context and state management |
| `src/lexer/` | Hand-written context-sensitive tokenizer |
| `src/lexer/quotes.rs` | Quote and escape handling |
| `src/lexer/heredoc.rs` | Here-document processing |
| `src/lexer/words.rs` | Word boundary detection |
| `src/lexer/word_builder.rs` | Word assembly with segments |
| `src/lexer/expansions.rs` | Parameter, command, arithmetic expansion parsing |
| `src/lexer/operators.rs` | Operator recognition |
| `src/parser/` | Recursive descent parser (top-level) |
| `src/parser/compound.rs` | Compound commands: if/while/for/case/select/coproc |
| `src/parser/conditional.rs` | Conditional expression parser `[[ ]]` |
| `src/parser/helpers.rs` | Common parsing utilities |
| `src/parser/word_parts.rs` | Word segment processing |
| `src/sexp/` | S-expression output via Display trait |
| `src/sexp/word.rs` | Word segment formatting |
| `src/sexp/ansi_c.rs` | ANSI-C quoting escapes |
| `src/format/` | Canonical bash reformatter (command substitution content) |
| `src/python.rs` | PyO3 bindings (feature-gated) |
| `tests/` | Integration tests using Parable's .tests format |
| `tests/integration.rs` | Integration test runner for .tests files |
| `tests/parable/` | Parable test corpus (36 files, 1,604 tests) |
| `tests/oracle/` | bash-oracle compatibility tests (11 files) |

## Code Limits

Expand Down Expand Up @@ -75,3 +86,5 @@ bash source code
```

Run with `cargo test`. The test runner reads `.tests` files, parses input, compares S-expression output.

Oracle tests in `tests/oracle/` provide additional coverage from bash-oracle fuzzing. Run with `just test-oracle`.
113 changes: 92 additions & 21 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,19 @@ Thank you for your interest in contributing to Rable! This project is a Rust rei
git clone https://github.com/mpecan/rable.git
cd rable

# Run all checks
# Run all checks (format, lint, test)
just check

# Full setup including Python environment
just setup
```

### Requirements

- **Rust 1.93+** — pinned in `rust-toolchain.toml`, installed automatically by rustup
- **Python 3.12+** — needed for Python bindings and fuzzing tools
- **[just](https://github.com/casey/just)** — task runner (`cargo install just` or `brew install just`)

## Development Workflow

1. **Make your changes**
Expand All @@ -33,16 +39,16 @@ All PRs must:
- Pass all unit tests
- Not introduce `unwrap()`, `expect()`, `panic!()`, or `todo!()` calls

## Code Style
### Code Limits

| Limit | Value |
|---|---|
| Line width | 100 chars |
| Function length | 60 lines max |
| Cognitive complexity | 15 max |
| Function arguments | 5 max |
| Limit | Value | Enforced by |
|---|---|---|
| Line width | 100 chars | `.rustfmt.toml` |
| Function length | 60 lines max | `clippy.toml` |
| Cognitive complexity | 15 max | `clippy.toml` |
| Function arguments | 5 max | `clippy.toml` |

These are enforced by `clippy.toml` and `.rustfmt.toml`.
These are enforced by clippy and rustfmt — `just check` will catch violations.

## Adding Support for New Bash Syntax

Expand All @@ -61,35 +67,100 @@ These are enforced by `clippy.toml` and `.rustfmt.toml`.

4. **Run the full suite**: `just test-parable` must show 1604/1604 (or more, if you added tests)

### Running a specific test file

To iterate quickly on a particular area:

```bash
just test-file 12_command_substitution # run a single test file
```

The `RABLE_TEST` environment variable filters which test file to run.

## Project Structure

```
src/
lib.rs Public API
ast.rs AST node types
token.rs Token types
error.rs Error types
lexer/ Context-sensitive tokenizer
parser/ Recursive descent parser
sexp/ S-expression output
format/ Canonical bash reformatter
python.rs PyO3 bindings (feature-gated)
lib.rs Public API: parse() entry point, re-exports
ast.rs AST node types (NodeKind enum with 50+ variants)
token.rs Token types and lexer output
error.rs Error types (ParseError, MatchedPairError)
context.rs Parsing context and state management
lexer/ Context-sensitive tokenizer
mod.rs Main lexer loop
quotes.rs Quote and escape handling
heredoc.rs Here-document processing
words.rs Word boundary detection
word_builder.rs Word assembly with segments
expansions.rs Parameter/command/arithmetic expansion parsing
operators.rs Operator recognition
tests.rs Lexer unit tests
parser/ Recursive descent parser
mod.rs Top-level parsing
compound.rs if/while/for/case/select/coproc
conditional.rs [[ ]] expression parsing
helpers.rs Common parsing utilities
word_parts.rs Word segment processing
tests.rs Parser unit tests
sexp/ S-expression output
mod.rs Main formatter
word.rs Word segment formatting
ansi_c.rs ANSI-C quoting escapes
format/ Canonical bash reformatter
mod.rs Used for command substitution content
python.rs PyO3 bindings (feature-gated under "python")
tests/
integration.rs Parable compatibility test runner
parable/ Test corpus from the Parable project
benchmark.py Performance benchmark
integration.rs Test runner for .tests files
parable/ Parable test corpus (36 files, 1,604 tests)
oracle/ bash-oracle compatibility tests (11 files)
run_tests.py Python test harness for Parable compatibility
benchmark.py Performance comparison vs Parable
fuzz.py Differential fuzzer (mutate/generate/minimize modes)
generate_oracle_tests.py Generate oracle tests from bash-oracle
examples/
basic.rs Basic usage example
```

## Python Bindings

The Python bindings are feature-gated under `python` and built with [maturin](https://www.maturin.rs/):

```bash
just venv # create virtual environment (one-time)
just develop # build and install in development mode
just test-python # run Parable's own test runner against Rable
just benchmark # compare performance
```

## Fuzzing

Rable includes a differential fuzzer that compares output against Parable to catch edge-case divergences:

```bash
just setup # one-time Python environment setup
just fuzz-mutate 50000 # mutate existing test inputs (default: 10,000)
just fuzz-generate 10000 # generate random bash fragments (default: 5,000)
just fuzz-minimize 'input' # minimize a failing input
```

If you find a divergence, you can generate oracle tests from it:

```bash
just fuzz-generate-tests # regenerate oracle test files (requires bash-oracle)
just test-oracle # run oracle test suite
```

## CI

CI runs on every push to `main` and on pull requests. It includes:

- **Lint** — format check + clippy
- **Test** — full Rust test suite + oracle compatibility report
- **Python** — build PyO3 bindings, run Python tests, Parable compatibility
- **Benchmark** — performance comparison vs Parable (PRs only)

Run `just ci` locally to replicate what CI does.

## Questions?

Open an issue on GitHub. We're happy to help!
Expand Down
Loading
Loading