Skip to content

Reorganize and expand test infrastructure - #146

Merged
steveklabnik merged 1 commit into
trunkfrom
steveklabnik/push-wqkzmslwsxyl
Aug 18, 2025
Merged

Reorganize and expand test infrastructure#146
steveklabnik merged 1 commit into
trunkfrom
steveklabnik/push-wqkzmslwsxyl

Conversation

@steveklabnik

Copy link
Copy Markdown
Collaborator
  • Add test utility crates (rue-snapshot, rue-test-utils) for shared testing functionality
  • Implement property-based testing for parser, semantic analyzer, optimizer, and HIR
  • Add comprehensive parser test suite with 70+ test cases and snapshot testing
  • Reorganize tests into spec/, integration/, property/, and e2e/ directories
  • Add skip directive support to test runner for temporarily disabled tests
  • Expand CI/CD workflows with property tests, spec compliance, coverage, and nightly testing
  • Add test analysis and visualization scripts for tracking test trends
  • Create arithmetic and type system integration tests for the compiler

@steveklabnik
steveklabnik force-pushed the steveklabnik/push-wqkzmslwsxyl branch from 31c07c2 to 64172d9 Compare August 15, 2025 00:16
@github-actions

github-actions Bot commented Aug 15, 2025

Copy link
Copy Markdown

🧪 Test Runner Results

Summary

Metric Value
Total Tests 40
Passed 29
Failed 0
Pass Rate 72.5%
Spec Coverage 0%

Changes from Baseline

Total tests: 0 → 40 (+40)
📈 Passed: 0 → 29 (+29)
🆕 40 new tests

Detailed Changes

🆕 New Tests

  • examples/advanced/assignment_demo.rue
  • examples/advanced/casting.rue
  • examples/advanced/structs.rue
  • examples/basic/countdown.rue
  • examples/basic/factorial.rue
  • examples/basic/fibonacci.rue
  • examples/basic/simple.rue
  • examples/control_flow/if_demo.rue
  • examples/control_flow/while_demo.rue
  • tests/spec/grammar/block_expression_value.rue
  • tests/spec/grammar/comparison_operators.rue
  • tests/spec/grammar/function_multiple_params.rue
  • tests/spec/grammar/if_else_chain.rue
  • tests/spec/grammar/modulo_negative.rue
  • tests/spec/grammar/modulo_operator.rue
  • tests/spec/grammar/operator_precedence_basic.rue
  • tests/spec/grammar/operator_precedence_complex.rue
  • tests/spec/grammar/recursion_factorial.rue
  • tests/spec/grammar/recursion_fibonacci.rue
  • tests/spec/grammar/run_fail_division_by_zero.rue
  • tests/spec/grammar/run_pass_arithmetic.rue
  • tests/spec/grammar/snapshot_asm_factorial.rue
  • tests/spec/grammar/snapshot_mir_simple.rue
  • tests/spec/grammar/while_loop_basic.rue
  • tests/spec/grammar/while_loop_nested.rue
  • tests/spec/runtime/arithmetic_overflow_wrap.rue
  • tests/spec/runtime/array_operations.rue
  • tests/spec/runtime/run_fail_bounds_check.rue
  • tests/spec/semantics/block_scope_isolation.rue
  • tests/spec/semantics/compile_fail_type_error.rue
  • tests/spec/semantics/compile_pass_basic.rue
  • tests/spec/semantics/control_flow_if.rue
  • tests/spec/semantics/function_param_shadowing.rue
  • tests/spec/semantics/multiple_directives.rue
  • tests/spec/semantics/shadowing_cross_scope.rue
  • tests/spec/semantics/shadowing_different_types.rue
  • tests/spec/semantics/shadowing_same_scope.rue
  • tests/spec/semantics/type_error_assignment.rue
  • tests/spec/semantics/type_error_operator.rue
  • tests/spec/semantics/type_error_return.rue

✅ No test regressions detected.

Specification Coverage
Rule Test Count

@steveklabnik
steveklabnik force-pushed the steveklabnik/push-wqkzmslwsxyl branch 5 times, most recently from 4cdb0f3 to b1e672e Compare August 15, 2025 04:05
@steveklabnik

Copy link
Copy Markdown
Collaborator Author

Gonna have to review this one next week

@steveklabnik
steveklabnik force-pushed the steveklabnik/push-wqkzmslwsxyl branch from b1e672e to 887c787 Compare August 18, 2025 17:50
@github-actions

github-actions Bot commented Aug 18, 2025

Copy link
Copy Markdown

🧪 Test Results

Specification Tests

  • Total: 31
  • Passed: 29
  • Failed: 0

@github-actions

github-actions Bot commented Aug 18, 2025

Copy link
Copy Markdown

🚀 Performance Benchmark Results

End-to-End Compilation Times

Program Time Std Dev
bench_large 38.51ms ±0.58ms
bench_xlarge 79.99ms ±0.80ms
bench_huge 177.17ms ±1.24ms

Performance vs Baseline

  • bench_large: 0.0% 🟢
  • bench_xlarge: 0.0% 🟢
  • bench_huge: 0.0% 🟢

✅ No performance regressions detected.

Note: Only programs >1000 lines are benchmarked for reliable measurements in CI.

@steveklabnik
steveklabnik force-pushed the steveklabnik/push-wqkzmslwsxyl branch 5 times, most recently from 19dbb48 to 35272dc Compare August 18, 2025 21:15
- Port ExecutionSnapshot types to rue-snapshot crate
- Remove duplicate snapshot modules from rue-parser
- Remove duplicate snapshot implementation from rue-runner
- Fix arithmetic.rs duplicate tests
- Generate missing snapshot files for corpus tests
- Add rue-snapshot dependency to Buck2 test configs
- All tests now passing in both Cargo and Buck2
@steveklabnik
steveklabnik force-pushed the steveklabnik/push-wqkzmslwsxyl branch from 35272dc to 20da08a Compare August 18, 2025 21:42
@steveklabnik
steveklabnik merged commit 3b30d8a into trunk Aug 18, 2025
23 checks passed
@steveklabnik
steveklabnik deleted the steveklabnik/push-wqkzmslwsxyl branch August 18, 2025 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant