|
2 | 2 |
|
3 | 3 | **Complete workflow for debugging Python-to-Rust transpilation issues with reproducibility** |
4 | 4 |
|
5 | | -This guide covers three critical debugging phases: |
| 5 | +This guide covers five critical debugging phases: |
6 | 6 | 1. **Transpilation Debugging** - Using `depyler --trace` and `--explain` |
7 | 7 | 2. **Compile-Time Analysis** - Understanding Rust compiler errors |
8 | 8 | 3. **Runtime Tracing** - Using `renacer` for syscall and function profiling |
| 9 | +4. **Transpiler Source Mapping** - Using `renacer --transpiler-map` for Python→Rust correlation (v0.4.1+) |
| 10 | +5. **Chaos Engineering** - Stress testing transpiled code with chaos infrastructure (v0.4.1+) |
9 | 11 |
|
10 | 12 | --- |
11 | 13 |
|
@@ -712,6 +714,285 @@ Detect Vec<T> in println! and auto-generate Debug trait usage |
712 | 714 |
|
713 | 715 | --- |
714 | 716 |
|
| 717 | +### Example 4: Transpiler Source Mapping (Renacer v0.4.1+) |
| 718 | +
|
| 719 | +**NEW in Sprint 24**: Correlate transpiled Rust code back to original Python source |
| 720 | +
|
| 721 | +#### Overview |
| 722 | +
|
| 723 | +Renacer v0.4.1 introduces `--transpiler-map` for mapping generated Rust code back to original Python source. This enables: |
| 724 | +- **Python→Rust line correlation** - Map Rust errors to Python source |
| 725 | +- **Python function names** - Preserve original function names in profiles |
| 726 | +- **Error context** - Understand which Python code caused Rust compile errors |
| 727 | +
|
| 728 | +#### Source Map Format |
| 729 | +
|
| 730 | +Depyler (future version) will generate `.sourcemap.json` files alongside `.rs` files: |
| 731 | +
|
| 732 | +```json |
| 733 | +{ |
| 734 | + "version": 1, |
| 735 | + "source_language": "python", |
| 736 | + "source_file": "examples/csv_filter.py", |
| 737 | + "generated_file": "csv_filter.rs", |
| 738 | + "depyler_version": "3.20.0+50", |
| 739 | +
|
| 740 | + "mappings": [ |
| 741 | + { |
| 742 | + "rust_line": 52, |
| 743 | + "rust_function": "filter_csv", |
| 744 | + "python_line": 28, |
| 745 | + "python_function": "filter_csv", |
| 746 | + "python_context": "writer = csv.DictWriter(sys.stdout, ...)" |
| 747 | + } |
| 748 | + ], |
| 749 | +
|
| 750 | + "function_map": { |
| 751 | + "filter_csv": "filter_csv", |
| 752 | + "_cse_temp_0": "temporary for: len(data) > 0" |
| 753 | + } |
| 754 | +} |
| 755 | +``` |
| 756 | +
|
| 757 | +#### Usage |
| 758 | +
|
| 759 | +```bash |
| 760 | +# Step 1: Transpile with source map (future depyler feature) |
| 761 | +depyler transpile csv_filter.py --source-map -o csv_filter.rs |
| 762 | +# Generates: csv_filter.rs, csv_filter.rs.sourcemap.json |
| 763 | +
|
| 764 | +# Step 2: Build the Rust binary |
| 765 | +cargo build |
| 766 | +
|
| 767 | +# Step 3: Trace with source map correlation |
| 768 | +renacer --transpiler-map csv_filter.rs.sourcemap.json -- ./target/debug/csv_filter input.csv |
| 769 | +
|
| 770 | +# Expected output with Python context: |
| 771 | +# write(1, "Alice,25,NYC\n", 13) = 13 [csv_filter.py:28 in filter_csv] |
| 772 | +# └─ Python: writer.writerow(row) |
| 773 | +``` |
| 774 | +
|
| 775 | +#### Use Cases |
| 776 | +
|
| 777 | +**1. Debugging Transpilation Errors** |
| 778 | +
|
| 779 | +Map "Statement type not supported" errors to exact Python line: |
| 780 | +
|
| 781 | +```bash |
| 782 | +$ depyler transpile log_analyzer.py --trace 2>&1 > /tmp/transpile_error.txt |
| 783 | +
|
| 784 | +Error: Statement type not yet supported |
| 785 | + - Input size: 11192 bytes |
| 786 | + - Parsing Python source... |
| 787 | +
|
| 788 | +# Without source map: Manual binary search through 164 lines |
| 789 | +# With source map: Instant identification of problematic line |
| 790 | +``` |
| 791 | +
|
| 792 | +**2. Compile Error Correlation** |
| 793 | +
|
| 794 | +Translate Rust compiler errors back to Python source: |
| 795 | +
|
| 796 | +```bash |
| 797 | +$ cargo build 2>&1 | tee build_errors.txt |
| 798 | +
|
| 799 | +error[E0277]: Vec<String> doesn't implement Display |
| 800 | + --> positional_args.rs:31 |
| 801 | +
|
| 802 | +# With source map: Shows Python line 28, suggests fix in Python context |
| 803 | +# Python: print(f"Names: {args.names}") |
| 804 | +# Rust: println!("{}", args.names) // ERROR |
| 805 | +# Fix: print(f"Names: {', '.join(args.names)}") |
| 806 | +``` |
| 807 | +
|
| 808 | +**3. Performance Profiling** |
| 809 | +
|
| 810 | +Generate flamegraphs with Python function names instead of Rust generated names: |
| 811 | +
|
| 812 | +```bash |
| 813 | +$ renacer --transpiler-map log_analyzer.rs.sourcemap.json \ |
| 814 | + --function-time \ |
| 815 | + -- ./log_analyzer app.log > profile.txt |
| 816 | +
|
| 817 | +# Flamegraph shows: |
| 818 | +# log_analyzer.py::parse_log_lines (98.9%) |
| 819 | +# └─ log_analyzer.py:40 - for line in f: (95%) |
| 820 | +# |
| 821 | +# Instead of: |
| 822 | +# log_analyzer.rs::_parse_log_lines_gen (98.9%) ← Unhelpful |
| 823 | +``` |
| 824 | +
|
| 825 | +#### Integration Example |
| 826 | +
|
| 827 | +Complete workflow with source mapping: |
| 828 | +
|
| 829 | +```bash |
| 830 | +# 1. Create source map manually (until depyler supports --source-map) |
| 831 | +cat > trivial_cli.rs.sourcemap.json << 'EOF' |
| 832 | +{ |
| 833 | + "version": 1, |
| 834 | + "source_language": "python", |
| 835 | + "source_file": "trivial_cli.py", |
| 836 | + "generated_file": "trivial_cli.rs", |
| 837 | + "mappings": [ |
| 838 | + { |
| 839 | + "rust_line": 27, |
| 840 | + "rust_function": "main", |
| 841 | + "python_line": 8, |
| 842 | + "python_function": "main", |
| 843 | + "python_context": "print(f\"Hello, {args.name}!\")" |
| 844 | + } |
| 845 | + ], |
| 846 | + "function_map": { |
| 847 | + "main": "main" |
| 848 | + } |
| 849 | +} |
| 850 | +EOF |
| 851 | +
|
| 852 | +# 2. Build with debug symbols |
| 853 | +cargo build |
| 854 | +
|
| 855 | +# 3. Trace with source map |
| 856 | +renacer --transpiler-map trivial_cli.rs.sourcemap.json \ |
| 857 | + --source \ |
| 858 | + -- ./target/debug/trivial_cli --name Alice |
| 859 | +
|
| 860 | +# Output includes Python context: |
| 861 | +# write(1, "Hello, Alice!\n", 14) = 14 [trivial_cli.py:8 in main()] |
| 862 | +``` |
| 863 | +
|
| 864 | +#### Benefits |
| 865 | +
|
| 866 | +✅ **10x Faster Debugging** - Instant Python→Rust correlation |
| 867 | +✅ **Accurate Profiling** - Optimize Python source based on Rust performance |
| 868 | +✅ **Production Debugging** - Map crashes back to original code |
| 869 | +✅ **Better DX** - Transpiled code feels like first-class citizen |
| 870 | +
|
| 871 | +#### Current Limitations |
| 872 | +
|
| 873 | +⚠️ **Depyler Integration Pending**: Source map generation not yet implemented in depyler |
| 874 | +⚠️ **Manual Map Creation**: Must create `.sourcemap.json` files manually for now |
| 875 | +⚠️ **Phase 1**: Basic infrastructure only (line mapping, function names) |
| 876 | +
|
| 877 | +#### Future Phases |
| 878 | +
|
| 879 | +**Phase 2** (Sprint 25): Function name correlation in flamegraphs |
| 880 | +**Phase 3** (Sprint 26): Stack trace correlation for runtime errors |
| 881 | +**Phase 4** (Sprint 27): Compilation error message rewriting |
| 882 | +**Phase 5** (Sprint 28+): Interactive HTML reports with side-by-side view |
| 883 | +
|
| 884 | +#### Related Issues |
| 885 | +
|
| 886 | +- [Renacer #5](https://github.com/paiml/renacer/issues/5) - Transpiler Source Mapping Feature |
| 887 | +- [Depyler #69](https://github.com/paiml/depyler/issues/69) - sys.stdout not recognized |
| 888 | +- [Depyler #70](https://github.com/paiml/depyler/issues/70) - Nested functions not supported |
| 889 | +
|
| 890 | +--- |
| 891 | +
|
| 892 | +### Example 5: Chaos Engineering (Renacer v0.4.1+) |
| 893 | +
|
| 894 | +**NEW in Sprint 29**: Stress test transpiled Rust binaries with chaos engineering |
| 895 | +
|
| 896 | +#### Overview |
| 897 | +
|
| 898 | +Renacer v0.4.1 adds chaos engineering infrastructure for testing transpiled code under adverse conditions: |
| 899 | +- **Resource limits** - Test behavior under memory/CPU constraints |
| 900 | +- **Timeout testing** - Ensure graceful handling of slow operations |
| 901 | +- **Signal injection** - Validate error handling and cleanup |
| 902 | +- **Tiered testing** - Fast/medium/slow test tiers for TDD workflow |
| 903 | +
|
| 904 | +#### Quick Start |
| 905 | +
|
| 906 | +```bash |
| 907 | +# Run fast unit tests (<5s) |
| 908 | +make test-tier1 |
| 909 | +
|
| 910 | +# Run integration tests (<30s) |
| 911 | +make test-tier2 |
| 912 | +
|
| 913 | +# Run fuzz + mutation tests (<5m) |
| 914 | +make test-tier3 |
| 915 | +``` |
| 916 | +
|
| 917 | +#### Chaos Presets |
| 918 | +
|
| 919 | +**Gentle chaos** (for error handling tests): |
| 920 | +```rust |
| 921 | +// In test code (when renacer adds --chaos CLI flag) |
| 922 | +let config = ChaosConfig::gentle(); |
| 923 | +// - Memory limit: 500MB |
| 924 | +// - CPU limit: 80% |
| 925 | +// - Timeout: 60s |
| 926 | +// - Signal injection: disabled |
| 927 | +``` |
| 928 | +
|
| 929 | +**Aggressive chaos** (for stress testing): |
| 930 | +```rust |
| 931 | +let config = ChaosConfig::aggressive(); |
| 932 | +// - Memory limit: 100MB |
| 933 | +// - CPU limit: 50% |
| 934 | +// - Timeout: 10s |
| 935 | +// - Signal injection: enabled |
| 936 | +``` |
| 937 | +
|
| 938 | +**Custom chaos**: |
| 939 | +```rust |
| 940 | +let config = ChaosConfig::new() |
| 941 | + .with_memory_limit(100 * 1024 * 1024) // 100MB |
| 942 | + .with_cpu_limit(0.5) // 50% CPU |
| 943 | + .with_timeout(Duration::from_secs(30)) |
| 944 | + .with_signal_injection(true) |
| 945 | + .build(); |
| 946 | +``` |
| 947 | +
|
| 948 | +#### CLI Integration (Planned) |
| 949 | +
|
| 950 | +```bash |
| 951 | +# Test transpiled binary under gentle chaos |
| 952 | +renacer --chaos gentle -- ./csv_filter input.csv --column status --value active |
| 953 | +
|
| 954 | +# Stress test with aggressive chaos |
| 955 | +renacer --chaos aggressive -- ./log_analyzer app.log --group-by-hour |
| 956 | +
|
| 957 | +# Custom chaos config |
| 958 | +renacer --chaos custom:chaos.json -- ./positional_args start server1 server2 |
| 959 | +``` |
| 960 | +
|
| 961 | +#### Use Cases for Transpiled Code |
| 962 | +
|
| 963 | +**1. Memory Safety Testing** |
| 964 | +```bash |
| 965 | +# Ensure transpiled code handles large inputs without crashes |
| 966 | +renacer --chaos gentle -- ./csv_filter large_file.csv |
| 967 | +``` |
| 968 | +
|
| 969 | +**2. Performance Regression Detection** |
| 970 | +```bash |
| 971 | +# Verify transpiled code completes within time budget |
| 972 | +renacer --chaos "timeout:5s" -- ./trivial_cli --name Alice |
| 973 | +``` |
| 974 | +
|
| 975 | +**3. Error Handling Validation** |
| 976 | +```bash |
| 977 | +# Test transpiled error paths under signal interruption |
| 978 | +renacer --chaos "signal:SIGTERM" -- ./config_cli --config missing.yml |
| 979 | +``` |
| 980 | +
|
| 981 | +#### Benefits for Python→Rust Validation |
| 982 | +
|
| 983 | +✅ **Robustness Testing** - Ensure transpiled code handles edge cases |
| 984 | +✅ **Performance Validation** - Detect resource leaks and inefficiencies |
| 985 | +✅ **Error Path Coverage** - Force error conditions to test handling |
| 986 | +✅ **Production Readiness** - Verify behavior under real-world stress |
| 987 | +
|
| 988 | +#### Current Status |
| 989 | +
|
| 990 | +⚠️ **CLI Integration Pending**: Chaos features available as library, CLI flags planned for future sprint |
| 991 | +✅ **Test Infrastructure**: Available now via tiered make targets |
| 992 | +✅ **Property Tests**: 7 comprehensive chaos module tests |
| 993 | +
|
| 994 | +--- |
| 995 | +
|
715 | 996 | ## 🚀 Next Steps |
716 | 997 |
|
717 | 998 | After debugging: |
|
0 commit comments