Goal: Reduce or eliminate the need for Python-side scripts when setting up MLE-Bench or custom tasks in sia_rust.
Part of the broader effort to deprecate the Python bridge (#137) and move toward a more self-contained Rust experience.
Tasks
- Port or re-implement the MLE-Bench dataset download/prep logic in Rust (or provide a clean one-command native path).
- Make task discovery and
evaluate contract work without requiring a Python environment for the core sia run flow.
- Keep Python
evaluate.py support during transition but mark as deprecated for new tasks.
Related: #137 (deprecation), #20 (original task system), #99 (new legal benchmark task).
Goal: Reduce or eliminate the need for Python-side scripts when setting up MLE-Bench or custom tasks in sia_rust.
Part of the broader effort to deprecate the Python bridge (#137) and move toward a more self-contained Rust experience.
Tasks
evaluatecontract work without requiring a Python environment for the coresia runflow.evaluate.pysupport during transition but mark as deprecated for new tasks.Related: #137 (deprecation), #20 (original task system), #99 (new legal benchmark task).