feat(challenges): add term-challenge WASM crate with evaluation logic - #38
Merged
Conversation
… logic Introduce the term-challenge-wasm crate under challenges/term-challenge-wasm/ that ports the term-challenge v2 ServerChallenge evaluation and validation logic to the WASM SDK (wasm32-unknown-unknown, no_std). The crate implements the Challenge trait from platform-challenge-sdk-wasm with a TermChallenge struct that: - Deserializes miner submissions (terminal interaction logs, task results) and challenge parameters (task definitions, optional LLM judge URL) from bincode - Scores submissions by computing pass/fail rates across difficulty tiers (easy/medium/hard) with aggregate scoring - Supports optional LLM-based judging via host_http_post() for tasks that require semantic evaluation of terminal output - Validates submission format and integrity (non-empty fields, score bounds, task count matching) The evaluation model is redesigned for WASM: all I/O-heavy operations (terminal command execution, output capture) happen on the host side. WASM receives pre-captured results and scores them deterministically. To support the register_challenge! macro requiring a const-initializable static, a ConstDefault trait is added to the SDK. The Challenge trait now requires ConstDefault (which itself requires Default), and the macro uses ConstDefault::DEFAULT instead of Default::default() for the static instance. New files: - challenges/term-challenge-wasm/Cargo.toml: cdylib crate with serde (no_std) and bincode (no_std) dependencies - challenges/term-challenge-wasm/src/lib.rs: Challenge trait implementation - challenges/term-challenge-wasm/src/scoring.rs: aggregate scoring algorithm - challenges/term-challenge-wasm/src/types.rs: no_std-compatible data types Modified files: - Cargo.toml: add term-challenge-wasm to workspace members - crates/challenge-sdk-wasm/src/lib.rs: add ConstDefault trait, update macro
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
term-challenge-wasmcrate that implements the terminal benchmark challenge evaluation as a WASM module targetingwasm32-unknown-unknown. This ports the v2 term-challenge scoring logic into the platform WASM SDK framework.Changes
challenges/term-challenge-wasm/: Newcdylibcrate implementing theChallengetrait fromplatform-challenge-sdk-wasmlib.rs:TermChallengestruct withevaluate()andvalidate()— deserializes bincode-encoded submissions and challenge params, runs scoring, and optionally calls an external LLM judge endpoint viahost_http_post()types.rs:no_std-compatible data types (Submission,TaskResult,TaskDefinition,ChallengeParams,Difficulty, LLM judge request/response) with serde derivesscoring.rs: Aggregate scoring logic — per-difficulty pass/fail tracking, pass-rate calculation, weight normalization, and summary formattingCargo.toml: Addedchallenges/term-challenge-wasmto workspace membersDesign Notes
std, notokio, no async — fullyno_stdwithalloconlyserde(no_std + alloc),bincode(no_std),platform-challenge-sdk-wasm(path dep)Supersedes #30 (which was closed due to force-push preventing reopen).