Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 10 additions & 20 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,7 @@ Each challenge defines:
- Submission formats and requirements
- Scoring algorithms

The `term-challenge` crate lives in-tree at `challenges/term-challenge/` and is compiled to WASM for production evaluation. External challenges import `platform-challenge-sdk` as a git dependency.

| Challenge | Location | Description |
|-----------|----------|-------------|
| Terminal Bench | [`challenges/term-challenge/`](challenges/term-challenge/) | Terminal task benchmark (WASM evaluation module) |
| Terminal Bench v2 | [`challenges/term-challenge-wasm/`](challenges/term-challenge-wasm/) | Terminal benchmark with LLM judge support (WASM `cdylib`) |
| *(others)* | *(external repos or `challenges/` subdirectories)* | *(challenge-specific)* |
Challenge crates are maintained in their own repositories and import `platform-challenge-sdk-wasm` as a git dependency. See the `challenges/` directory for instructions on adding a new challenge.

---

Expand Down Expand Up @@ -49,19 +43,19 @@ flowchart LR
Develop your agent following the challenge-specific requirements. Challenge crates implement the `Challenge` trait from `platform-challenge-sdk-wasm`:

```rust
// Example: challenges/term-challenge/src/lib.rs
// Example: my-challenge/src/lib.rs
use platform_challenge_sdk_wasm::{Challenge, EvaluationInput, EvaluationOutput};

pub struct TermChallenge;
pub struct MyChallenge;

impl Challenge for TermChallenge {
fn name(&self) -> &'static str { "term-challenge" }
impl Challenge for MyChallenge {
fn name(&self) -> &'static str { "my-challenge" }
fn version(&self) -> &'static str { "0.1.0" }
fn evaluate(&self, input: EvaluationInput) -> EvaluationOutput { /* ... */ }
fn validate(&self, input: EvaluationInput) -> bool { /* ... */ }
}

platform_challenge_sdk_wasm::register_challenge!(TermChallenge, TermChallenge::new());
platform_challenge_sdk_wasm::register_challenge!(MyChallenge, MyChallenge::new());
```

**Check the challenge documentation** for the correct submission format and evaluation criteria.
Expand Down Expand Up @@ -140,9 +134,8 @@ Each challenge defines its own scoring algorithm in its `evaluate()` method. Val
Build and test challenge WASM modules locally:

```bash
# Build the WASM artifacts
cargo build --release --target wasm32-unknown-unknown -p term-challenge
cargo build --release --target wasm32-unknown-unknown -p term-challenge-wasm
# Build a challenge WASM artifact (example)
cargo build --release --target wasm32-unknown-unknown -p my-challenge

# Run workspace tests
cargo test
Expand All @@ -163,8 +156,6 @@ flowchart TB
Platform --> Validator[validator-node]
Platform --> Runtime[wasm-runtime-interface]
Platform --> P2P[p2p-consensus]
Platform --> TC[challenges/term-challenge]
Platform --> TCW[challenges/term-challenge-wasm]
```

**Workspace crates** (from `Cargo.toml`):
Expand All @@ -182,10 +173,9 @@ flowchart TB
- `crates/p2p-consensus` — libp2p gossipsub + DHT consensus
- `crates/wasm-runtime-interface` — WASM runtime host interface
- `bins/validator-node` — main validator binary
- `bins/platform-cli` — CLI for downloading and managing challenge CLIs
- `bins/utils` — CLI utilities
- `bins/mock-subtensor` — mock Bittensor node for testing
- `challenges/term-challenge` — Terminal Bench WASM challenge
- `challenges/term-challenge-wasm` — Terminal Bench v2 WASM challenge (LLM judge)
- `tests` — integration tests

**Non-workspace crate** (exists on disk but not in workspace members):
Expand All @@ -198,7 +188,7 @@ flowchart TB
## Getting Started

1. **Choose a challenge** you want to participate in
2. **Read the challenge documentation** (e.g., `challenges/term-challenge/`)
2. **Read the challenge documentation** for your chosen challenge
3. **Understand the submission format** from the challenge's types and evaluation logic
4. **Submit** through the P2P network
5. **Monitor** your submission status and scores
Expand Down
18 changes: 0 additions & 18 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 1 addition & 4 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,11 @@ members = [
"crates/p2p-consensus",
"crates/wasm-runtime-interface",
"crates/challenge-sdk-wasm",
"challenges/term-challenge-wasm",
"bins/validator-node",
"bins/utils",
"bins/mock-subtensor",
"bins/platform-cli",
"tests",
"challenges/term-challenge",
"challenges/term-challenge-wasm",
]
# Note: Challenges are in separate repositories and import platform-challenge-sdk as a git dependency
# Note: WASM runtime removed - updates via git, version checked at handshake
Expand Down Expand Up @@ -103,7 +100,7 @@ w3f-bls = { git = "https://github.com/opentensor/bls", branch = "fix-no-std" }
too_many_arguments = "allow"
large_enum_variant = "allow"
type_complexity = "allow"
await_holding_lock = "warn" # TODO: Fix async lock issues properly
await_holding_lock = "warn"
collapsible_match = "allow"
collapsible_if = "allow"

Expand Down
46 changes: 35 additions & 11 deletions bins/platform-cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,22 +49,12 @@ fn default_true() -> bool {

impl Default for PlatformConfig {
fn default() -> Self {
let mut challenges = HashMap::new();
challenges.insert(
"term-challenge".to_string(),
ChallengeConfig {
github_repo: "PlatformNetwork/term-challenge".to_string(),
binary_name: "term-cli".to_string(),
command_alias: "term".to_string(),
auto_update: true,
},
);
Self {
network: NetworkConfig {
rpc_endpoint: "wss://chain.platform.network".to_string(),
netuid: 100,
},
challenges,
challenges: HashMap::new(),
}
}
}
Expand Down Expand Up @@ -262,10 +252,44 @@ fn find_matching_asset(assets: &[GitHubAsset]) -> Option<&GitHubAsset> {

// ==================== GitHub API ====================

/// Validate that a GitHub repo string is in the expected `owner/repo` format.
///
/// Prevents URL path injection when the value is interpolated into API URLs.
/// Only alphanumeric characters, hyphens, underscores, and dots are permitted
/// in each segment.
fn validate_github_repo(repo: &str) -> Result<()> {
let parts: Vec<&str> = repo.split('/').collect();
if parts.len() != 2 {
anyhow::bail!(
"Invalid github_repo '{}': must be in 'owner/repo' format",
repo
);
}
for part in &parts {
if part.is_empty() {
anyhow::bail!(
"Invalid github_repo '{}': owner and repo must not be empty",
repo
);
}
if !part
.chars()
.all(|c| c.is_ascii_alphanumeric() || c == '-' || c == '_' || c == '.')
{
anyhow::bail!(
"Invalid github_repo '{}': contains disallowed characters",
repo
);
}
}
Ok(())
}

async fn fetch_latest_release(
client: &reqwest::Client,
github_repo: &str,
) -> Result<GitHubRelease> {
validate_github_repo(github_repo)?;
let url = format!("{}/repos/{}/releases/latest", GITHUB_API_BASE, github_repo);
debug!("Fetching latest release from {}", url);

Expand Down
17 changes: 0 additions & 17 deletions challenges/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ This directory contains challenge crates that integrate with the Platform valida
challenges/
├── README.md # This file
├── compiled/ # Built WASM artifacts (generated by build-wasm.sh)
├── term-challenge/ # Terminal benchmark challenge (WASM)
└── [your-challenge]/ # Your custom challenge crate
```

Expand Down Expand Up @@ -53,19 +52,6 @@ sequenceDiagram
- Must support state persistence for hot-reload.
- Must produce deterministic results for consensus.

## Term Challenge (Terminal Bench)

The `term-challenge-wasm` crate provides the Terminal Bench challenge as a WASM module. To build it:

```bash
# Build term-challenge specifically
./scripts/build-wasm.sh term-challenge-wasm

# The compiled WASM will be in challenges/compiled/term_challenge_wasm.wasm
```

See the [term-challenge repository](https://github.com/PlatformNetwork/term-challenge) for agent development, task definitions, and scoring details.

## Build WASM Artifacts

```bash
Expand All @@ -74,9 +60,6 @@ See the [term-challenge repository](https://github.com/PlatformNetwork/term-chal

# Build all challenge crates (discovers crates under challenges/*/)
./scripts/build-wasm.sh

# Example: build term-challenge
./scripts/build-wasm.sh term-challenge-wasm
```

The build script will:
Expand Down
13 changes: 0 additions & 13 deletions challenges/term-challenge-wasm/Cargo.toml

This file was deleted.

Loading