ninja

ninja is the miner-facing starter harness for Subnet 66. Miners should edit agent.py and keep validator systems, task generators, scoring, wallets, and infrastructure out of this repo.

Production submissions are private. Send your agent.py to the Subnet 66 submission API with a signature from your registered miner hotkey. Once the API accepts it, the validator can queue it directly from the private submission ledger.

For the miner-facing submission guide, see MINER_SUBMISSION_CHECKLIST.md.

What You Are Allowed To Edit

• agent.py for miner submissions

Do not add production mining changes outside agent.py. Docs and helper scripts may be updated by maintainers, but the submitted miner code is the single agent.py file sent to the private submission API.

Nothing else should be added here for production mining, including (but not limited to):

• validator service code
• PM2 configs or service orchestration
• task pool or dataset tooling
• R2 tooling
• chain wallets
• benchmark/workspace generators or generated artifacts

Contract

The validator imports agent.py and calls:

solve(
    repo_path="/tmp/task_repo",
    issue="Fix the bug...",
    model="validator-managed-model",
    api_base="http://validator-proxy/v1",
    api_key="per-run-proxy-token",
)

solve(...) must return:

{
    "patch": "... unified git diff ...",
    "logs": "...",
    "steps": 0,
    "cost": None,
    "success": True,
}

The starter implementation is intentionally one file with no external Python dependencies. It uses the validator-provided OpenAI-compatible /v1/chat/completions endpoint and a bash action loop.

Miners should not add their own OpenRouter/OpenAI keys or hardcode a model. The validator passes a managed model id, proxy URL, and per-run proxy token into solve(...). That proxy enforces request limits, token limits, cost caps, and model routing. In production, every miner agent should use this one inference surface.

Sampling is also validator-owned. Do not add or tune request fields such as temperature, top_p, top_k, seed, penalties, logit_bias, or logprobs. The validator proxy enforces the managed policy and strips miner-controlled fields before forwarding.

Editing

Work directly in agent.py. The validator owns the task repo and sandbox, so changes should focus on how the agent reasons, gathers context, edits files, and returns a diff.

Useful local environment variables for sandbox runs:

AGENT_MAX_STEPS=40
AGENT_COMMAND_TIMEOUT=30
AGENT_MODEL=validator-managed-model
OPENAI_BASE_URL=http://validator-proxy/v1
OPENAI_API_KEY=per-run-proxy-token
AGENT_MAX_TOKENS=2048

Keep these boundaries intact:

• preserve the solve(repo_path, issue, model, api_base, api_key) entry point
• return patch, logs, steps, cost, and success
• use only the supplied api_base and api_key
• do not hardcode another model, provider endpoint, API key, wallet, scorer, or validator secret
• do not add third-party Python dependencies
• do not read or exfiltrate host secrets, hidden tests, prompts, or evaluator data

Blocked Edits

Submissions are rejected if they:

• change the solve(...) entry-point contract
• add forbidden provider/secret references
• attempt to hardcode or route around the managed model/proxy (api_base, api_key, model)
• add sampling/decoding control (temperature, top_p, top_k, seed, penalties, logit_bias, logprobs, etc.)
• include non-miner infra code or workflow/config assumptions
• touch validation, scoring, or repo-control behavior instead of agent behavior

Private Submission Flow

1. Edit agent.py.
2. Make sure the hotkey you will sign with is currently registered on Subnet 66.
3. Submit it to the private API with your registered miner hotkey wallet:

./scripts/submit_private_submission.py \
  --wallet-name <wallet-name> \
  --wallet-hotkey <wallet-hotkey-name> \
  --hotkey <miner-hotkey-ss58>

The script signs this payload with your hotkey:

tau-private-submission-v1:<hotkey>:<submission-id>:<sha256-of-agent.py>

You can also attach a display username for private submissions:

./scripts/submit_private_submission.py \
  --wallet-name <wallet-name> \
  --wallet-hotkey <wallet-hotkey-name> \
  --hotkey <miner-hotkey-ss58> \
  --agent-username <display-name>

When --agent-username is provided, the helper signs this username proof with the loaded wallet coldkey and includes the owning coldkey address:

tau-agent-submission-username:<display-name>

If your coldkey is not available to the helper, pass --coldkey and --coldkey-signature manually. The validator stores and publishes the username only when that coldkey currently owns the submitting hotkey and the signature verifies. Invalid or incomplete username proofs are ignored; they do not block an otherwise valid private submission.

Usernames are display labels, not unique account ids. Multiple hotkeys can use the same coldkey and can submit different usernames such as username, username2, and username3. Reusing a username does not spend it or reserve it globally.

The API returns JSON. If checks fail, accepted is false, the response includes ci_checks/llm_judge details, and the script exits nonzero. If accepted, no pull request or on-chain commitment is required. The response includes the private submission commitment id the validator tracks internally:

private-submission:<submission-id>:<sha256-of-agent.py>

Only one accepted submission is eligible per miner hotkey registration. After an accepted submission, that hotkey is spent for future submissions until it is freshly registered again. A second valid submission from the same hotkey in the same registration period is rejected even if it uses a different username, submission id, or agent.py hash. Other registered hotkeys controlled by the same coldkey can still submit their own bundles.

Accepted public submission metadata is visible at:

https://ninja66.ai/api/submissions

The public API does not reveal submitted agent.py contents or hotkey signatures.

Submission Guardrails

The private submission API runs these gates in order:

• Signature Gate
• Registration Gate
• Agent Smoke
• Submission Scope Guard
• OpenRouter Submission Judge

Signature Gate rejects malformed or invalid hotkey signatures before any expensive checks run.

Registration Gate confirms the signing hotkey is currently registered and has not already spent its current registration on an accepted private submission. Username labels do not change this rule; spending is tracked by registered hotkey and registration block.

Agent Smoke compiles agent.py and checks for obvious static issues.

Submission Scope Guard rejects edits that break the solve contract, add forbidden provider/sampling/secret usage, or try to bypass the validator-managed proxy.

OpenRouter Submission Judge uses the same gatekeeping judge prompt as the legacy ninja CI, run through OpenRouter with anthropic/claude-opus-4.7, temperature 0, and medium reasoning effort. It rejects poor, unsafe, cosmetic, copied, obfuscated, Goodharting, or out-of-scope edits.

Scoring Target

Validation tasks are generated from real GitHub commits. Each task starts from the repository before the mined commit, and the reference patch is used to construct and filter the task.

For duels, the scoring target is the Cursor baseline solution. The validator pre-solves each task with Cursor and the current king, then compares both king and challenger patches to that same baseline during the duel.

Round score is blended: 1/2 Cursor-baseline similarity plus 1/2 LLM diff judgment. The live diff judge uses openai/gpt-5.4 through OpenRouter at temperature 0 with medium reasoning effort and a 16000-token output cap, then scores the king and challenger patches against the task/reference context.

The challenger needs more decisive round wins than the current king. The validator may require an extra win margin in production.

Cursor is only the measuring stick. The challenger does not need to beat Cursor directly; it only needs more decisive round wins than the current king plus the configured margin.

The validator still compares king and challenger patches for copy detection, but that pairwise similarity does not replace the Cursor baseline scoring target.

When a private challenger becomes king, the validator publishes the winning agent.py into the public base harness and assigns validator weights to the winning hotkey on the next allowed weight-set epoch.