Skip to content

feat: add Solidity smart contract security audit environment#398

Open
0xbyt4 wants to merge 1 commit intoNousResearch:mainfrom
0xbyt4:feature/solidity-audit-env
Open

feat: add Solidity smart contract security audit environment#398
0xbyt4 wants to merge 1 commit intoNousResearch:mainfrom
0xbyt4:feature/solidity-audit-env

Conversation

@0xbyt4
Copy link
Contributor

@0xbyt4 0xbyt4 commented Feb 26, 2026

Summary

  • New community environment that trains LLMs to detect security vulnerabilities in Solidity smart contracts
  • Uses darkknight25/Smart_Contract_Vulnerability_Dataset (~2,000 labeled code snippets, MIT license)
  • Multi-component reward function: vulnerability detection (0.25), category match with fuzzy matching (0.35), description quality via Jaccard similarity (0.25), format compliance (0.15)
  • Follows existing patterns from sql_query_env and cybersecurity_sigma environments

Files

File Purpose
solidity_audit_env.py Main environment (BaseEnv subclass)
scoring.py Reward function helpers
dataset_loader.py HuggingFace dataset loading with robust JSONL parsing
test_scoring.py 46 unit tests for scoring logic
requirements.txt Dependencies (datasets, pyyaml)
README.md Documentation and usage instructions

Test plan

  • 46/46 unit tests passing (pytest test_scoring.py -v)
  • black, ruff, flake8 all clean
  • Import check successful (from solidity_audit_env import SolidityAuditEnv)
  • Process mode: dataset loads successfully (1998/2000 entries, 2 malformed rows skipped), API calls execute correctly (verified with Gemini API)

New community environment that trains LLMs to detect security
vulnerabilities in Solidity smart contracts using the
darkknight25/Smart_Contract_Vulnerability_Dataset from HuggingFace.

- Multi-component reward: vuln detection (0.25), category match (0.35),
  description quality (0.25), format compliance (0.15)
- Robust JSONL loader that handles malformed rows in upstream dataset
- 46 unit tests for all scoring functions
- Follows sql_query_env and cybersecurity_sigma patterns
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant