Skip to content

developing.md

Latest commit

 

History

History
171 lines (127 loc) · 4.69 KB

developing.md

File metadata and controls

171 lines (127 loc) · 4.69 KB

Development Guide

This guide covers setting up your development environment, running tests, and contributing code to the Skill Scanner.

Prerequisites

  • Python 3.10+ - Required for running the project
  • Git - For version control
  • uv - Fast Python package manager (installation instructions below)

Environment Setup

1. Clone the Repository

git clone https://github.com/cisco-ai-defense/skill-scanner
cd skill-scanner

2. Install uv

uv is our recommended package manager for fast, reliable dependency management.

# macOS/Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows (PowerShell)
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

3. Install Dependencies

# Install all dependencies including dev extras
uv sync --all-extras

4. Install Pre-commit Hooks

uv run pre-commit install

This ensures code quality checks run automatically before each commit.

5. Verify Setup

# Run tests to verify everything works
uv run pytest tests/ -q

# Run linting
uv run pre-commit run --all-files

Development Workflow

Running Tests

# Run all tests
uv run pytest tests/ -v --tb=short

# Run specific test file
uv run pytest tests/test_scanner.py -v

# Run specific test
uv run pytest tests/test_scanner.py::test_scan_safe_skill -v

# Run with coverage report
uv run pytest tests/ -v --tb=short --cov=skill_scanner --cov-report=html

For detailed testing requirements, see TESTING.md.

Code Quality

All checks run via pre-commit:

uv run pre-commit run --all-files

This runs:

  • ruff: Linting and formatting
  • pre-commit-hooks: Whitespace, file, and config hygiene checks
  • gitleaks: Secret detection
  • addlicense: Apache 2.0 license headers
  • check-taxonomy: Validates taxonomy enum parity with Cisco taxonomy profile

Run mypy separately:

uv run mypy skill_scanner

Before Submitting a PR

  1. Ensure all pre-commit hooks pass
  2. Add/update tests for your changes
  3. Run the full test suite
  4. Update documentation if needed
  5. Follow commit message conventions

Project Structure

skill_scanner/
├── __init__.py
├── api/               # FastAPI REST endpoints
├── cli/               # argparse-based CLI commands and policy TUI
├── config/            # Configuration and constants
├── core/
│   ├── analyzers/     # Security analyzers (static, bytecode, pipeline, behavioral, LLM)
│   ├── reporters/     # Output formatters (JSON, SARIF, Markdown)
│   ├── rules/         # Rule loaders (patterns.py, yara_scanner.py)
│   ├── rule_registry.py  # Centralized rule registry and pack loader
│   ├── static_analysis/  # AST parsing and dataflow analysis
│   ├── loader.py      # Skill package loader
│   ├── models.py      # Data models
│   ├── scan_policy.py # Policy engine (single source of truth for all knobs)
│   └── scanner.py     # Main scanner orchestrator
├── data/
│   ├── packs/
│   │   └── core/
│   │       ├── pack.yaml       # Rule pack manifest
│   │       ├── signatures/     # YAML regex detection rules
│   │       ├── yara/           # YARA detection rules
│   │       └── python/         # Python check modules
│   ├── prompts/                # LLM analysis prompts
│   ├── default_policy.yaml     # Balanced policy preset
│   ├── strict_policy.yaml      # Strict policy preset
│   └── permissive_policy.yaml  # Permissive policy preset
├── hooks/             # Pre-commit hooks
├── threats/           # Threat taxonomy
└── utils/             # Shared utilities
tests/
├── conftest.py        # Shared fixtures
└── test_*.py          # Test files
evals/
├── runners/           # Benchmark and eval runners
├── policies/          # Policy presets for benchmarking
└── skills/            # Evaluation skill samples

Running Individual Analyzers

# Core analyzers only (default: static + bytecode + pipeline)
skill-scanner scan /path/to/skill

# With behavioral analysis
skill-scanner scan /path/to/skill --use-behavioral

# With LLM analysis (requires API key)
skill-scanner scan /path/to/skill --use-llm

# With trigger specificity analysis
skill-scanner scan /path/to/skill --use-trigger

# All analyzers
skill-scanner scan /path/to/skill --use-behavioral --use-llm --use-virustotal

# Cross-skill overlap analysis
skill-scanner scan-all /path/to/skills --check-overlap

Versioning

This project follows Semantic Versioning.