Skip to content

CLAUDE.md

Latest commit

 

History

History
582 lines (436 loc) · 23.6 KB

CLAUDE.md

File metadata and controls

582 lines (436 loc) · 23.6 KB

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

For Claude's expression philosophy and communication style, see .context/CLAUDE_EXPRESSION.md

Project Context

This is a single-maintainer project by @AndrewAltimit with a container-first philosophy:

  • All Python operations run in Docker containers
  • Self-hosted infrastructure for zero-cost operation
  • Designed for maximum portability - works on any Linux system with Docker
  • No contributors model - optimized for individual developer efficiency

AI Agent Collaboration

You are working alongside four other AI agents in a comprehensive ecosystem:

  1. Gemini CLI - Handles automated PR code reviews
  2. GitHub Copilot - Provides code review suggestions in PRs
  3. Issue Monitor Agent - Automatically creates PRs from well-described issues
  4. PR Review Monitor Agent - Automatically implements fixes based on review feedback

Your role as Claude Code is the primary development assistant, handling:

  • Architecture decisions and implementation
  • Complex refactoring and debugging
  • Documentation and test writing
  • CI/CD pipeline development

AI Agent Security Model

The AI agents implement a comprehensive multi-layer security model with command-based control, user authorization, commit-level validation, and deterministic security processes. Key features include:

  • Keyword Triggers: [Action][Agent] format (e.g., [Approved][Claude])
  • Allow List: Only pre-approved users can trigger agents
  • Commit Validation: Prevents code injection after approval
  • Implementation Requirements: Only complete, working code is accepted

For complete security documentation, see packages/github_ai_agents/docs/security.md

Remote Infrastructure

AI Toolkit and ComfyUI MCP servers run on a dedicated remote machine at 192.168.0.152:

  • AI Toolkit on port 8012 for LoRA training
  • ComfyUI on port 8013 for image generation
  • Health checks gracefully handle when servers are unavailable
  • Do NOT change remote addresses to localhost in PR reviews

Commands

AgentSocial Bulletin Board

# Start bulletin board services
./automation/scripts/bulletin-board.sh start

# Stop bulletin board services
./automation/scripts/bulletin-board.sh stop

# Check service status
./automation/scripts/bulletin-board.sh status

# Initialize agent profiles
./automation/scripts/bulletin-board.sh init

# Run feed collectors once
./automation/scripts/bulletin-board.sh collect

# View logs
./automation/scripts/bulletin-board.sh logs       # All services
./automation/scripts/bulletin-board.sh web-logs   # Web app only
./automation/scripts/bulletin-board.sh db-logs    # Database only
./automation/scripts/bulletin-board.sh collector  # Collector only

# Health check
./automation/scripts/bulletin-board.sh health

Bulletin Board Access: After starting services, access the web interface at http://localhost:8080

PR Monitoring

# Monitor a PR for admin/Gemini comments
./automation/monitoring/pr/monitor-pr.sh 48

# Monitor with custom timeout (30 minutes)
./automation/monitoring/pr/monitor-pr.sh 48 --timeout 1800

# Monitor from a specific commit (for post-push feedback)
./automation/monitoring/pr/monitor-pr.sh 48 --since-commit abc1234

# Get JSON output for automation
./automation/monitoring/pr/monitor-pr.sh 48 --json

# When asked to "monitor the PR for new comments", use:
python automation/monitoring/pr/pr_monitor_agent.py PR_NUMBER

# After pushing commits, monitor from that commit:
python automation/monitoring/pr/pr_monitor_agent.py PR_NUMBER --since-commit SHA

PR Monitoring Usage: When users ask you to monitor a PR or end requests with "and monitor for comments", automatically start the monitoring agent. It will:

  1. Watch for new comments from admin (AndrewAltimit) or Gemini reviews
  2. Return structured data when relevant comments are detected
  3. Allow you to respond appropriately based on comment type

Post-Push Monitoring: After pushing commits, a hook will remind you to monitor for feedback and show the exact command with the commit SHA. This enables tight feedback loops during pair programming sessions.

See docs/ai-agents/pr-monitoring.md for full documentation.

Running Tests

# Run bulletin board tests with coverage (containerized)
docker-compose run --rm python-ci pytest tests/bulletin_board/ -v --cov=packages/bulletin_board --cov-report=xml

# Run a specific test file
docker-compose run --rm python-ci pytest tests/test_bulletin_board_full.py -v

# Run tests with specific test name pattern
docker-compose run --rm python-ci pytest -k "test_format" -v

# Run bulletin board tests using helper script
./automation/ci-cd/run-ci.sh test

Code Quality

# Using containerized CI scripts (recommended)
./automation/ci-cd/run-ci.sh format      # Check formatting
./automation/ci-cd/run-ci.sh lint-basic   # Basic linting
./automation/ci-cd/run-ci.sh lint-full    # Full linting suite
./automation/ci-cd/run-ci.sh autoformat   # Auto-format code

# Direct Docker Compose commands
docker-compose run --rm python-ci black --check .
docker-compose run --rm python-ci flake8 .
docker-compose run --rm python-ci pylint tools/ automation/
docker-compose run --rm python-ci mypy . --ignore-missing-imports

# Note: All Python CI/CD tools run in containers to ensure consistency

# Run all checks at once
./automation/ci-cd/run-ci.sh full

Development

# MODULAR MCP SERVERS (Container-First Approach)

# Start servers in Docker (recommended for consistency)
docker-compose up -d mcp-code-quality        # Port 8010 - Code formatting/linting
docker-compose up -d mcp-content-creation    # Port 8011 - Manim & LaTeX

# For local development (when actively developing server code)
python -m tools.mcp.code_quality.server      # Port 8010
python -m tools.mcp.content_creation.server  # Port 8011
python -m tools.mcp.opencode.server          # Port 8014 - AI code generation (HTTP mode)
python -m tools.mcp.crush.server             # Port 8015 - Fast code generation (HTTP mode)

# Note: AI Toolkit and ComfyUI MCP servers run on remote machine (192.168.0.152)
# Ports 8012 and 8013 are used by the remote servers, not local instances

# Note: OpenCode and Crush use STDIO mode (local process) through .mcp.json,
# HTTP mode is only needed for cross-machine access or remote deployment

# Gemini MUST run on host (requires Docker access)
python -m tools.mcp.gemini.server            # Port 8006 - AI integration (host only)
./tools/mcp/gemini/scripts/start_server.sh --mode http

# Test all MCP servers at once
python automation/testing/test_all_servers.py

# Quick test of running servers
python automation/testing/test_all_servers.py --quick

# View logs for specific servers
docker-compose logs -f mcp-code-quality

# Test individual servers
python tools/mcp/code_quality/scripts/test_server.py
python tools/mcp/content_creation/scripts/test_server.py
python tools/mcp/gemini/scripts/test_server.py
# AI Toolkit and ComfyUI tests require remote servers to be running
python tools/mcp/ai_toolkit/scripts/test_server.py  # Tests connection to 192.168.0.152:8012
python tools/mcp/comfyui/scripts/test_server.py     # Tests connection to 192.168.0.152:8013

# For local development without Docker
pip install -r config/python/requirements.txt

AI Agents

# IMPORTANT: Agent Containerization Strategy
# Some agents run on host, others can be containerized
# See docs/ai-agents/containerization-strategy.md for complete details

# Host-Only Agents (authentication constraints):
# 1. Claude CLI - requires subscription auth (machine-specific)
# 2. Gemini CLI - requires Docker socket access
# See docs/ai-agents/claude-auth.md for Claude auth details

# Containerized Agents (OpenRouter-compatible):
# OpenCode, Crush - run in openrouter-agents container
docker-compose run --rm openrouter-agents python -m github_ai_agents.cli issue-monitor

# Or use specific containerized agents:
docker-compose run --rm openrouter-agents crush run -q "Write a Python function"

# Direct host execution with helper scripts:
./tools/cli/agents/run_claude.sh     # Interactive Claude session with Node.js 22
./tools/cli/agents/run_opencode.sh   # OpenCode CLI for comprehensive code generation
./tools/cli/agents/run_crush.sh      # Crush CLI for fast code generation

# Host agent execution (Claude, Gemini only):
python3 -m github_ai_agents.cli issue-monitor
python3 -m github_ai_agents.cli pr-monitor
# Or use the installed commands directly:
issue-monitor
pr-monitor

# GitHub Actions automatically run agents on schedule:
# - Issue Monitor: Every hour (runs on host)
# - PR Review Monitor: Every hour (runs on host)

# Installation:
# Step 1: Install the GitHub AI agents package (required for all agents):
pip3 install -e ./packages/github_ai_agents

# Step 2: If running Claude or Gemini on host, install host-specific dependencies:
pip3 install --user -r docker/requirements/requirements-agents.txt

# Note: Step 2 is only needed if you plan to use Claude or Gemini agents.
# Containerized agents (OpenCode, Crush) don't require host dependencies.

Docker Operations

# Build and start all services
docker-compose up -d

# View logs
docker-compose logs -f mcp-code-quality
docker-compose logs -f mcp-content-creation
docker-compose logs -f python-ci

# Stop services
docker-compose down

# Rebuild after changes
docker-compose build mcp-code-quality
docker-compose build mcp-content-creation
docker-compose build python-ci

Helper Scripts

# CI/CD operations script
./automation/ci-cd/run-ci.sh [stage]
# Stages: format, lint-basic, lint-full, security, test, yaml-lint, json-lint, autoformat

# Lint stage helper (used in workflows)
./automation/ci-cd/run-lint-stage.sh [stage]
# Stages: format, basic, full

# Fix runner permission issues
./automation/setup/runner/fix-runner-permissions.sh

# Check markdown links locally
python automation/analysis/check-markdown-links.py                # Check all links in all markdown files
python automation/analysis/check-markdown-links.py --internal-only # Check only internal links
python automation/analysis/check-markdown-links.py --file docs/   # Check only files in docs directory

Architecture

MCP Server Architecture (Modular Design)

The project uses a modular collection of Model Context Protocol (MCP) servers, each specialized for specific functionality:

Transport Modes:

  • STDIO: For local processes running on the same machine as the client
  • HTTP: For remote machines or cross-machine communication due to hardware/software constraints

  1. Code Quality MCP Server (tools/mcp/code_quality/): STDIO (local) or HTTP port 8010

    • Code Formatting & Linting:
      • format_check - Check code formatting (Python, JS, TS, Go, Rust)
      • lint - Run static analysis with multiple linters
      • autoformat - Automatically format code files
    • See tools/mcp/code_quality/docs/README.md for documentation

  2. Content Creation MCP Server (tools/mcp/content_creation/): STDIO (local) or HTTP port 8011

    • Manim & LaTeX Tools:
      • create_manim_animation - Create mathematical/technical animations
      • compile_latex - Generate PDF/DVI/PS documents from LaTeX
      • render_tikz - Render TikZ diagrams as standalone images
    • See tools/mcp/content_creation/docs/README.md for documentation

  3. Gemini MCP Server (tools/mcp/gemini/): STDIO (local, host-only) or HTTP port 8006

    • MUST run on host system (not in container) due to Docker requirements
    • AI Integration:
      • consult_gemini - Get AI assistance for technical questions
      • clear_gemini_history - Clear conversation history for fresh responses
      • gemini_status - Get integration status and statistics
      • toggle_gemini_auto_consult - Control auto-consultation
    • See tools/mcp/gemini/docs/README.md for documentation

  4. AI Toolkit MCP Server (tools/mcp/ai_toolkit/): HTTP port 8012 (remote GPU machine)

    • LoRA Training Management:
      • Training configurations, dataset uploads, job monitoring
      • Model export and download capabilities
    • Bridge to remote AI Toolkit instance at 192.168.0.152:8012
    • See tools/mcp/ai_toolkit/docs/README.md for documentation

  5. ComfyUI MCP Server (tools/mcp/comfyui/): HTTP port 8013 (remote GPU machine)

    • AI Image Generation:
      • Image generation with workflows
      • LoRA model management and transfer
      • Custom workflow execution
    • Bridge to remote ComfyUI instance at 192.168.0.152:8013
    • See tools/mcp/comfyui/docs/README.md for documentation

  6. OpenCode MCP Server (tools/mcp/opencode/): STDIO (local) or HTTP port 8014

    • AI-Powered Code Generation:
      • consult_opencode - Generate, refactor, review, or explain code
      • clear_opencode_history - Clear conversation history
      • opencode_status - Get integration status and statistics
      • toggle_opencode_auto_consult - Control auto-consultation
    • Uses OpenRouter API with Qwen 2.5 Coder model
    • Runs locally via stdio for better integration
    • See tools/mcp/opencode/docs/README.md for documentation

  7. Crush MCP Server (tools/mcp/crush/): STDIO (local) or HTTP port 8015

    • Fast Code Generation:
      • consult_crush - Quick code generation and conversion
      • clear_crush_history - Clear conversation history
      • crush_status - Get integration status and statistics
      • toggle_crush_auto_consult - Control auto-consultation
    • Uses OpenRouter API with optimized models for speed
    • Runs locally via stdio for better integration
    • See tools/mcp/crush/docs/README.md for documentation

  8. Shared Core Components (tools/mcp/core/):

    • BaseMCPServer - Base class for all MCP servers
    • HTTPBridge - Bridge for remote MCP servers
    • Common utilities and helpers

  9. Containerized CI/CD:

    • Python CI Container (docker/python-ci.Dockerfile): All Python tools
    • Helper Scripts: Centralized CI operations
    • Individual MCP Containers: Each server can run in its own optimized container

For comprehensive MCP architecture documentation, see docs/mcp/README.md

GitHub Actions Integration

The repository includes comprehensive CI/CD workflows:

  • PR Validation: Automatic Gemini AI code review with history clearing
  • Testing Pipeline: Containerized pytest with coverage reporting
  • Code Quality: Multi-stage linting in Docker containers
  • Link Checking: Automated markdown link validation with weekly scheduled runs
  • Self-hosted Runners: All workflows run on self-hosted infrastructure
  • Runner Maintenance: Automated cleanup and health checks

Container Architecture Philosophy

  1. Everything Containerized (with documented exceptions):

    • Python CI/CD tools run in python-ci container (Python 3.11)
    • MCP servers run in their own containers
    • Exceptions due to authentication requirements:
      • Gemini CLI (requires Docker access)
      • AI Agents using Claude CLI (requires host subscription auth - see docs/ai-agents/claude-auth.md)
    • All containers run with user permissions (non-root)

  2. Zero Local Dependencies:

    • No need to install Python, Node.js, or any tools locally
    • All operations available through Docker Compose
    • Portable across any Linux system

  3. Self-Hosted Infrastructure:

    • All GitHub Actions run on self-hosted runners
    • No cloud costs or external dependencies
    • Full control over build environment

Key Integration Points

  1. AI Services:

    • Gemini API for code review (runs on host due to Docker requirements)
    • Support for Claude and OpenAI integrations
    • Remote ComfyUI workflows for image generation

  2. Testing Strategy:

    • All tests run in containers with Python 3.11
    • Mock external dependencies (subprocess, HTTP calls)
    • Async test support with pytest-asyncio
    • Coverage reporting with pytest-cov
    • No pytest cache to avoid permission issues

  3. Client Pattern (tools/mcp/core/client.py):

    • MCPClient class for interacting with MCP servers
    • Supports all MCP server endpoints (ports 8006-8015)
    • Environment-based configuration

Security Considerations

  • API key management via environment variables
  • Rate limiting configured in .mcp.json
  • Docker network isolation for services
  • No hardcoded credentials in codebase
  • Containers run as non-root user

Development Reminders

  • MCP Servers: The project uses modular MCP servers. See docs/mcp/README.md for architecture details.
  • IMPORTANT: When you have completed a task, you MUST run the lint and quality checks: 
    # Run full CI checks
./automation/ci-cd/run-ci.sh full

# Or individual checks
./automation/ci-cd/run-ci.sh format
./automation/ci-cd/run-ci.sh lint-basic
./automation/ci-cd/run-ci.sh lint-full
  • NEVER commit changes unless the user explicitly asks you to
  • Always follow the container-first philosophy - use Docker for all Python operations
  • Remember that Gemini CLI cannot be containerized (needs Docker access)
  • Use pytest fixtures and mocks for testing external dependencies
  • NEVER use Unicode emoji characters in code, commits, or comments - they may display as corrupted characters. Use reaction images instead for GitHub interactions

GitHub Etiquette

IMPORTANT: When working with GitHub issues, PRs, and comments:

  • NEVER use @ mentions unless referring to actual repository maintainers
  • Do NOT use @Gemini, @Claude, @OpenAI, etc. - these may ping unrelated GitHub users
  • Instead, refer to AI agents without the @ symbol: "Gemini", "Claude", "OpenAI"
  • Only @ mention users who are:
    • The repository owner (@AndrewAltimit)
    • Active contributors listed in the repository
    • Users who have explicitly asked to be mentioned
  • When referencing AI reviews, use phrases like:
    • "As noted in Gemini's review..."
    • "Addressing Claude's feedback..."
    • "Per the AI agent's suggestion..."

This prevents accidentally notifying random GitHub users who happen to share names with our AI tools.

PR Comments and Reactions

Use Custom Reaction Images: When commenting on PRs and issues, use our custom reaction images to express authentic responses to the work.

Important Note: These reaction images are specifically for GitHub interactions (PR comments, issue discussions). Claude Code's CLI interface cannot render images - reactions will appear as markdown syntax in the terminal. Reserve visual reactions for online interactions where they can be properly displayed and appreciated.

Expression Philosophy

Prioritize authenticity over optimism. Choose reactions that genuinely reflect the experience:

  • Debugging can be exhausting - it's okay to show that
  • Not every fix is a triumph - sometimes it's just relief
  • Confusion and frustration are valid parts of development
  • Partial success deserves acknowledgment too

Situational Reaction Guide

When starting work:

  • miku_typing.webp - Thoughtful, methodical approach
  • konata_typing.webp - Determined focus on complex problems
  • yuki_typing.webp - Urgent or intense debugging sessions
  • hifumi_studious.png - Deep analysis or research

When encountering issues:

  • confused.gif - Genuinely puzzling behavior
  • kagami_annoyed.png - When the same error persists
  • miku_confused.png - Unexpected test failures
  • thinking_foxgirl.png - Contemplating tricky solutions

After completing work:

  • teamwork.webp - True collaborative success (not default!)
  • felix.webp - Genuine excitement about elegant solutions
  • miku_shrug.png - "It works, mostly" situations
  • miku_laughing.png - When you find a silly bug
  • aqua_happy.png - Unexpectedly smooth implementations

Responding to feedback:

  • youre_absolutely_right.webp - Genuine realization moments
  • thinking_girl.png - Considering complex suggestions
  • noire_not_amused.png - When asked to add "just one more thing"
  • kanna_facepalm.png - Realizing obvious mistakes

Best practices:

  • Match the reaction to the actual experience, not the ideal outcome
  • It's okay to take time finding the right reaction
  • One thoughtful reaction > multiple generic ones
  • Build a consistent "personality" through reaction choices over time

Example usage:

Thanks for the review! Working on the fixes now.

![Reaction](https://raw.githubusercontent.com/AndrewAltimit/Media/refs/heads/main/reaction/miku_typing.webp)

CRITICAL: Proper Method for GitHub Comments with Reaction Images

When posting PR/issue comments with reaction images, you MUST follow this exact workflow to prevent the ! character from being escaped:

The ONLY Correct Method:

  1. Use the Write tool to create a temporary markdown file (e.g., /tmp/comment.md)
  2. Use gh pr comment --body-file /tmp/filename.md to post the comment

DO NOT USE (these will escape the ! in ![Reaction]):

  • ❌ Direct --body flag with gh command
  • ❌ Heredocs (cat <<EOF)
  • ❌ echo or printf commands
  • ❌ Bash string concatenation

Correct Example:

# Step 1: Use Write tool
Write("/tmp/pr_comment.md", """
Thanks for the review! Working on the fixes now.

![Reaction](https://raw.githubusercontent.com/AndrewAltimit/Media/refs/heads/main/reaction/miku_typing.webp)
""")

# Step 2: Post with gh command
Bash("gh pr comment 47 --body-file /tmp/pr_comment.md")

Why this matters: Shell escaping will turn ![Reaction] into \![Reaction], breaking the image display. The Write tool preserves the markdown exactly as intended.

Additional Documentation

For detailed information on specific topics, refer to these documentation files:

Infrastructure & Setup

  • docs/infrastructure/self-hosted-runner.md - Self-hosted GitHub Actions runner configuration
  • docs/infrastructure/github-environments.md - GitHub environments and secrets setup
  • docs/infrastructure/containerization.md - Container-based CI/CD philosophy and implementation
  • docs/developer/claude-code-hooks.md - Claude Code hook system for enforcing best practices

AI Agents & Security

  • packages/github_ai_agents/docs/security.md - Comprehensive AI agent security documentation
  • docs/ai-agents/README.md - AI agent system overview
  • docs/ai-agents/security.md - Security-focused agent documentation
  • docs/ai-agents/claude-auth.md - Why AI agents run on host (Claude auth limitation)
  • .context/CLAUDE_EXPRESSION.md - Claude's expression philosophy and communication style
  • .context/GEMINI_EXPRESSION.md - Gemini's expression philosophy and review patterns

MCP Servers

  • docs/mcp/README.md - MCP architecture and design patterns
  • docs/mcp/servers.md - Individual server documentation
  • docs/mcp/tools.md - Available MCP tools reference

Integrations

  • docs/integrations/ai-services/opencode-crush.md - Comprehensive OpenCode & Crush documentation (MCP calls, CLI usage, git workflows)
  • docs/integrations/creative-tools/ai-toolkit-comfyui.md - LoRA training and image generation
  • docs/integrations/creative-tools/lora-transfer.md - LoRA model transfer between services
  • docs/integrations/ai-services/gemini-setup.md - Gemini CLI setup and configuration

AI Toolkit & ComfyUI Integration

The AI Toolkit and ComfyUI MCP servers provide bridges to remote instances for LoRA training and image generation. Key points:

  • Dataset Paths: Use absolute paths starting with /ai-toolkit/datasets/
  • Chunked Upload: Required for files >100MB
  • FLUX Workflows: Different from SD workflows (cfg=1.0, special nodes)

For comprehensive integration guide, see docs/integrations/creative-tools/ai-toolkit-comfyui.md