Feat/local docker sandbox

.gitignore

@@ -189,14 +189,13 @@ dist
 agent_logs.txt
 workspace/
 tmp/
-data/file_store
-data/workspace
-data/logs
-data/events.db
+data/
 output/
 
 .vscode/
 .envrc
 
 # local only scripts
 start_tool_server.sh
+docker/.stack.env.local
+scripts/local/

README.md

@@ -29,19 +29,157 @@ II-Agent Chat also feature within II-Agent that lets you work across multiple mo
 <https://github.com/user-attachments/assets/02ef0612-1696-4311-9002-14f36842ff3e>
 
 
-## Key Features:
-* Full-Stack Development: Complete web app scaffolding and iterative development. From initial setup to deployment, II-Agent handles the entire development lifecycle with intelligent code generation and optimization.
-* Slide Creation: Transform short briefs into polished presentations. Create professional slides and decks with intelligent content structuring, design suggestions, and automated formatting.
-* Deep Research: Comprehensive research capabilities through tight integration with II-Researcher. Conduct thorough investigations, analyze data, and generate detailed reports with our specialized research agent.
+## Key Features
+
+* **Full-Stack Development**: Complete web app scaffolding and iterative development. From initial setup to deployment, II-Agent handles the entire development lifecycle with intelligent code generation and optimization.
+* **Slide Creation**: Transform short briefs into polished presentations. Create professional slides and decks with intelligent content structuring, design suggestions, and automated formatting.
+* **Deep Research**: Comprehensive research capabilities through tight integration with II-Researcher. Conduct thorough investigations, analyze data, and generate detailed reports with our specialized research agent.
+* **Local Docker Sandbox**: Run sandboxes entirely on your own machine with Docker — no cloud dependencies, no data leaving your network. Ideal for air-gapped, NDA-protected, or self-hosted environments.
+* **Agent-Human Handoff**: When the agent encounters CAPTCHAs, login forms, or 2FA, it opens a browser-based noVNC session so you can interact directly, then resumes automation once you're done.
+* **Media Generation**: Built-in DALL-E 3 image generation and Sora video generation with cost tracking and multiple aspect ratio support.
+* **Extended Thinking**: Claude 4 extended thinking with configurable token budgets, interleaved thinking during tool use, and optional 1M context window.
+* **Tool Execution Safety**: Tiered timeout system (120s tool-level, 300s MCP backstop) with 2-second interrupt polling — prevents indefinitely hung sessions and supports mid-execution cancellation.
 
 ## SWE-Bench Pro
 
 <img width="1778" height="1060" alt="swepro" src="https://github.com/user-attachments/assets/e955538d-986a-4c74-96b9-dbeb56e803e1" />
 
 
+## Deployment Models
+
+II-Agent supports two sandbox deployment modes through a pluggable provider architecture:
+
+| | Cloud (E2B) | Local (Docker) |
+|---|---|---|
+| **Isolation** | Firecracker micro-VMs | Docker containers |
+| **Network** | Public (ngrok tunnel) | Localhost only |
+| **Startup** | ~150ms (pre-warmed) | 2–5s (cold start) |
+| **Data location** | E2B infrastructure | Your machine |
+| **Cost** | Per-use billing | Free (your hardware) |
+| **Best for** | Production, quick start | Privacy, air-gapped, self-hosted |
+
+Set `SANDBOX_PROVIDER=docker` or `SANDBOX_PROVIDER=e2b` in your environment to choose.
+
 
 ## Installation
 
 For the latest installation and deployment instructions, please refer to our [official guide](https://intelligent-internet.github.io/ii-agent-prod/)
 
 [![Installation Guide](https://img.youtube.com/vi/wPpeJMbdGi4/maxresdefault.jpg)](https://www.youtube.com/watch?v=wPpeJMbdGi4)
+
+### Local Docker Quick Start
+
+Run II-Agent entirely on your own machine with no cloud dependencies:
+
+```bash
+# 1. Build the sandbox image (Python, Node.js, Playwright, noVNC, code-server)
+docker build -t ii-agent-sandbox:latest -f e2b.Dockerfile .
+
+# 2. Configure environment
+cp docker/.stack.env.local.example docker/.stack.env.local
+# Edit docker/.stack.env.local — set JWT_SECRET_KEY and at least one LLM API key
+
+# 3. Start the stack
+scripts/stack_control.sh start --local
+
+# 4. Access
+#   Frontend:       http://localhost:1420
+#   Backend API:    http://localhost:8000
+#   Sandbox Server: http://localhost:8100
+```
+
+For detailed setup, see [docs/docs/local-docker-sandbox.md](docs/docs/local-docker-sandbox.md).
+
+### Architecture (Local Mode)
+
+```
+┌─────────────┐
+│   Frontend   │
+│   (:1420)    │
+└──────┬───────┘
+       │ WebSocket
+       ▼
+┌─────────────┐     ┌───────────┐     ┌───────────┐
+│   Backend    │◄───►│   Redis   │     │ Postgres  │
+│   (:8000)    │     │  (:6379)  │     │  (:5433)  │
+└──────┬───────┘     └───────────┘     └───────────┘
+       │
+  ┌────┴─────┐
+  ▼          ▼
+┌──────────┐ ┌─────────────┐
+│ Sandbox  │ │ Tool Server │
+│  Server  │ │   (:1236)   │
+│ (:8100)  │ └─────────────┘
+└────┬─────┘
+     │ Docker API
+     ▼
+┌──────────────────────────────────┐
+│    Sandbox Containers            │
+│  ┌──────────┐  ┌──────────┐     │
+│  │ Sandbox1 │  │ Sandbox2 │ ... │
+│  │ Playwright│  │ noVNC   │     │
+│  │ noVNC    │  │ code-svr │     │
+│  └──────────┘  └──────────┘     │
+└──────────────────────────────────┘
+```
+
+Each sandbox container gets 6 host ports allocated from a configurable pool (default 30000–30999), exposing noVNC (6080), code-server (9000), MCP (6060), and dev-server ports.
+
+
+## Stack Management
+
+The unified `scripts/stack_control.sh` script manages the full Docker stack:
+
+```bash
+scripts/stack_control.sh start [--local]              # Start services
+scripts/stack_control.sh stop [--local]               # Stop services
+scripts/stack_control.sh restart [--local] [service]  # Restart (picks up env changes)
+scripts/stack_control.sh rebuild [--local] [service]  # Rebuild image + restart
+scripts/stack_control.sh status [--local]             # Show service status and URLs
+scripts/stack_control.sh logs [--local] [service] [-f]# View/follow logs
+scripts/stack_control.sh build [--local]              # Build sandbox image
+scripts/stack_control.sh setup [--local]              # Create .stack.env from template
+scripts/stack_control.sh wake [--local] [uuid]        # Wake stopped sandbox containers
+```
+
+The `--local` flag switches between cloud (E2B) and local (Docker) compose configurations.
+
+
+## Agent-Human-Agent Handoff
+
+When the agent's browser encounters a CAPTCHA, login form, or 2FA challenge:
+
+1. The agent calls `expose_port(6080)` to obtain a noVNC URL
+2. The agent shares the URL and pauses
+3. You open the noVNC link in your browser and interact with the sandbox's Chromium directly
+4. You confirm completion via the chat UI
+5. The agent resumes automation from a fresh screenshot
+
+This works because both Playwright (agent) and noVNC (you) share the same X11 display (`:99`) inside the sandbox via `x11vnc --shared`.
+
+
+## Configuration
+
+### Key Environment Variables
+
+| Variable | Default | Description |
+|---|---|---|
+| `SANDBOX_PROVIDER` | `e2b` | Sandbox backend: `e2b` or `docker` |
+| `SANDBOX_DOCKER_IMAGE` | `ii-agent-sandbox:latest` | Docker image for local sandboxes |
+| `SANDBOX_PORT_RANGE_START` | `30000` | Start of host port pool |
+| `SANDBOX_PORT_RANGE_END` | `30999` | End of host port pool |
+| `LOCAL_MODE` | `false` | Enable local-only features |
+| `STORAGE_PROVIDER` | `gcs` | Storage backend: `gcs` or `local` |
+| `ORPHAN_CLEANUP_ENABLED` | `true` | Auto-remove sandboxes with no active sessions |
+| `ORPHAN_CLEANUP_INTERVAL_SECONDS` | `300` | Cleanup check interval |
+
+See `docker/.stack.env.local.example` for the full list of configurable variables.
+
+
+## Utility Scripts
+
+| Script | Description |
+|---|---|
+| `scripts/stack_control.sh` | Unified Docker stack lifecycle management |
+| `scripts/admin_credits.sh` | Query and manage user credits in PostgreSQL |
+| `scripts/html_to_pdf.py` | Convert HTML slides/pages to multi-page PDF via Playwright |

docker/.stack.env.local.example

@@ -0,0 +1,154 @@
+# ============================================================================
+# ii-agent Local-Only Environment Configuration
+# ============================================================================
+# This configuration is for running ii-agent with LOCAL Docker sandboxes
+# instead of E2B cloud. All data stays on your machine - suitable for
+# privileged/NDA-protected data.
+#
+# Copy this file to .stack.env.local and configure the required values.
+# ============================================================================
+
+# ============================================================================
+# SANDBOX PROVIDER (NEW - Docker instead of E2B)
+# ============================================================================
+# Use "docker" for local sandboxes or "e2b" for E2B cloud
+SANDBOX_PROVIDER=docker
+
+# Docker image to use for local sandboxes (build with: docker build -t ii-agent-sandbox:latest -f e2b.Dockerfile .)
+SANDBOX_DOCKER_IMAGE=ii-agent-sandbox:latest
+
+# Optional: Docker network for sandboxes to join (useful if MCP server is in a container)
+# SANDBOX_DOCKER_NETWORK=ii-agent-network
+
+# ============================================================================
+# DATABASE CONFIGURATION
+# ============================================================================
+# Use a different port if native PostgreSQL is running on 5432
+POSTGRES_PORT=5433
+POSTGRES_USER=iiagent
+POSTGRES_PASSWORD=iiagent
+POSTGRES_DB=iiagentdev
+
+# Database URLs for services (using internal docker hostname)
+DATABASE_URL=postgresql://iiagent:iiagent@postgres:5432/iiagentdev
+
+# Sandbox server database
+SANDBOX_DB_NAME=ii_sandbox
+SANDBOX_DATABASE_URL=postgresql+asyncpg://iiagent:iiagent@postgres:5432/ii_sandbox
+
+# ============================================================================
+# REDIS CONFIGURATION
+# ============================================================================
+REDIS_PORT=6379
+REDIS_URL=redis://redis:6379/0
+REDIS_SESSION_URL=redis://redis:6379/1
+
+# ============================================================================
+# SERVICE PORTS
+# ============================================================================
+FRONTEND_PORT=1420
+BACKEND_PORT=8000
+TOOL_SERVER_PORT=1236
+SANDBOX_SERVER_PORT=8100
+
+# Port for MCP server inside sandboxes
+MCP_PORT=6060
+
+# ============================================================================
+# FRONTEND CONFIGURATION
+# ============================================================================
+FRONTEND_BUILD_MODE=production
+
+# API URL that the frontend uses to reach the backend.
+# IMPORTANT: For mobile/remote device access, use your machine's IP address
+# (e.g., http://192.168.x.x:8000) instead of localhost.
+# This MUST match LOCAL_STORAGE_URL_BASE below (same host) for file uploads to work.
+VITE_API_URL=http://localhost:8000
+
+# Disable Google OAuth for local setup (optional - set to enable)
+VITE_GOOGLE_CLIENT_ID=
+
+# Disable Stripe for local setup
+VITE_STRIPE_PUBLISHABLE_KEY=
+
+# Disable Sentry for local setup
+VITE_SENTRY_DSN=
+
+# ============================================================================
+# AUTHENTICATION (Required)
+# ============================================================================
+# Generate with: openssl rand -hex 32
+JWT_SECRET_KEY=CHANGE_ME_USE_openssl_rand_hex_32
+
+# For local-only mode, you can use the demo user
+# Enable demo mode to skip OAuth
+DEMO_MODE=true
+
+# ============================================================================
+# LLM PROVIDER API KEYS (At least one required)
+# ============================================================================
+# OpenAI
+OPENAI_API_KEY=
+
+# Anthropic Claude
+ANTHROPIC_API_KEY=
+
+# Google Gemini
+GEMINI_API_KEY=
+
+# Groq
+GROQ_API_KEY=
+
+# Fireworks
+FIREWORKS_API_KEY=
+
+# OpenRouter (access to multiple models)
+OPENROUTER_API_KEY=
+
+# ============================================================================
+# MCP SERVER CONFIGURATION (Optional - for your local MCP server)
+# ============================================================================
+# If you have a local MCP server running, configure it here
+# This URL is accessible from within sandbox containers
+
+# For MCP server running on host machine:
+# MCP_SERVER_URL=http://host.docker.internal:6060
+
+# For MCP server running in a Docker container on the same network:
+# MCP_SERVER_URL=http://mcp-server:6060
+
+# ============================================================================
+# OPTIONAL SERVICES
+# ============================================================================
+# These are not required for local-only mode
+
+# ============================================================================
+# LOCAL FILE STORAGE (for uploads and assets)
+# ============================================================================
+# URL base for serving uploaded files to browsers.
+# IMPORTANT: This MUST use the same host as VITE_API_URL above.
+# - Use localhost for local-only access
+# - Use your machine's IP (e.g., http://192.168.x.x:8000/files) for mobile/remote access
+# If mismatched, file uploads will fail on mobile devices because the browser
+# tries to upload to a URL it can't reach.
+LOCAL_STORAGE_URL_BASE=http://localhost:8000/files
+
+# Internal URL for container-to-container file access (usually doesn't need changing)
+LOCAL_STORAGE_INTERNAL_URL_BASE=http://backend:8000/files
+
+# Image search (Serper)
+# SERPER_API_KEY=
+
+# Web search (Tavily)
+# TAVILY_API_KEY=
+
+# Cloud storage (not needed for local mode)
+# GCS_BUCKET_NAME=
+# GOOGLE_APPLICATION_CREDENTIALS=
+
+# ============================================================================
+# E2B CONFIGURATION (NOT NEEDED for local Docker mode)
+# ============================================================================
+# Leave these empty when using SANDBOX_PROVIDER=docker
+# E2B_API_KEY=
+# NGROK_AUTHTOKEN=

docker/backend/Dockerfile

@@ -30,7 +30,7 @@ RUN fc-cache -fv
 RUN --mount=type=cache,target=/root/.cache/uv \
     --mount=type=bind,source=uv.lock,target=uv.lock \
     --mount=type=bind,source=pyproject.toml,target=pyproject.toml \
-    uv sync --locked --no-install-project --no-dev
+    uv sync --locked --prerelease=allow --no-install-project --no-dev
 
 # Install Playwright in a single layer
 RUN uv run playwright install --with-deps chromium
@@ -39,7 +39,7 @@ RUN uv run playwright install --with-deps chromium
 # Installing separately from its dependencies allows optimal layer caching
 COPY . /app
 RUN --mount=type=cache,target=/root/.cache/uv \
-    uv sync --locked --no-dev
+    uv sync --locked --prerelease=allow --no-dev
 
 RUN chmod +x /app/start.sh
 RUN chmod +x /app/scripts/run_sandbox_timeout_extension.sh