Skip to content

SETUP.md

Latest commit

 

History

History
103 lines (72 loc) · 2.11 KB

SETUP.md

File metadata and controls

103 lines (72 loc) · 2.11 KB
Raw

Quick Setup Guide

Step 1: Install LiteLLM

pip install 'litellm[proxy]'

Step 2: Set up your API key

Create a .env file in your home directory or export it:

export ANTHROPIC_API_KEY="your-api-key-here"

Step 3: Start LiteLLM Proxy

In a separate terminal, run:

litellm --model claude-sonnet-4-20250514 --api_key $ANTHROPIC_API_KEY

Or with a config file (litellm_config.yaml):

model_list:
  - model_name: claude-sonnet-4-20250514
    litellm_params:
      model: anthropic/claude-sonnet-4-20250514
      api_key: os.environ/ANTHROPIC_API_KEY

Then:

litellm --config litellm_config.yaml

The proxy should start on http://localhost:4000 by default.

Step 4: Configure the App

Edit .env.local in this directory:

LITELLM_API_URL=http://localhost:4000/v1/chat/completions
LITELLM_API_KEY=sk-1234  # Can be any string if your proxy doesn't require auth

Step 5: Run the App

npm run dev

Open http://localhost:3000 in your browser.

Troubleshooting

LiteLLM Not Running

Make sure you see output like:

INFO:     Uvicorn running on http://0.0.0.0:4000 (Press CTRL+C to quit)

API Errors

  1. Check that LiteLLM is running on port 4000
  2. Verify your ANTHROPIC_API_KEY is set correctly
  3. Check the browser console and terminal for error messages
  4. Try testing the LiteLLM endpoint directly:
curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-1234" \
  -d '{
    "model": "claude-sonnet-4-20250514",
    "messages": [{"role": "user", "content": "Say hello"}]
  }'

Port Already in Use

If port 3000 or 4000 is in use:

For the app:

PORT=3001 npm run dev

For LiteLLM:

litellm --port 4001 --model claude-sonnet-4-20250514

Then update .env.local accordingly.

Alternative: Direct Anthropic API

If you don't want to use LiteLLM, you can modify app/api/claude/route.ts to call the Anthropic API directly. The code currently converts between OpenAI and Anthropic formats to support LiteLLM.