Update Dockerfile for uv and pyproject.toml migration - #260
Merged
Conversation
- Replace setup.py with pyproject.toml - Use uv instead of pip for faster, more reliable package management - Install all optional dependencies [generative,v1] - Remove redundant manual pip installs (now in pyproject.toml extras) - Update workflow to trigger on pyproject.toml changes - Enable workflow_dispatch for manual builds Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 169b955f0d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The rewardbench-gen entry point uses rewardbench/generative.py which imports google.generativeai (old SDK), not google.genai (new SDK). Both packages are needed for full compatibility. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update rewardbench/generative.py to use the modern google-genai SDK instead of the legacy google-generativeai. This aligns it with generative_v2.py and simplifies dependencies. Move google-generativeai to [v1] extra for any legacy code that still needs the old SDK. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This reverts commit 4d7e495.
- Pin torch>=2.1,<=2.8 in pyproject.toml for compatibility with prebuilt flash-attn wheels - Add torchvision and torchaudio to dependencies - Remove separate torch install from Dockerfile (now in pyproject.toml) - Use prebuilt flash-attn 2.8.3 wheel from GitHub releases instead of building from source (significantly faster Docker builds) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
vllm 0.13.0 requires torch==2.9.0, but flash-attn prebuilt wheels only exist up to torch 2.8. Since we need vllm 0.13.0, we must build flash-attn from source (slower Docker builds). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
flash-attn cannot be installed because: 1. vllm 0.13.0 requires torch 2.9, but prebuilt wheels only exist up to torch 2.8 2. Building from source requires excessive memory and is very slow vllm has its own optimized attention kernels, so this is fine for inference. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
GitHub Actions has sufficient memory to build flash-attn from source. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
vllm is CUDA-only and doesn't support macOS. Adding sys_platform == 'linux' marker allows Mac users to run uv sync --all-extras and still get all the API-based providers (openai, anthropic, google-genai, together). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Pin vllm==0.13.0 on all Linux platforms (prevents torch version floating) - Add MAX_JOBS=4 to flash-attn build to limit parallelism and avoid OOM Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
GitHub Actions has sufficient memory; MAX_JOBS was only needed locally. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Copy dependency manifests (pyproject.toml, uv.lock) first for caching - Use uv sync --frozen --no-install-project to install deps (cached layer) - Copy source code after deps (only invalidates later layers) - Use uv sync --frozen --no-editable for deployment-style install - flash-attn installed last after torch is finalized Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Move flash-attn install BEFORE source code copy so it's cached unless dependencies change (not on every code change) - Add docker/build-push-action with GHA layer caching (cache-from/to) - This should dramatically speed up subsequent builds Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Base image (Dockerfile): - Uses api + v1 extras (no vllm) - Pins torch <=2.8 - Uses prebuilt flash-attn wheel (~5-10 min build) - For: reward models, API-based judges (OpenAI, Anthropic, etc.) vLLM image (Dockerfile.vllm): - Uses generative + v1 extras (includes vllm) - vllm pins torch 2.9 - Builds flash-attn from source (~45 min build) - For: local LLM inference judges Also split extras in pyproject.toml: - api: API clients only (openai, anthropic, google-genai, together) - vllm: vllm only (pins torch) - generative: api + vllm (backwards compatible) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Document api, vllm, generative extras in CLAUDE.md - Add Docker images table to both CLAUDE.md and README.md - Update build instructions for both images Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add --system flag to uv pip install commands in Dockerfiles (required when no venv is present in Docker) - Fix black formatting in rewardbench.py Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
vLLM 0.13.0 includes its own flash attention implementation, so we do not need to build flash-attn from source. This saves ~30 minutes of build time. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- uv sync uses lockfile which has torch 2.9, so we need to downgrade after - Update flash-attn to v2.8.3 which has prebuilt wheels for torch 2.8 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- openjdk: no Java code in project - git-lfs: hf_transfer downloads models via HTTP - parallel: code uses Python threads, not GNU parallel Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Caching adds ~5-10 min upload overhead per build. With infrequent builds and fast base image (prebuilt flash-attn), caching overhead isn't worth it. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan
Generated with Claude Code