Skip to content

chore: make uv.lock the real install path - #518

Merged
smaramwbc merged 1 commit into
mainfrom
chore/adopt-uv-as-install-path
Oct 9, 2026
Merged

smaramwbc merged 1 commit into
mainfrom
chore/adopt-uv-as-install-path

Conversation

@smaramwbc

Copy link
Copy Markdown
Owner

Replaces #517, which deleted the lockfile. That was the wrong fix, for a reason worth writing down.

What was actually wrong

uv.lock was installed by nothing. CI ran pip install -e ".[dev]", the Dockerfile ran pip install ".[llm]", and both re-resolved pyproject.toml on every build. A repo-wide grep finds no uv sync, uv run, uv pip or setup-uv anywhere. The lock had drifted far enough to pin eight direct dependencies below pyproject.toml's own lower bounds while CI stayed green, which is only possible if nothing reads it.

But it was not dead weight. GitHub's dependency graph is built from it. All 99 graphed PyPI packages come from uv.lock, and all 27 Dependabot alerts in this repo's history cite manifest=uv.lock — none has ever come from pyproject.toml. Static transitive analysis is unsupported for pip, so the lockfile is the only reason indirect dependencies are monitored at all.

That explains the three open alerts:

alert lock pins image actually installs fix version
#25 fsspec, HIGH CVSS 8.8 (GHSA-27vj-qcqg-25rc) 2026.4.0 2026.9.0 2026.6.0
#26 multidict, MEDIUM 6.7.1 6.9.1 6.9.1
#27 Mako, MEDIUM 1.3.12 1.4.3 1.4.2

The detector was aimed at fiction. Nothing shipped was vulnerable, and equally, a CVE in the versions that do ship would have gone unreported. Deleting the file would have cleared the alerts by removing the only thing capable of raising them.

What this does instead

Makes the lockfile true, so the graph describes the artifact.

uv lock --upgrade, not a plain uv lock. This is the trap in the whole change. A plain relock treats existing pins as resolution preferences: it bumps only the 15 packages pyproject.toml's bounds force and leaves fsspec, Mako and multidict exactly where they are. Promoting that to the real install path would have converted three phantom alerts into three genuine ones and downgraded the image.

uv export + uv pip install --system, not uv sync. --system installs into the same interpreter pip used, so ruff, alembic, pytest and start.sh's bare python / alembic / uvicorn all keep resolving. uv sync would create a .venv nothing activates and would prune pip out of the image. Plain uv pip install is also deliberate over uv pip sync / --exact, which prune.

--locked everywhere, never --frozen. --frozen succeeds on a drifted lock and exports the stale pins — that is the #327 undeclared-runtime-dependency failure waiting to happen again, invisible to startup, pytest and docker-smoke because the imports are function-local. --locked fails instead. CI also runs uv lock --check as an explicit gate. It fails on real pyproject-vs-lock drift but not merely because a newer release exists upstream, since resolution prefers existing pins.

setup-uv is pinned by commit SHA. The action publishes no floating major past v7; @v8, @v9 and @v10 all return 404, so uses: astral-sh/setup-uv@v10 would fail at action-resolution time before any step ran.

Both Dockerfile install steps stay. The second one is still what puts server/ into site-packages; without it alembic upgrade head dies with ModuleNotFoundError: server and the container exits 1 while uvicorn alone still looks fine.

dependabot.yml replaces the pip entry with uv rather than adding one. The pip ecosystem cannot update uv.lock — it rewrites pyproject.toml alone — and uv.lock records each requirement's constraint string verbatim, so a pyproject-only edit invalidates the lock. Keeping both entries would fail uv lock --check on every weekly pip PR. dependency-type: all is required because every alert this repo has ever had was against an indirect pin, and the entry is grouped into one PR because it now covers ~95 packages.

bump_version.py writes uv.lock too. The lock records the project's own version, so a release bumping pyproject.toml alone would fail uv lock --check with a message that never mentions the version.

Verification

Checked against the shipped artifact, not on exit codes.

The rebuilt image installs the same dependency set it does today:

  • boto3 / botocore are one patch newer (1.43.110 vs 1.43.109, released since this morning's build)
  • packaging is now pinned explicitly at 26.3 — the exact version python:3.11-slim already provides, so no change in what is importable
  • the five packages leaving the lock are typer's tree (typer, rich, markdown-it-py, mdurl, shellingham), dropped because huggingface-hub 1.33.0 no longer requires it. Absent from today's image and imported nowhere in the repo.

docker build plus the docker-smoke sequence pass locally against a pgvector/pgvector:pg16 service: migrations run through 0033, /healthz returns {"status":"ok"}, both scripts.bootstrap_* modules import from /app (they resolve as a namespace package from the working directory), site-packages/server is present and non-editable, and pip survives.

The CI install path was reproduced in an isolated venv: the hashed export installs under hash-checking mode, ruff 0.16.10 / alembic 1.20.0 / pytest 9.1.1 land on PATH, and server imports from any cwd.

The release path was exercised: bump_version.py 1.6.0 now updates uv.lock alongside pyproject, README, the three issue templates and the two Helm files, and both bump_version.py --check and uv lock --check stay green.

ruff clean. 1194 unit tests pass. 320 of 321 integration tests pass — test_distributed_rate_limit_blocks_after_limit fails identically on a baseline pyproject-resolved install, so it predates this change and is a local environment difference, not a regression.

Trade-off

The uv updater writes uv.lock only, so it will not raise pyproject.toml's lower bounds the way the pip entry did. Those floors matter to consumers of the published package, so bumping them becomes a deliberate step at release time rather than an automated PR.

Follow-ups

uv.lock was installed by nothing. CI ran pip install -e ".[dev]" and the
Dockerfile ran pip install ".[llm]", both resolving pyproject.toml fresh on
every build, so the lockfile described a dependency set that never shipped. It
had drifted far enough to pin eight direct dependencies BELOW pyproject's own
lower bounds while CI stayed green.

The file was not dead, though. GitHub's dependency graph is built from it: all
99 graphed PyPI packages come from uv.lock, and all 27 Dependabot alerts in this
repo's history cite manifest=uv.lock, none from pyproject.toml. Static
transitive analysis is unsupported for pip, so the lock is the only reason
indirect dependencies are monitored at all. That is why the three open alerts
are about fsspec 2026.4.0, Mako 1.3.12 and multidict 6.7.1 while the image
actually installs 2026.9.0, 1.4.3 and 6.9.1: the detector was pointed at
fiction. It would also have stayed silent about a CVE in the versions that do
ship.

So rather than delete the lockfile and lose transitive alerting, this makes it
true.

- uv.lock regenerated with uv lock --upgrade, not a plain uv lock. A plain
  relock keeps existing pins as resolution preferences and would have left all
  three alerting versions in place, then started shipping them for real.
- Dockerfile and CI install from the lock through uv export + uv pip install
  --system. --system keeps everything in the interpreter that pip used, so
  ruff, alembic, pytest and start.sh's bare python/alembic/uvicorn all still
  resolve. uv sync would have hidden them in a .venv and pruned pip out of the
  image.
- --locked everywhere, never --frozen. --frozen succeeds on a drifted lock and
  exports the stale pins; --locked fails. CI also runs uv lock --check as an
  explicit gate, which fails on real pyproject-vs-lock drift but not merely
  because a newer release exists upstream.
- The Dockerfile keeps both install steps. The second one is still what puts
  server/ into site-packages, without which alembic upgrade head dies with
  ModuleNotFoundError before uvicorn starts.
- dependabot.yml replaces the pip entry with uv rather than adding one. The pip
  ecosystem cannot update uv.lock, and a pyproject-only edit invalidates the
  lock, so keeping both would fail uv lock --check on every weekly pip PR.
  dependency-type all is needed because every alert this repo has ever had was
  against an indirect pin, and the updates are grouped into one PR because the
  entry now covers ~95 packages.
- bump_version.py writes uv.lock too. The lock records the project's own
  version, so a release that bumped pyproject alone would fail uv lock --check
  with a message that never mentions the version.

Verified against the shipped artifact rather than on exit codes: the rebuilt
image installs the same dependency set it does today, with boto3 and botocore
one patch newer and packaging now pinned explicitly at the 26.3 the base image
already provided. The five packages leaving the lock are typer's tree, dropped
because huggingface-hub 1.33.0 no longer requires it; they are absent from
today's image and imported nowhere.

docker build plus the docker-smoke sequence pass locally: migrations run,
/healthz returns ok, both scripts.bootstrap_* modules import from /app,
site-packages/server is present and non-editable, pip survives. The CI install
path was reproduced in an isolated venv: the hashed export installs, ruff,
alembic and pytest land on PATH, and server imports from any cwd. ruff clean,
1194 unit tests pass, 320 of 321 integration tests pass. The one failure,
test_distributed_rate_limit_blocks_after_limit, fails identically on a baseline
pyproject-resolved install, so it predates this change and is a local
environment difference.

Trade-off worth recording: the uv updater writes uv.lock only, so it will not
raise pyproject.toml's lower bounds the way the pip entry did. Those floors
matter to consumers of the published package, so bumping them becomes a
deliberate step at release time instead of an automated PR.

Review pass before merge added five things:

- UV_COMPILE_BYTECODE=1. pip byte-compiles on install and uv does not, so
  without it the image shipped 0 .pyc where it previously shipped 6,788 and
  every fresh container paid source compilation on first import. Now 7,187.
- uv is bind-mounted from a named stage rather than COPYed in. It is
  build-time only, and a COPY baked its 47MB into a layer that no later rm
  could reclaim. With bytecode restored and the binary gone the image is
  861MB against today's 850MB.
- README.md dropped from the dependency-layer COPY. That step now only runs
  uv export, which reads pyproject.toml and uv.lock, so the README was pure
  cache-busting weight on the layer. The project build still gets it from
  COPY . .
- AGENTS.md now says the lockfile has to be refreshed with a dependency
  change. It is the file an agent reads before touching dependencies, so
  leaving it pointing at a bare pip install is how the lock rots again.
- dependabot.yml states versioning-strategy: lockfile-only rather than
  inheriting it from the auto default, and lists python:uv explicitly because
  labels replaces Dependabot's defaults instead of adding to them.
@smaramwbc
smaramwbc force-pushed the chore/adopt-uv-as-install-path branch from 460defa to a20eeff Compare October 9, 2026 07:40
@smaramwbc
smaramwbc merged commit d53f72b into main Oct 9, 2026
10 checks passed
@smaramwbc
smaramwbc deleted the chore/adopt-uv-as-install-path branch October 9, 2026 07:43
smaramwbc added a commit that referenced this pull request Oct 9, 2026
#519 moved the Dockerfile's uv stage from 0.11.12 to 0.12.24 and all nine
checks passed, because nothing compares the two uv pins. Dependabot's docker
ecosystem tracks the `FROM ghcr.io/astral-sh/uv:<version>` stage, and its
github-actions ecosystem tracks the `uses:` SHA, but no updater touches a
workflow `with:` input. So CI's `version:` string is the one pin nobody owns,
and it falls a release further behind on every Dockerfile bump.

That drift is invisible by construction: both paths run `uv export --locked`
against the same lockfile, so while the two releases agree the exported pins
are identical and every check stays green. Verified that is the case today, by
running both uv binaries in containers against the same copy of main: the
exports are byte-identical for the dev and llm extras, on amd64 and arm64, and
`uv lock --check` passes under 0.12.24 against the revision-3 lock 0.11.12
wrote. So #519 was inert. It still retired the invariant it was supposed to
preserve.

Brings CI to 0.12.24 and adds tests/test_uv_pin_consistency.py, which asserts
the Dockerfile stage and the setup-uv input name the same release, that both
are exact X.Y.Z versions rather than a floating tag, and that the input exists
at all. Exercised against four mutations: pins drifted apart, the input
removed, a `latest` tag, and the aligned case.

`[tool.uv] required-version` would also collapse the two pins into one and uv
would enforce it, but it is the wrong tool here. Dependabot's updater runs its
own uv build, and uv exits non-zero when the running version does not satisfy
the field, so an exact pin would break the weekly lockfile refresh that #518
exists to enable. A range satisfies the field while still allowing the two pins
to differ, which is the bug being guarded against.

uv.lock is deliberately untouched: 0.12.24 accepts the committed revision-3
lock as-is. A future `uv lock` write will migrate it to revision 5, which
belongs in whichever PR actually performs that write, not here.

No functional change to the image. The Dockerfile diff is comments, correcting
two that still claimed the pin tracked "the uv that wrote the lockfile" and the
binary's size, now 36MB.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant