pyproject.toml declares 1 of ~15 runtime deps while advertising a console entry point

[usathyan]

Found during the 2026-07-25 full-repo review. Filed for the record — not proposed for action right now.

Status: intentionally deferred

pyproject.toml declares exactly one runtime dependency (chonkie>=1.0). Undeclared but required at runtime: sift-kg, kreuzberg, pydantic, networkx, pyyaml, httpx, fastapi, sse-starlette, and more.

This is already documented in the file's own header comment, which states that declaring the full dependency graph is "a future migration (post-v3.0)" and that uv pip install -e . alone "will NOT produce a working install". That is a deliberate decision, and this issue does not argue against it.

Why it is still worth recording

The same file also declares a console entry point:

[project.scripts]
epistract = "epistract.cli:main"

So uv pip install -e . does not fail cleanly — it produces an installed, importable, broken CLI. A user who installs the package the conventional way gets a command that exists and then fails at import time on the first heavy dependency, rather than an error at install time telling them to run scripts/setup.sh.

That interaction between the partial dependency declaration and the advertised entry point is the part worth revisiting when the deferral is lifted.

Possible resolutions, for whenever this is picked up

1. Declare the full runtime dependency set (the documented eventual plan).
2. Keep the partial declaration but move the heavy imports behind a lazy path in cli.py, so the CLI's stdlib-only subcommands work and the rest fails with an actionable message.
3. Add an [project.optional-dependencies] extra (e.g. pipeline) so uv pip install -e ".[pipeline]" is a supported, discoverable path.

No action taken pending a decision on which of these is wanted.

[usathyan]

Heads-up so this issue does not read as stale: PR #39 adds a [tool.ruff] section to pyproject.toml — the same file this issue is about.

Different section, no conflict, and the [project].dependencies gap described here is untouched and still deferred. Flagging it only because this issue says the file is not being modified, and a reader arriving after #39 merges would find otherwise.

[chrisdavidson]

Reviewed this against main @ a362da7. The premise no longer reproduces — the specific defect described here was fixed after the issue was filed. Details below, plus two structural blockers that change the cost estimate on the proposed options.

1. uv pip install -e . no longer produces a broken CLI

The issue states the install yields an "installed, importable, broken" command that "fails at import time on the first heavy dependency." I tested this in two throwaway venvs containing only epistract + chonkie, with a hermetic HOME (otherwise the real ~/.epistract registry leaks into the run):

Install mode	--help	projects list	init	add files	index	search	status
-e (editable)	ok	ok	ok	ok	ok	ok	ok
non-editable	ok	ok	ok	ok	ok	ok	ok

core/cli.py imports only stdlib at module level (lines 25–33); every heavy dependency sits behind a lazy import inside its handler — cli.py:137-140 (enhance), cli.py:245 (export), index_db.py:79 (kreuzberg). Failures are already actionable:

$ epistract export --format okf --project demo
Error: OKF export requires dependencies that are not installed (No module named 'unidecode').
Run `scripts/setup.sh` (or `uv pip install sift-kg`) to install them.

$ epistract index --project demo          # corpus contains a PDF
Warning: cannot extract text from b.pdf: No module named 'kreuzberg'
Project 'demo': indexed 1 document(s) (1 chunks) ...
  extraction failed: corpus/b.pdf

So proposed option 2 is substantially already implemented. Whatever is done here should not re-do it.

2. Two blockers that gate options 1 and 3

The three options read as independent and roughly comparable in cost. They aren't — option 1 silently contains a repackaging migration, because declaring the deps alone still does not yield a working install.

Blocker A — from core.X imports break under the installed package name. [tool.setuptools.package-dir] maps core/ → the epistract package, but 8 core modules import their siblings as core.* (71 such lines across core/, tests/, scripts/, commands/):

$ python -c "import epistract.run_sift"
ModuleNotFoundError: No module named 'core'

Affected: run_sift.py, ingest_documents.py, build_extraction.py, domain_resolver.py, domain_wizard.py, label_epistemic.py, label_communities.py, okf_export.py. These work in-repo only because the repo root is on sys.path. normalize_extractions.py:43-51 additionally mutates sys.path. Notably, cli.py is the one module that gets this right (from . import index_db) — which is exactly why the light CLI is the part that works.

Blocker B — repo-root-relative path anchoring breaks in a non-editable install. 7 sites anchor on Path(__file__).parent.parent, meaning "repo root" in a checkout but site-packages/ once installed. domains/ isn't declared as package data, so it isn't shipped at all:

$ python -c "from epistract.domain_resolver import DOMAINS_DIR, list_domains; \
             print(DOMAINS_DIR, DOMAINS_DIR.exists()); print(list_domains())"
/…/site-packages/domains  False
[]

Sites: domain_resolver.py:25, run_sift.py:120, run_sift.py:239, label_epistemic.py:48, label_epistemic.py:432, build_extraction.py:21, normalize_extractions.py:43. Editable installs mask this because __file__ still points into the checkout.

Consequence for option 3: adding a [pipeline] extra without first fixing A and B installs ~2 GB of dependencies (sentence-transformers/PyTorch) to reach code that then cannot import. It is not independently deliverable — it needs option 1's real scope underneath it.

3. Two smaller inaccuracies in the current pyproject header

• The single declared dependency is the one the entry point never needs. chonkie is imported only by core/chunk_document.py, reachable only from scripts/extract_corpus.py (not packaged) and tests/. Nothing reachable from epistract.cli:main imports it — index_db.py:107 has its own stdlib chunker fallback. The header's claim that the light subcommands "run with only stdlib + chonkie" overstates it: they run on stdlib alone.
• The stated rationale is partly unbacked. The header justifies the chonkie entry as letting "CI linters … discover the addition," but .github/workflows/ does not exist — there is no CI.

4. Suggested reframing

The question upstream of "which dependency-declaration strategy" is which channel pip install epistract is meant to serve. Every pipeline command invokes scripts by path out of the checkout — roughly 19 ${CLAUDE_PLUGIN_ROOT}/…/*.py, 22 scripts/*.py, 10 python3 -m core.cli — and zero use the installed console script; scripts/setup.sh never runs pip install -e .. On that evidence the pip surface is a second, narrower product: the stdlib-only project/index/search CLI.

If that reading is right, the cheap and honest resolution is to finish that contract rather than migrate: correct the header (drop the "will NOT produce a working install" claim and the CI rationale), move chonkie out of [project].dependencies (leaving it accurately empty, or behind a chunking extra for extract_corpus.py), and audit the remaining lazy-import messages. Full installability (options 1 + 3 together, with A and B fixed) is real work worth doing only when someone actually needs pip-installed pipeline execution — and its hardest part isn't the dep list, it's that relative imports break the ${CLAUDE_PLUGIN_ROOT}/core/*.py direct-script invocations the plugin relies on.

Two things worth splitting out

• epistract init demo --domain no-such-domain succeeds silently. registry.py:144 stores domain verbatim and never calls resolve_domain, so the light CLI never consults DOMAINS_DIR — which is why init works non-editable despite list_domains() returning []. Adding validation would immediately expose Blocker B.
• No CI exists to catch any of this.

Reproduction

uv venv iso && VIRTUAL_ENV=$PWD/iso uv pip install /path/to/epistract   # non-editable, no -e
export HOME=$(mktemp -d)          # hermetic: else ~/.epistract registry leaks in
./iso/bin/epistract init demo --domain drug-discovery
echo "Aspirin inhibits COX-2." > a.txt
./iso/bin/epistract add files a.txt --project demo && ./iso/bin/epistract index --project demo
./iso/bin/epistract search aspirin --project demo          # all succeed, stdlib only
./iso/bin/python -c "import epistract.run_sift"            # ModuleNotFoundError: No module named 'core'
./iso/bin/python -c "from epistract.domain_resolver import list_domains; print(list_domains())"   # []