We are using LiteParse in a legal/contract document indexing pipeline and are trying to understand the expected behavior/performance of target_pages.
Context: PyMuPDF vs LiteParse
We previously used PyMuPDF/PyMuPDF4LLM for PDF to Markdown extraction.
PyMuPDF worked operationally:
- fast
- easy to split by page
- predictable page mapping
But the Markdown quality was not good enough for legal/construction contracts:
- weaker table preservation
- weaker header/section continuity
- clauses and schedules were harder to reconstruct
- downstream retrieval lost useful contract structure
LiteParse gives much better Markdown for our use case. When we parse the full PDF, the output preserves much more of the useful legal/contract structure. This is why we want to use LiteParse.
The problem: full PDF vs 100-page windows
Full-PDF LiteParse output works best quality-wise, but full-document parsing is too memory-heavy for our Fly.io worker environment.
So we changed our pipeline to use workers that parse the PDF in 100-page LiteParse windows:
parser = liteparse.LiteParse(
output_format="markdown",
target_pages="1-101",
quiet=True,
num_workers=1,
)
result = parser.parse(file_path)Our worker then:
- claims a document indexing job
- starts a LiteParse window, e.g. pages
1-101
- waits for
parser.parse(file_path) to return
- maps the Markdown back to pages
- runs our contract structure engine
- writes pages/chunks/tables to SQLite/FTS
The important point: for the failing PDFs, we are stuck at step 3. We never get back from parser.parse(file_path) for the first 100-page window. Our structure engine and SQLite indexing never run for those documents.
Current live behavior
We tested this on Fly.io with:
- LiteParse: 2.1.1
- Python: 3.11
- Linux container
- 4GB RAM
- 2 shared CPUs
- worker concurrency: 3
- max active project indexing jobs: 2
- LiteParse window size: 100 pages
- LiteParse
num_workers=1
- our own timeout removed
Two workers are currently active, each parsing a different PDF window.
Worker 1
Document:
Current window:
Timeline:
- started:
2026-06-23T03:06:39Z
- still running at:
2026-06-23T03:53:17Z
- no
parser.parse(file_path) return after ~46 minutes
- no downstream page mapping/indexing has started
Worker 2
Document:
Current window:
Timeline:
- started:
2026-06-23T03:08:09Z
- still running at:
2026-06-23T03:53:19Z
- no
parser.parse(file_path) return after ~45 minutes
- no downstream page mapping/indexing has started
Machine state:
- 4GB RAM
- about 1.9GB still available
- each LiteParse child process around 800MB RSS
- no OOM crash
- worker heartbeats still alive
So this does not currently look like SQLite, FTS, queueing, our contract structure engine, or an OOM kill. It looks like LiteParse is spending a very long time inside the first target_pages="1-101" parse.
Previous timeout behavior
Before removing our timeout, we had an 1800s timeout around each LiteParse window.
On a 2GB machine, the following failed:
- 379-page PDF, 8.4MB:
- pages
1-101 timed out after 1800s
- 493-page PDF, 34.1MB:
- pages
1-101 timed out after 1800s
- 1668-page PDF, 120.1MB:
- pages
1-101 timed out after 1800s
After moving to 4GB RAM and removing our timeout, the first two active jobs are still stuck in the same place: the first 100-page LiteParse window.
Other observations
Some smaller PDFs do work:
- 70-page PDF, 6.9MB:
- completed successfully
- but first extraction took around 10-12 minutes
Another PDF completed extraction but had page mapping drift:
- 181-page PDF, 2.7MB:
- extraction completed
- Markdown quality was useful
- page split count drifted by 8 pages
What we are trying to understand
Full-PDF LiteParse gives the best Markdown for these contracts, but is hard to run safely in a worker. 100-page target_pages windows seemed like the natural operational compromise, but in practice some first windows do not return.
Questions:
- Does
target_pages="1-101" actually limit expensive work to that page range, or can LiteParse still perform expensive whole-document analysis before returning?
- Is it expected that a 100-page
target_pages window can take longer than 45 minutes on legal/contract PDFs?
- Is there a recommended page-window size for this kind of document?
- Are there settings that preserve LiteParse’s table/header quality but reduce runtime for windowed parsing?
- Is there a debug/progress mode that can show which internal stage is taking time?
- Is page split drift expected in Markdown output?
- What is the recommended way to map LiteParse Markdown back to source pages?
- Are there PDF features that commonly cause
target_pages extraction to stall?
- For people migrating from PyMuPDF/PyMuPDF4LLM, is there a recommended LiteParse configuration for contract/legal PDFs?
We cannot publicly attach these commercial/legal PDFs, but we can provide sanitized logs/diagnostics or test against a public contract PDF if useful.
We are using LiteParse in a legal/contract document indexing pipeline and are trying to understand the expected behavior/performance of
target_pages.Context: PyMuPDF vs LiteParse
We previously used PyMuPDF/PyMuPDF4LLM for PDF to Markdown extraction.
PyMuPDF worked operationally:
But the Markdown quality was not good enough for legal/construction contracts:
LiteParse gives much better Markdown for our use case. When we parse the full PDF, the output preserves much more of the useful legal/contract structure. This is why we want to use LiteParse.
The problem: full PDF vs 100-page windows
Full-PDF LiteParse output works best quality-wise, but full-document parsing is too memory-heavy for our Fly.io worker environment.
So we changed our pipeline to use workers that parse the PDF in 100-page LiteParse windows:
Our worker then:
1-101parser.parse(file_path)to returnThe important point: for the failing PDFs, we are stuck at step 3. We never get back from
parser.parse(file_path)for the first 100-page window. Our structure engine and SQLite indexing never run for those documents.Current live behavior
We tested this on Fly.io with:
num_workers=1Two workers are currently active, each parsing a different PDF window.
Worker 1
Document:
Current window:
target_pages="1-101"Timeline:
2026-06-23T03:06:39Z2026-06-23T03:53:17Zparser.parse(file_path)return after ~46 minutesWorker 2
Document:
Current window:
target_pages="1-101"Timeline:
2026-06-23T03:08:09Z2026-06-23T03:53:19Zparser.parse(file_path)return after ~45 minutesMachine state:
So this does not currently look like SQLite, FTS, queueing, our contract structure engine, or an OOM kill. It looks like LiteParse is spending a very long time inside the first
target_pages="1-101"parse.Previous timeout behavior
Before removing our timeout, we had an 1800s timeout around each LiteParse window.
On a 2GB machine, the following failed:
1-101timed out after 1800s1-101timed out after 1800s1-101timed out after 1800sAfter moving to 4GB RAM and removing our timeout, the first two active jobs are still stuck in the same place: the first 100-page LiteParse window.
Other observations
Some smaller PDFs do work:
Another PDF completed extraction but had page mapping drift:
What we are trying to understand
Full-PDF LiteParse gives the best Markdown for these contracts, but is hard to run safely in a worker. 100-page
target_pageswindows seemed like the natural operational compromise, but in practice some first windows do not return.Questions:
target_pages="1-101"actually limit expensive work to that page range, or can LiteParse still perform expensive whole-document analysis before returning?target_pageswindow can take longer than 45 minutes on legal/contract PDFs?target_pagesextraction to stall?We cannot publicly attach these commercial/legal PDFs, but we can provide sanitized logs/diagnostics or test against a public contract PDF if useful.