Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
9757905
feat: expose PDF document provenance metadata
hexapode Jul 27, 2026
24cd52c
fixes
logan-markewich Jul 29, 2026
f791505
fix(markdown): don't backslash-escape inside inline code spans
shuvamk Jul 30, 2026
3ffc614
fix: add proper RTL and LTR text detection
logan-markewich Jul 31, 2026
05f389b
add missing file
logan-markewich Aug 1, 2026
093ddba
table improvements
logan-markewich Aug 2, 2026
520413d
table improvements: two-row tables + soft-wrap row grouping
logan-markewich Aug 2, 2026
08225ac
tables: split PDFium merged runs on real word geometry
logan-markewich Aug 2, 2026
87d54ef
tables: read rowspans off the rules so merged cells stop reading as e…
logan-markewich Aug 2, 2026
46e95b4
tables: require a material share before a leaf vetoes the global rule…
logan-markewich Aug 2, 2026
8aad56b
tables: detect two-column tables enclosed in a booktabs rule band
logan-markewich Aug 2, 2026
18be0de
docs: record round 4 of the table/TEDS work
logan-markewich Aug 2, 2026
bd96364
tables: ignore vertical rules too short to be column boundaries
logan-markewich Aug 2, 2026
6e1557b
tables: recover a ruled table's unruled outer rows and columns
logan-markewich Aug 2, 2026
09c9080
tables: split a grid component at row bands no vertical rule spans
logan-markewich Aug 2, 2026
54aed20
docs: record round 5 of the table/TEDS work
logan-markewich Aug 2, 2026
b7bc071
cleanup 1
logan-markewich Aug 2, 2026
950aced
final cleanup
logan-markewich Aug 2, 2026
48dc7f4
remove tracking file
logan-markewich Aug 2, 2026
aba35f0
Merge pull request #394 from run-llama/logan/table-improvements
logan-markewich Aug 3, 2026
05d4556
Merge pull request #393 from run-llama/logan/lrt-rtl-fixes
logan-markewich Aug 3, 2026
ef4f6bb
Merge pull request #381 from run-llama/codex/document-provenance-meta…
logan-markewich Aug 3, 2026
d2ada91
v2.11.0
logan-markewich Aug 3, 2026
0f579ee
pdfium v1.5.0
logan-markewich Aug 3, 2026
29177bd
docs: make the agent skill guide canonical and link it from the index
logan-markewich Aug 3, 2026
d35f369
nit
logan-markewich Aug 3, 2026
3ba5820
Merge pull request #399 from run-llama/docs/canonical-agent-skill
logan-markewich Aug 3, 2026
8ad92a3
fix: extract visible AcroForm values as text
Aug 4, 2026
d9ee62e
fix: exclude non-widget annotations from form flattening
Aug 4, 2026
c3b61b8
fix: address review of AcroForm text extraction
logan-markewich Aug 5, 2026
a3cf266
fix cargo versions
logan-markewich Aug 5, 2026
cb9e1cf
Merge pull request #401 from Neel49/agent/extract-visible-acroform-text
logan-markewich Aug 5, 2026
f7b7fb3
v2.11.1
logan-markewich Aug 5, 2026
bfcf5d2
feat: expose source document page count
hexapode Aug 10, 2026
e46b369
feat: expose source document page count
hexapode Aug 10, 2026
3ee4ec3
feat(node): add bounded-memory batch parsing
hexapode Aug 10, 2026
5dd6339
feat: support tolerant page extraction errors
hexapode Aug 10, 2026
bcd8d77
feat: return screenshots from parse
hexapode Aug 10, 2026
629f883
fix: preserve text at rotated page edges
hexapode Aug 11, 2026
3f84983
nit
logan-markewich Aug 11, 2026
984ed28
Merge pull request #408 from run-llama/pierre/expose-total-pages
logan-markewich Aug 11, 2026
c4730ec
Merge branch 'main' into pierre/bounded-memory-parse
logan-markewich Aug 11, 2026
1ed4ecc
remove duplicate def
logan-markewich Aug 11, 2026
2365679
refactor into core lib
logan-markewich Aug 11, 2026
ee7da84
Fix bad merge
logan-markewich Aug 11, 2026
8a12f58
fix wasm cicd
logan-markewich Aug 11, 2026
e5dc0a3
pin miniflare?
logan-markewich Aug 11, 2026
7d3f93a
review: clarify batch API surface, add deterministic session cleanup
logan-markewich Aug 11, 2026
e047893
nit
logan-markewich Aug 11, 2026
53e4fc8
Merge pull request #409 from run-llama/pierre/bounded-memory-parse
logan-markewich Aug 11, 2026
beca73a
Merge branch 'main' into pierre/tolerant-page-errors
logan-markewich Aug 11, 2026
80f62c7
fix: harden tolerant page-error handling across surfaces
logan-markewich Aug 12, 2026
5109b46
Merge pull request #410 from run-llama/pierre/tolerant-page-errors
logan-markewich Aug 12, 2026
2d02d4a
Merge branch 'main' into pierre/parse-screenshots
logan-markewich Aug 12, 2026
ce6c504
feat(wasm): support extract_screenshots in the WASM build
logan-markewich Aug 12, 2026
696367e
Merge pull request #411 from run-llama/pierre/parse-screenshots
logan-markewich Aug 12, 2026
a8466f9
Merge branch 'main' into pierre/rotation-aware-page-bounds
logan-markewich Aug 12, 2026
a8c193c
Merge pull request #412 from run-llama/pierre/rotation-aware-page-bounds
logan-markewich Aug 12, 2026
bd35c9f
fix(markdown): size code-span fence to longest backtick run; unify sp…
logan-markewich Aug 12, 2026
814da0c
Merge pull request #390 from shuvamk/fix/markdown-no-escape-in-code-s…
logan-markewich Aug 13, 2026
2fd644a
v2.12.0
logan-markewich Aug 13, 2026
9d740a4
fix: keep side-by-side ruled grids separate
Chengyunlai Jul 31, 2026
26b79a6
fix: keep projected lines within ruled tables
Chengyunlai Aug 15, 2026
a66ecaf
fix: keep table reordering inside its xy-cut leaf
logan-markewich Aug 18, 2026
6c96a22
nit
logan-markewich Aug 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion .github/workflows/ci-wasm.yml
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,13 @@ jobs:
path: packages/wasm/pkg/

- name: Install Miniflare
run: npm install miniflare@latest
# Pinned, NOT @latest: miniflare@latest currently resolves to a 5.x
# alpha whose `new Miniflare()` options schema replaced the 4.x
# single-worker form (`modules: [...]`) used by
# scripts/edge-compat/wasm-test.mjs with a nested `workers: [...]`
# shape. There is no root package.json/lockfile in this repo, so the
# version has to live here. Bump deliberately, re-running the test.
run: npm install --no-save miniflare@4.20260730.0

- name: Run edge runtime parse test
run: node scripts/edge-compat/wasm-test.mjs
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ temp/
!integration_tests_data/receipt.png
!integration_tests_data/sample3.docx.doc
!integration_tests_data/sample.pdf
!integration_tests_data/filled_acroform.pdf
deno.json

# Python files
Expand Down
13 changes: 7 additions & 6 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 14 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,8 @@ npx skills add run-llama/llamaparse-agent-skills --skill liteparse

Or copy-pasting the [`SKILL.md`](https://github.com/run-llama/llamaparse-agent-skills/blob/main/skills/liteparse/SKILL.md) file to your own skills setup.

See the [Agent Skill guide](https://developers.llamaindex.ai/liteparse/guides/agent-skill/?utm_source=github&utm_medium=liteparse) for requirements and usage patterns.

## CLI Usage

The CLI is the same across all installations (`npm`, `pip`, `cargo install`).
Expand Down Expand Up @@ -257,7 +259,18 @@ link annotations. The field is absent by default; enabled untagged pages contain

Parse results (Rust/Node/Python APIs) carry the document's `/Info` `creator`
and `producer` entries when present; these are API-only and never appear in
CLI JSON output. Enable `--extract-content-bounds` (Rust/Python
CLI JSON output. Enable `extract_document_metadata` (JavaScript/WASM
`extractDocumentMetadata`) to add `doc_meta`/`docMeta`, a provenance object
with the `/Info` creation/modification dates, PDF version and encryption
permissions, signature state, incremental-save markers, trailer ID comparison,
the document catalog's XMP packet (capped at 64 KiB, with `xmp_truncated`
when it was cut), and source file size. It is off by default because it
streams the whole source file once; it is absent for inputs converted from a
non-PDF format, where the facts would describe the intermediate PDF rather
than your file. `xmp` needs a structural parse of the document, so it is
skipped (left absent) for sources over 16 MiB and in WASM builds — the other
fields are unaffected.
Enable `--extract-content-bounds` (Rust/Python
`extract_content_bounds`, JavaScript/WASM `extractContentBounds`) to add a
per-page `content_bounds`: the union bbox of the page's top-level content
objects in viewport coords (absent for empty pages). Enable
Expand Down
8 changes: 4 additions & 4 deletions crates/liteparse-napi/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "liteparse-napi"
version = "2.10.1"
version = "2.12.0"
edition.workspace = true
license.workspace = true
repository.workspace = true
Expand All @@ -14,11 +14,11 @@ default = ["tesseract"]
tesseract = ["liteparse/tesseract"]

[dependencies]
liteparse = { package = "liteparse", version = "2.10.1", path = "../liteparse", default-features = false }
pdfium-sys = { package = "liteparse-pdfium-sys", version = "1.4.0", path = "../pdfium-sys" }
liteparse = { package = "liteparse", version = "2.12.0", path = "../liteparse", default-features = false }
pdfium-sys = { package = "liteparse-pdfium-sys", version = "1.5.0", path = "../pdfium-sys" }
napi = { version = "2", features = ["async", "serde-json", "napi9"] }
napi-derive = "2"
pdfium = { package = "liteparse-pdfium", version = "1.4.0", path = "../pdfium" }
pdfium = { package = "liteparse-pdfium", version = "1.5.0", path = "../pdfium" }
image = { version = "0.25", default-features = false, features = ["png"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
Expand Down
102 changes: 100 additions & 2 deletions crates/liteparse-napi/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ use napi_derive::napi;
mod types;

use types::{
JsLiteParseConfig, JsPageComplexityStats, JsPageInput, JsParseResult, JsScreenshotResult,
JsTextItem,
JsLiteParseConfig, JsPageComplexityStats, JsPageInput, JsParseBatch, JsParseResult,
JsScreenshotResult, JsTextItem,
};

/// Main LiteParse parser class.
Expand Down Expand Up @@ -48,6 +48,43 @@ impl LiteParse {
Ok(JsParseResult::from_rust(&result, &self.config))
}

/// Open a document for bounded-memory batch parsing. Internal plumbing
/// for the JS wrapper's `parseBatches()` — prefer that; it also closes
/// the session for you.
///
/// Converts a non-PDF source once, then yields `batchSize` pages at a time
/// via `nextBatch()` (default 25). Cross-page passes (repeated
/// header/footer removal, image deduplication) see only the pages in their
/// own batch, so output can differ from a whole-document `parse()`.
#[napi]
pub async fn open_batch_session(
&self,
input: Either<String, Buffer>,
batch_size: Option<u32>,
) -> Result<ParseSession> {
use liteparse::types::PdfInput;

let pdf_input = match input {
Either::A(path) => PdfInput::Path(path),
Either::B(buf) => PdfInput::Bytes(buf.to_vec()),
};
let batch_size = batch_size
.map(|v| v as usize)
.unwrap_or(liteparse::DEFAULT_PAGE_BATCH_SIZE);

let session = self
.inner
.open_batch_session(pdf_input, batch_size)
.await
.map_err(|e| Error::from_reason(e.to_string()))?;

Ok(ParseSession {
total_pages: session.total_pages(),
inner: std::sync::Arc::new(tokio::sync::Mutex::new(Some(session))),
config: self.config.clone(),
})
}

/// Parse from pre-extracted pages, skipping PDFium text extraction.
///
/// The caller supplies pages already populated with text items in viewport
Expand Down Expand Up @@ -134,6 +171,67 @@ impl LiteParse {
}
}

/// A document opened once and parsed in bounded page batches. Internal
/// plumbing for the JS wrapper's `parseBatches()` — prefer that.
///
/// Created by `LiteParse.openBatchSession()`. The converted-PDF temporary
/// file for a non-PDF source lives as long as the session, so conversion is
/// paid once no matter how many batches are consumed. Call `close()` when
/// abandoning the session early — otherwise that temp file waits for GC.
#[napi]
pub struct ParseSession {
/// The core session is `&mut` per batch, but napi hands out `&self`, so
/// the mutation is serialized here. Concurrent `nextBatch()` calls queue
/// rather than interleave, which also keeps batch order well-defined.
/// `None` after `close()`.
inner: std::sync::Arc<tokio::sync::Mutex<Option<liteparse::ParseSession>>>,
config: liteparse::config::LiteParseConfig,
total_pages: u32,
}

#[napi]
impl ParseSession {
/// Total pages in the source document, before `maxPages` or batching.
#[napi(getter)]
pub fn total_pages(&self) -> u32 {
self.total_pages
}

/// Parse and return the next batch, or `null` once every page within
/// `maxPages` has been yielded. Rejects if the session is closed.
#[napi]
pub async fn next_batch(&self) -> Result<Option<JsParseBatch>> {
let inner = self.inner.clone();
let mut session = inner.lock().await;
let batch = session
.as_mut()
.ok_or_else(|| Error::from_reason("session is closed"))?
.next_batch()
.await
.map_err(|e| Error::from_reason(e.to_string()))?;

Ok(batch.map(|batch| JsParseBatch {
start_page: batch.start_page,
end_page: batch.end_page,
result: JsParseResult::from_rust(&batch.result, &self.config),
}))
}

/// Release the session's resources now — most importantly the converted
/// temporary PDF for a non-PDF source, which otherwise lives until the
/// JS object is garbage collected. Idempotent; `nextBatch()` rejects
/// afterwards.
#[napi]
pub async fn close(&self) -> Result<()> {
let inner = self.inner.clone();
let mut session = inner.lock().await;
// Dropping the core session drops the conversion guard, which
// removes the temp file.
session.take();
Ok(())
}
}

/// Search text items for phrase matches, returning merged items with combined bounding boxes.
#[napi]
pub fn search_items(
Expand Down
Loading