Description
All three distributions install a lit binary, but the npm one produces different
--format text output from the cargo and pip ones: it omits the --- Page N ---
separators. text is the default --format for all three CLIs, so lit parse multi-page.pdf installed from npm silently loses every page boundary.
The cargo and pip CLIs both route OutputFormat::Text through
text::format_text(&result.pages), which prefixes every page with \n--- Page N ---\n:
crates/liteparse/src/main.rs:427
crates/liteparse-python/src/cli.rs:307
crates/liteparse/src/output/text.rs:4
The Node CLI instead writes result.text for any non-json format
(packages/node/src/cli.ts:172, and the same in batch-parse at
packages/node/src/cli.ts:441). For --format text, ParseResult.text is the page
texts joined by \n\n with no page markers (crates/liteparse/src/parser.rs:609-613).
One thing worth flagging for whoever picks this up: result.text is the right
source for --format markdown — the Rust CLI uses result.text.clone() for that
branch — so this can't be a blanket switch to format_text(). Only the text branch
should change.
I'm happy to send a PR if you'd like it fixed this way; I opened an issue first since it
changes the default stdout of a shipped CLI, and that felt like your call rather than
mine.
Steps to Reproduce
Both CLIs on the same document, same flags, from a checkout of main (d2f7626,
v2.10.1):
# cargo CLI
cargo build -p liteparse --no-default-features
./target/debug/lit parse demo/docs/apple-10k-2024.pdf --no-ocr --max-pages 2 --format text -q
# npm CLI (same tree, packages/node/src/cli.ts against a locally built napi module)
node packages/node/src/cli.ts parse demo/docs/apple-10k-2024.pdf --no-ocr --max-pages 2 --format text -q
Result:
|
bytes |
--- Page markers |
first line |
cargo lit |
6801 |
2 |
--- Page 1 --- |
npm lit |
6768 |
0 |
UNITED STATES |
The 33-byte delta is exactly 2 x 16 (the two \n--- Page N ---\n separators) plus the
one trailing newline that println! adds and process.stdout.write does not.
Expected: the npm CLI emits the same page-delimited text as the cargo and pip CLIs.
Actual: page boundaries are absent, so downstream consumers can't tell where page 1 ends
and page 2 begins.
Error Message
No error — the output is silently different.
LiteParse Version
2.10.1
Operating System
macOS (Apple Silicon)
Node.js Version
v26.4.0
Additional Context
Two further divergences I noticed while checking this, which the change above would
not close — flagging them so nobody assumes the two CLIs are byte-identical
afterwards:
- Trailing newline: the Rust and Python CLIs use
println!, the Node CLI uses
process.stdout.write, so the Node output is one byte shorter even ignoring the
separators.
- JSON float formatting differs between the surfaces (e.g.
612.0 vs 612).
I built the napi module locally (cargo build -p liteparse-napi --no-default-features,
copied to packages/node/liteparse.node) and ran packages/node/src/cli.ts directly
rather than installing the published npm package, so the reproduction above is against
this tree's source, not a registry install. The relevant source is unchanged between
bc43cd4 and d2f7626.
There are currently no tests over packages/node — ci-node.yml only smoke-tests
getConfig — which is presumably why this went unnoticed; #261 already tracks adding
node e2e tests.
Description
All three distributions install a
litbinary, but the npm one produces different--format textoutput from the cargo and pip ones: it omits the--- Page N ---separators.
textis the default--formatfor all three CLIs, solit parse multi-page.pdfinstalled from npm silently loses every page boundary.The cargo and pip CLIs both route
OutputFormat::Textthroughtext::format_text(&result.pages), which prefixes every page with\n--- Page N ---\n:crates/liteparse/src/main.rs:427crates/liteparse-python/src/cli.rs:307crates/liteparse/src/output/text.rs:4The Node CLI instead writes
result.textfor any non-jsonformat(
packages/node/src/cli.ts:172, and the same inbatch-parseatpackages/node/src/cli.ts:441). For--format text,ParseResult.textis the pagetexts joined by
\n\nwith no page markers (crates/liteparse/src/parser.rs:609-613).One thing worth flagging for whoever picks this up:
result.textis the rightsource for
--format markdown— the Rust CLI usesresult.text.clone()for thatbranch — so this can't be a blanket switch to
format_text(). Only thetextbranchshould change.
I'm happy to send a PR if you'd like it fixed this way; I opened an issue first since it
changes the default stdout of a shipped CLI, and that felt like your call rather than
mine.
Steps to Reproduce
Both CLIs on the same document, same flags, from a checkout of
main(d2f7626,v2.10.1):
Result:
--- Pagemarkerslit--- Page 1 ---litUNITED STATESThe 33-byte delta is exactly
2 x 16(the two\n--- Page N ---\nseparators) plus theone trailing newline that
println!adds andprocess.stdout.writedoes not.Expected: the npm CLI emits the same page-delimited text as the cargo and pip CLIs.
Actual: page boundaries are absent, so downstream consumers can't tell where page 1 ends
and page 2 begins.
Error Message
No error — the output is silently different.
LiteParse Version
2.10.1
Operating System
macOS (Apple Silicon)
Node.js Version
v26.4.0
Additional Context
Two further divergences I noticed while checking this, which the change above would
not close — flagging them so nobody assumes the two CLIs are byte-identical
afterwards:
println!, the Node CLI usesprocess.stdout.write, so the Node output is one byte shorter even ignoring theseparators.
612.0vs612).I built the napi module locally (
cargo build -p liteparse-napi --no-default-features,copied to
packages/node/liteparse.node) and ranpackages/node/src/cli.tsdirectlyrather than installing the published npm package, so the reproduction above is against
this tree's source, not a registry install. The relevant source is unchanged between
bc43cd4andd2f7626.There are currently no tests over
packages/node—ci-node.ymlonly smoke-testsgetConfig— which is presumably why this went unnoticed; #261 already tracks addingnode e2e tests.