Skip to content

[Bug] npm CLI lit parse --format text omits the --- Page N --- separators the cargo and pip CLIs emit #391

Description

@shuvamk

Description

All three distributions install a lit binary, but the npm one produces different
--format text output from the cargo and pip ones: it omits the --- Page N ---
separators. text is the default --format for all three CLIs, so lit parse multi-page.pdf installed from npm silently loses every page boundary.

The cargo and pip CLIs both route OutputFormat::Text through
text::format_text(&result.pages), which prefixes every page with \n--- Page N ---\n:

  • crates/liteparse/src/main.rs:427
  • crates/liteparse-python/src/cli.rs:307
  • crates/liteparse/src/output/text.rs:4

The Node CLI instead writes result.text for any non-json format
(packages/node/src/cli.ts:172, and the same in batch-parse at
packages/node/src/cli.ts:441). For --format text, ParseResult.text is the page
texts joined by \n\n with no page markers (crates/liteparse/src/parser.rs:609-613).

One thing worth flagging for whoever picks this up: result.text is the right
source for --format markdown — the Rust CLI uses result.text.clone() for that
branch — so this can't be a blanket switch to format_text(). Only the text branch
should change.

I'm happy to send a PR if you'd like it fixed this way; I opened an issue first since it
changes the default stdout of a shipped CLI, and that felt like your call rather than
mine.

Steps to Reproduce

Both CLIs on the same document, same flags, from a checkout of main (d2f7626,
v2.10.1):

# cargo CLI
cargo build -p liteparse --no-default-features
./target/debug/lit parse demo/docs/apple-10k-2024.pdf --no-ocr --max-pages 2 --format text -q

# npm CLI (same tree, packages/node/src/cli.ts against a locally built napi module)
node packages/node/src/cli.ts parse demo/docs/apple-10k-2024.pdf --no-ocr --max-pages 2 --format text -q

Result:

bytes --- Page markers first line
cargo lit 6801 2 --- Page 1 ---
npm lit 6768 0 UNITED STATES

The 33-byte delta is exactly 2 x 16 (the two \n--- Page N ---\n separators) plus the
one trailing newline that println! adds and process.stdout.write does not.

Expected: the npm CLI emits the same page-delimited text as the cargo and pip CLIs.
Actual: page boundaries are absent, so downstream consumers can't tell where page 1 ends
and page 2 begins.

Error Message

No error — the output is silently different.

LiteParse Version

2.10.1

Operating System

macOS (Apple Silicon)

Node.js Version

v26.4.0

Additional Context

Two further divergences I noticed while checking this, which the change above would
not close — flagging them so nobody assumes the two CLIs are byte-identical
afterwards:

  • Trailing newline: the Rust and Python CLIs use println!, the Node CLI uses
    process.stdout.write, so the Node output is one byte shorter even ignoring the
    separators.
  • JSON float formatting differs between the surfaces (e.g. 612.0 vs 612).

I built the napi module locally (cargo build -p liteparse-napi --no-default-features,
copied to packages/node/liteparse.node) and ran packages/node/src/cli.ts directly
rather than installing the published npm package, so the reproduction above is against
this tree's source, not a registry install. The relevant source is unchanged between
bc43cd4 and d2f7626.

There are currently no tests over packages/nodeci-node.yml only smoke-tests
getConfig — which is presumably why this went unnoticed; #261 already tracks adding
node e2e tests.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions