Partially garbled OCR pages report high confidence, empty warnings, and no hosted recommendation: the page-level mean cannot express a bad region

[xmasyx]

Summary

A page whose OCR output is partly garbage comes back with ocr_confidence high, warnings empty, and hosted_recommended false. The page-level mean is the only quality signal that reaches the caller, and a mean answers "is this page mostly fine?" while an integrator needs "is any part of this page unusable?". Those two questions diverge exactly on the common real-world page: body text plus a small dense region (chart axis, legend, footnote strip, stamp).

The result is silent: garbled characters sit inline with good prose, at the same reported confidence, with nothing marking them.

Reproduction

Synthetic page, one page, image-only (no text layer — pdftotext returns zero characters), composed of two rasterized regions:

• body text at normal size: Sample readings by bucket / The series below is a routine measurement table.
• a strip of tick labels reduced to roughly 3pt: 0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 210 220 230

const { processPdfWithOcr } = require('@firecrawl/pdf-inspector');
const r = await processPdfWithOcr(readFileSync('mixed-quality.pdf'), { mode: 'Auto' });
console.log(r.markdown);
console.log(r.pagesRecommendingHosted, r.pages[0].provenance.ocrConfidence, r.pages[0].provenance.warnings);

Observed

Sample readings by bucket The series below is a routine measurement table.

## 0230203040506070809010011012013010150160170 10190200210220

pagesRecommendingHosted: []
ocrConfidence:           0.966
warnings:                []

The body text is transcribed perfectly. The strip loses every separator, drops 140 and 180, and pulls the wrapped 230 up to position 2 — so the region is not merely unformatted, its values are wrong. It is then promoted to an ## heading. Nothing in the result distinguishes those characters from the correct line above them.

Same shape on a real document: a page of study notes with an embedded chart returned the axis as one glued digit run and rendered the legend SD = 10 / SD = 50 as SD=110, again with warnings: [], pagesRecommendingHosted: [], and page confidence 0.95.

Why it happens

Reading main at 1.17.0:

• src/vision/contracts.rs:144 — every recognition span carries its own confidence.
• src/vision/contracts.rs:156 and :201 — the page keeps only mean_confidence / ocr_confidence, "mean across accepted spans". The distribution is discarded at that boundary.
• src/vision/contracts.rs:38,49 — minimum_confidence defaults to 0.0, so by default no weak span is dropped; the weak spans stay in the output and in the mean.
• src/vision/pipeline.rs:81,92 — hosted_recommendation_confidence defaults to 0.5, compared against that page-level value.
• src/vision/pipeline.rs:447-450 — pages_recommending_hosted filters on provenance.hosted_recommended.

Twenty bad spans among two hundred good ones cannot move a mean below 0.5, so the threshold can never fire on a partially bad page. The signal is structurally unable to express the failure it is meant to catch. docs/ocr-runtime.md states that pages_recommending_hosted marks pages that are "empty, low-confidence, or still appear incomplete" — a partially garbled page is none of the three by page-level mean, yet it is the case where an integrator most needs the warning.

This is the OCR-side twin of #352, where a page-level CipherGarbleStats signal is bypassed by a page that mixes healthy and garbled fonts. Same architecture, different pipeline: a per-page aggregate cannot describe a sub-page defect.

Suggested fix

Both parts are additive and backward compatible; the span confidences already exist, nothing new has to be computed.

1. Keep the shape of the distribution, not just its centre. In PageProvenance (src/vision/contracts.rs, next to ocr_confidence), add:

   /// Lowest accepted span confidence on this page.
   pub min_confidence: Option<f32>,
   /// Accepted spans below the hosted-recommendation threshold.
   pub low_confidence_spans: u32,

   A caller can then flag the page — or the region — without guessing. Today there is no field that could carry this information.

2. Make the hosted recommendation a worst-region test in OR with the mean. In the routing decision behind hosted_recommended (src/vision/pipeline.rs), recommend hosted when a contiguous run of spans below hosted_recommendation_confidence exceeds a small character budget, in addition to the existing mean comparison. A run of adjacent weak spans is exactly the signature of one bad region on an otherwise good page, and it is invisible to any average.

An alternative that needs no new field: change the default minimum_confidence from 0.0 to a non-zero value so weak spans are dropped rather than emitted. I would argue against it on its own — dropping is also silent, and a missing region is harder to notice than a wrong one — but combined with (1) it would at least keep known-bad characters out of the markdown.

Environment

@firecrawl/pdf-inspector 1.17.0, Node bindings, macOS arm64, PDFium native-v7988, ONNX Runtime 1.27.0, mode: 'Auto', defaults otherwise. The recogniser quality is not the complaint here and may well be platform-dependent — the reporting path is the same on every platform, and that is what this issue is about.