Grid projection: low-confidence diagonal-watermark fragments still split continuous lines (follow-up to #284)

[marcosmarf27]

Summary

Follow-up to #284, as requested in the closing comment. We tested main @ a56b9ff (post-#287, since no release with the fix is published yet) and #287 clearly fixes the vertical-sidebar interference — but low-confidence noise fragments from the diagonal watermark still get interleaved mid-line, splitting otherwise continuous body lines.

Specific doc/page (repro)

• Doc: Matrícula Ruim - Pg 2 nota 1 - PÉSSIMO.pdf — from the zip attached to Grid projection scrambles single-column, tightly-spaced typewritten scans even when OCR boxes are correct #284: TEstes arquivos OCR (2).zip
• Page: 1 (single-page PDF)
• This is the exact page the R-8-2559 snippets in Grid projection scrambles single-column, tightly-spaced typewritten scans even when OCR boxes are correct #284 came from.

lit parse "Matrícula Ruim - Pg 2 nota 1 - PÉSSIMO.pdf" \
  --ocr-server-url http://localhost:8829/ocr --ocr-language pt --target-pages 1 --dpi 150

(OCR server: PaddleOCR with the updated ocr/paddleocr/server.py forwarding polygon per the new OCR API spec.)

Environment

• liteparse main @ a56b9ff (reports lit 2.0.7), built with cargo install --git ... --no-default-features
• PaddleOCR (HTTP OCR server returning bbox + confidence + polygon), language=pt, DPI 150

What #287 fixed ✅

On v2.0.5/v2.0.7, fragments of the vertical sidebar text were interleaved into body lines (stray or, radores. tokens splitting lines). On main @ a56b9ff those are gone from the body and the rotated sidebar text now lands at the bottom of the page, as intended. We also saw no regressions (and slightly better diacritics/recall) on our other regression docs (159-page judicial PDF, dense 5-page matrícula).

What still reproduces ⚠️

The diagonal watermark produces scattered low-confidence detections (pat, 69, a to, aça 8, BRAU, Forpk, …). The grid projection places them in the middle of otherwise-correct lines, splitting a single continuous typewritten line into column-like fragments with large gaps:

R-8-2559 - C'edula Rural Hipotecária.- Financiado
do Brasil s/A.- Emitentes: [NAME]                pat     Marco An-
                                                    69
tônio [NAME] e Paulo Roberto [NAME]           a to        ta e lu
                                              aça 8    a  do paga
                                                      leferido:UC:
98.269,91.- Taxa de juros: 10% a.a.- Forpk Bens           prestações
vencíveis em 01.12.87, 01.12.88 e 01.12              ecados: O imove
retro matriculado.- HipOTECa CEDULAR     BRAU

Expected (what a naive (y, x) sort of the same OCR boxes produces — readable):

R-8-2559 - Cédula Rural Hipotecária.- Financiador: [BANK] S/A.- Emitentes: [NAME],
[NAME] e [NAME].- ... Valor do crédito deferido: Cz$ 98.269,91.- Taxa de juros:
10% a.a.- ... HIPOTECA CEDULAR DE 1º GRAU SEM CONCORRÊNCIA DE TERCEIROS.

As measured in #284, the underlying OCR boxes are fine (91% of boxes with confidence ≥ 0.85, correct bboxes) — this is purely an assembly/ordering effect.

Suggested direction

The interfering fragments share two properties: low confidence and no horizontal neighbors on the same text line (they are isolated specks from the watermark). Possible mitigations:

• Don't let isolated low-confidence OCR boxes create anchors/column splits — treat them as floating items (placed after the line, or dropped below, similar to what feat: Add HTTP OCR rotation handling #287 does for large rotated blobs).
• Or: when a detected "column gap" only exists because of a low-confidence fragment, merge the flanking segments back into one line.

Happy to test any branch against our document set (typewritten Brazilian registry pages with watermarks) — turnaround is quick on our side.

[logan-markewich]

Since you have the OCR server, you could filter these items yourself in the server before returning results. Either way, will try to take a look, but no garuntees honestly