Skip to content

fix(extraction): lower confidence for empty text output - #443

Open
yzxcj797 wants to merge 1 commit into
firecrawl:mainfrom
yzxcj797:fix/272-empty-extraction-confidence
Open

fix(extraction): lower confidence for empty text output#443
yzxcj797 wants to merge 1 commit into
firecrawl:mainfrom
yzxcj797:fix/272-empty-extraction-confidence

Conversation

@yzxcj797

@yzxcj797 yzxcj797 commented Aug 21, 2026

Copy link
Copy Markdown

Problem

Full processing can classify a PDF as TextBased, suppress its undecodable text, return empty Markdown, route pages to OCR, and still report confidence 1.0. That makes an unusable result look certain to callers.

Change

  • Reduce Full-mode result confidence to 0.0 when a TextBased PDF yields no non-whitespace Markdown.
  • Keep Analyze and DetectOnly confidence unchanged because those modes intentionally do not produce Markdown.
  • Document the result confidence field.
  • Cover suppressed output, Analyze mode, and sparse-but-usable text with regressions.

This addresses the confident-empty-extraction portion of #272.

Validation

  • New regression fails on pristine main and passes here.
  • cargo test --verbose: 1015 lib + 168 integration + 3 bin + 2 doc tests pass.
  • cargo clippy -- -D warnings and cargo clippy --features ocr -- -D warnings pass.
  • Root and WASM cargo fmt -- --check pass.
  • git diff --check passes.

Summary by cubic

Prevents overconfident empty extractions: in Full mode, TextBased PDFs that produce no usable Markdown now return confidence 0.0 instead of 1.0. This stops callers from treating unusable results as certain; Analyze and DetectOnly modes are unchanged.

  • Full-mode confidence is set to 0.0 when pdf_type is TextBased and the Markdown is empty or whitespace; adds a debug log for visibility.

  • Documents the PdfProcessResult.confidence semantics.

  • Adds integration tests covering suppressed output (now 0.0), Analyze mode retaining detector confidence (1.0), and sparse-but-usable text retaining 1.0.

  • Impact: No API changes. Callers that gate retries or fallbacks on confidence should treat 0.0 as an empty-extraction signal in Full mode.

Written for commit 19f2e53. Summary will update on new commits.

Review in cubic

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Shadow auto-approve: would auto-approve. Focused bug fix: empty Markdown from a TextBased PDF in Full mode now reports confidence 0.0 instead of 1.0, with the condition isolated and covered by tests. No public shape change, no rollout or config tradeoff, and the field's semantics are documented.

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant