Skip to content

About

Python API for Patchfinder

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

PyPatchFinder

Advanced document field extraction using vision-language models and OCR technology. PyPatchFinder combines the power of transformer-based vision models with intelligent patch-based analysis to extract structured information from complex documents.

Features

  • Multi-Model Support: Configurable Hugging Face models with Phi-3 Vision as default
  • Flexible OCR: EasyOCR integration with custom OCR support
  • Patch-Based Analysis: Intelligent document tiling for improved accuracy
  • Multiple Data Types: Support for numerical, date, and text field extraction
  • Real Confidence Scoring: Geometric mean of model token probabilities for accurate confidence assessment
  • Batch Processing: Optional multi-field extraction for efficient processing of multiple fields
  • Simple Single-Field Extraction: Clean, straightforward single field extraction without complexity
  • Spatial Localization: Field location detection with bounding box visualization
  • Fuzzy Matching: Levenshtein distance-based text matching for OCR error correction

Installation

Basic Installation (CPU/GPU compatible)

git clone https://github.com/CATALOG-Historic-Records/pypatchfinder.git
cd pypatchfinder
pip install -e .

GPU Installation with Flash Attention (Optional)

For enhanced GPU performance with CUDA support:

git clone https://github.com/CATALOG-Historic-Records/pypatchfinder.git
cd pypatchfinder
pip install -e ".[gpu]"

Note: Flash attention requires CUDA-capable GPU and CUDA toolkit. If installation fails, the package will work without it using standard attention mechanisms.

Development Installation

git clone https://github.com/CATALOG-Historic-Records/pypatchfinder.git
cd pypatchfinder
pip install -e ".[dev]"
# Optional: Add GPU support
pip install -e ".[dev,gpu]"

Compatibility Note

This package uses Phi-3.5-vision-instruct. The newer model provides:

  • ✅ Reliable batch processing for multiple field extraction
  • ✅ Multi-frame image support for better context understanding
  • ✅ ~1.1x faster inference with improved stability and accuracy
  • ✅ Better document understanding and field extraction accuracy
  • ✅ Fixes previous batch processing hang issues

Key improvements over Phi-3-vision:

  • More stable inference without hanging
  • Better accuracy on complex documents
  • Proper GPU utilization
  • Consistent performance across different field types

GPU vs CPU Performance

PyPatchFinder works on both CPU and GPU with automatic fallback:

  • CPU Mode: Works out of the box on any system, slower inference
  • GPU Mode: Automatic when CUDA is available, significantly faster
  • GPU + Flash Attention: Optional enhanced performance with pip install -e ".[gpu]"

The package automatically detects available hardware and uses the best option available.

Tested Dependencies

The package has been tested and verified with:

torch==2.7.1
torchvision==0.22.1
transformers==4.43.0

Note: PyPatchFinder requires transformers==4.43.0 for optimal Phi-3.5-vision-instruct support!

Quick Start

from pypatchfinder import PatchFinder
from PIL import Image

# Initialize with default settings
finder = PatchFinder()

# Load your document image
image = Image.open("document.pdf")

# Extract a single field - simple and straightforward
result = finder.extract_field(
    image=image,
    field_name="county",
    data_type="text",
    num_patches=9
)

print(f"Extracted value: {result.value}")
print(f"Confidence: {result.confidence:.3f}")
print(f"Location: {result.location}")  # (left, top, width, height) bounding box

# For multiple fields, you can use batch processing or individual extraction
# Individual extraction (recommended for simplicity):
county = finder.extract_field(image, "county", "text")
date = finder.extract_field(image, "date", "date") 
amount = finder.extract_field(image, "amount", "numerical")

# Or batch processing (if you prefer):
fields = [("county", "text"), ("date", "date"), ("amount", "numerical")]
batch_results = finder.batch_extract(image, fields, num_patches=9)
for field_name, result in batch_results.items():
    print(f"{field_name}: {result.value} (confidence: {result.confidence:.3f})")

Usage

Single Field Extraction

from pypatchfinder import PatchFinder
from PIL import Image

finder = PatchFinder()

# Extract a single field (automatically optimized)
result = finder.extract_field(
    image=Image.open("document.pdf"),
    field_name="county",
    data_type="text"
)

print(f"County: {result.value} (confidence: {result.confidence:.3f})")

Configuration Options

# Control early stopping (default: 0.95 confidence threshold)
result = finder.extract_field(image, "county", "text", early_stopping_threshold=0.9)

# Disable early stopping entirely
result = finder.extract_field(image, "county", "text", early_stopping_threshold=None)

# Control batch processing
results = finder.batch_extract(
    image, 
    fields, 
    batch_size=4,  # Manual batch size control
    enable_optimizations=False,  # Disable optimizations for debugging
    early_stopping_threshold=0.8  # Custom stopping threshold
)

# GPU OCR configuration (enabled by default with CPU fallback)
from pypatchfinder import OCRConfig
ocr_config = OCRConfig(gpu=True, confidence_threshold=0.6)
finder = PatchFinder(ocr_config=ocr_config)

Advanced Usage

Custom Model Configuration

from transformers import AutoModelForCausalLM, AutoProcessor
from pypatchfinder import PatchFinder

# Load custom model
model = AutoModelForCausalLM.from_pretrained("your-custom-model")
processor = AutoProcessor.from_pretrained("your-custom-model")

# Initialize with custom model
finder = PatchFinder(
    model=model,
    processor=processor,
    device="cuda"
)

Custom OCR Configuration

import easyocr
from pypatchfinder import PatchFinder

# Configure custom OCR
reader = easyocr.Reader(['en'], gpu=True)

finder = PatchFinder(
    ocr_reader=reader,
    ocr_confidence_threshold=0.7
)

Field Extraction with Different Data Types

# Extract numerical field
depth_result = finder.extract_field(
    image=image,
    field_name="total_depth",
    data_type="numerical",
    num_patches=12
)

# Extract date field
date_result = finder.extract_field(
    image=image,
    field_name="completion_date",
    data_type="date",
    num_patches=6
)

# Extract text field
county_result = finder.extract_field(
    image=image,
    field_name="county",
    data_type="text",
    num_patches=9
)

API Reference

PatchFinder Class

The main class for document field extraction.

Parameters

  • model: Hugging Face model for vision-language processing (default: Phi-3 Vision)
  • processor: Model processor (default: Phi-3 processor)
  • device: Device for model inference (default: "auto")
  • ocr_reader: OCR reader instance (default: EasyOCR)
  • ocr_confidence_threshold: Minimum OCR confidence (default: 0.5)
  • fuzzy_similarity_threshold: Minimum similarity for fuzzy matching (default: 0.6)
  • max_edit_distance: Maximum Levenshtein distance for fuzzy matching (default: 3)

Methods

  • extract_field(image, field_name, data_type, num_patches=9): Extract field value
  • create_patches(image, num_patches=9): Create document patches
  • find_best_patch(patches, field_name): Find best patch for field extraction
  • plot_patches(patches): Visualize document patches
  • plot_field_location(image, result): Visualize field location

Data Types

  • "text": General text fields (e.g., county names, operator names)
  • "numerical": Numerical values (e.g., depths, pressures, temperatures)
  • "date": Date fields in various formats

Examples

Basic Field Extraction

from pypatchfinder import PatchFinder
from PIL import Image

finder = PatchFinder()

# Extract county information
result = finder.extract_field(
    image=Image.open("well_report.pdf"),
    field_name="county",
    data_type="text"
)

print(f"County: {result.value}")
print(f"Confidence: {result.confidence:.2f}")

Visualization

# Plot document patches
finder.plot_patches(patches)

# Plot field location with bounding box (GUI mode)
finder.plot_field_location(image, result, show_plot=True)

# Non-GUI mode for scripts/servers (recommended)
fig = finder.plot_field_location(
    image, result, 
    show_plot=False, 
    return_figure=True
)
fig.savefig("field_location.png", dpi=150, bbox_inches='tight')
plt.close(fig)  # Clean up memory

Memory Management

PyPatchFinder includes optional manual memory cleanup methods:

# Option 1: Let Python/CUDA handle cleanup automatically (recommended)
finder = PatchFinder()
result = finder.extract_field(image, "county", "text")
# Memory will be cleaned up automatically when finder goes out of scope

# Option 2: Explicit cleanup (if you want manual control)
finder = PatchFinder()
# ... use finder ...
finder.cleanup()  # Explicitly free GPU memory

# Option 3: Context manager (automatically calls cleanup on exit)
with PatchFinder() as finder:
    result = finder.extract_field(image, "county", "text")
    # Memory cleaned up when exiting context

Note: Manual cleanup via cleanup() or context managers is optional. Python and CUDA generally handle memory management well automatically. Use manual cleanup only if you need explicit control over when resources are freed.

Contributing

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/amazing-feature)
  3. Commit your changes (git commit -m 'Add amazing feature')
  4. Push to the branch (git push origin feature/amazing-feature)
  5. Open a Pull Request

Testing

Run the test suite:

pytest tests/

Run with coverage:

pytest tests/ --cov=pypatchfinder

License

This project is licensed under the MIT License - see the LICENSE file for details.

Citation

If you use PyPatchFinder in your research, please cite:

@software{pypatchfinder,
  title={PyPatchFinder: Advanced Document Field Extraction},
  author={CATALOG Historic Records},
  year={2024},
  url={https://github.com/CATALOG-Historic-Records/pypatchfinder}
}

About

Python API for Patchfinder

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages