Advanced document field extraction using vision-language models and OCR technology. PyPatchFinder combines the power of transformer-based vision models with intelligent patch-based analysis to extract structured information from complex documents.
- Multi-Model Support: Configurable Hugging Face models with Phi-3 Vision as default
- Flexible OCR: EasyOCR integration with custom OCR support
- Patch-Based Analysis: Intelligent document tiling for improved accuracy
- Multiple Data Types: Support for numerical, date, and text field extraction
- Real Confidence Scoring: Geometric mean of model token probabilities for accurate confidence assessment
- Batch Processing: Optional multi-field extraction for efficient processing of multiple fields
- Simple Single-Field Extraction: Clean, straightforward single field extraction without complexity
- Spatial Localization: Field location detection with bounding box visualization
- Fuzzy Matching: Levenshtein distance-based text matching for OCR error correction
git clone https://github.com/CATALOG-Historic-Records/pypatchfinder.git
cd pypatchfinder
pip install -e .For enhanced GPU performance with CUDA support:
git clone https://github.com/CATALOG-Historic-Records/pypatchfinder.git
cd pypatchfinder
pip install -e ".[gpu]"Note: Flash attention requires CUDA-capable GPU and CUDA toolkit. If installation fails, the package will work without it using standard attention mechanisms.
git clone https://github.com/CATALOG-Historic-Records/pypatchfinder.git
cd pypatchfinder
pip install -e ".[dev]"
# Optional: Add GPU support
pip install -e ".[dev,gpu]"This package uses Phi-3.5-vision-instruct. The newer model provides:
- ✅ Reliable batch processing for multiple field extraction
- ✅ Multi-frame image support for better context understanding
- ✅ ~1.1x faster inference with improved stability and accuracy
- ✅ Better document understanding and field extraction accuracy
- ✅ Fixes previous batch processing hang issues
Key improvements over Phi-3-vision:
- More stable inference without hanging
- Better accuracy on complex documents
- Proper GPU utilization
- Consistent performance across different field types
PyPatchFinder works on both CPU and GPU with automatic fallback:
- CPU Mode: Works out of the box on any system, slower inference
- GPU Mode: Automatic when CUDA is available, significantly faster
- GPU + Flash Attention: Optional enhanced performance with
pip install -e ".[gpu]"
The package automatically detects available hardware and uses the best option available.
The package has been tested and verified with:
torch==2.7.1
torchvision==0.22.1
transformers==4.43.0
Note: PyPatchFinder requires transformers==4.43.0 for optimal Phi-3.5-vision-instruct support!
from pypatchfinder import PatchFinder
from PIL import Image
# Initialize with default settings
finder = PatchFinder()
# Load your document image
image = Image.open("document.pdf")
# Extract a single field - simple and straightforward
result = finder.extract_field(
image=image,
field_name="county",
data_type="text",
num_patches=9
)
print(f"Extracted value: {result.value}")
print(f"Confidence: {result.confidence:.3f}")
print(f"Location: {result.location}") # (left, top, width, height) bounding box
# For multiple fields, you can use batch processing or individual extraction
# Individual extraction (recommended for simplicity):
county = finder.extract_field(image, "county", "text")
date = finder.extract_field(image, "date", "date")
amount = finder.extract_field(image, "amount", "numerical")
# Or batch processing (if you prefer):
fields = [("county", "text"), ("date", "date"), ("amount", "numerical")]
batch_results = finder.batch_extract(image, fields, num_patches=9)
for field_name, result in batch_results.items():
print(f"{field_name}: {result.value} (confidence: {result.confidence:.3f})")from pypatchfinder import PatchFinder
from PIL import Image
finder = PatchFinder()
# Extract a single field (automatically optimized)
result = finder.extract_field(
image=Image.open("document.pdf"),
field_name="county",
data_type="text"
)
print(f"County: {result.value} (confidence: {result.confidence:.3f})")# Control early stopping (default: 0.95 confidence threshold)
result = finder.extract_field(image, "county", "text", early_stopping_threshold=0.9)
# Disable early stopping entirely
result = finder.extract_field(image, "county", "text", early_stopping_threshold=None)
# Control batch processing
results = finder.batch_extract(
image,
fields,
batch_size=4, # Manual batch size control
enable_optimizations=False, # Disable optimizations for debugging
early_stopping_threshold=0.8 # Custom stopping threshold
)
# GPU OCR configuration (enabled by default with CPU fallback)
from pypatchfinder import OCRConfig
ocr_config = OCRConfig(gpu=True, confidence_threshold=0.6)
finder = PatchFinder(ocr_config=ocr_config)from transformers import AutoModelForCausalLM, AutoProcessor
from pypatchfinder import PatchFinder
# Load custom model
model = AutoModelForCausalLM.from_pretrained("your-custom-model")
processor = AutoProcessor.from_pretrained("your-custom-model")
# Initialize with custom model
finder = PatchFinder(
model=model,
processor=processor,
device="cuda"
)import easyocr
from pypatchfinder import PatchFinder
# Configure custom OCR
reader = easyocr.Reader(['en'], gpu=True)
finder = PatchFinder(
ocr_reader=reader,
ocr_confidence_threshold=0.7
)# Extract numerical field
depth_result = finder.extract_field(
image=image,
field_name="total_depth",
data_type="numerical",
num_patches=12
)
# Extract date field
date_result = finder.extract_field(
image=image,
field_name="completion_date",
data_type="date",
num_patches=6
)
# Extract text field
county_result = finder.extract_field(
image=image,
field_name="county",
data_type="text",
num_patches=9
)The main class for document field extraction.
model: Hugging Face model for vision-language processing (default: Phi-3 Vision)processor: Model processor (default: Phi-3 processor)device: Device for model inference (default: "auto")ocr_reader: OCR reader instance (default: EasyOCR)ocr_confidence_threshold: Minimum OCR confidence (default: 0.5)fuzzy_similarity_threshold: Minimum similarity for fuzzy matching (default: 0.6)max_edit_distance: Maximum Levenshtein distance for fuzzy matching (default: 3)
extract_field(image, field_name, data_type, num_patches=9): Extract field valuecreate_patches(image, num_patches=9): Create document patchesfind_best_patch(patches, field_name): Find best patch for field extractionplot_patches(patches): Visualize document patchesplot_field_location(image, result): Visualize field location
"text": General text fields (e.g., county names, operator names)"numerical": Numerical values (e.g., depths, pressures, temperatures)"date": Date fields in various formats
from pypatchfinder import PatchFinder
from PIL import Image
finder = PatchFinder()
# Extract county information
result = finder.extract_field(
image=Image.open("well_report.pdf"),
field_name="county",
data_type="text"
)
print(f"County: {result.value}")
print(f"Confidence: {result.confidence:.2f}")# Plot document patches
finder.plot_patches(patches)
# Plot field location with bounding box (GUI mode)
finder.plot_field_location(image, result, show_plot=True)
# Non-GUI mode for scripts/servers (recommended)
fig = finder.plot_field_location(
image, result,
show_plot=False,
return_figure=True
)
fig.savefig("field_location.png", dpi=150, bbox_inches='tight')
plt.close(fig) # Clean up memoryPyPatchFinder includes optional manual memory cleanup methods:
# Option 1: Let Python/CUDA handle cleanup automatically (recommended)
finder = PatchFinder()
result = finder.extract_field(image, "county", "text")
# Memory will be cleaned up automatically when finder goes out of scope
# Option 2: Explicit cleanup (if you want manual control)
finder = PatchFinder()
# ... use finder ...
finder.cleanup() # Explicitly free GPU memory
# Option 3: Context manager (automatically calls cleanup on exit)
with PatchFinder() as finder:
result = finder.extract_field(image, "county", "text")
# Memory cleaned up when exiting contextNote: Manual cleanup via cleanup() or context managers is optional. Python and CUDA generally handle memory management well automatically. Use manual cleanup only if you need explicit control over when resources are freed.
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
Run the test suite:
pytest tests/Run with coverage:
pytest tests/ --cov=pypatchfinderThis project is licensed under the MIT License - see the LICENSE file for details.
If you use PyPatchFinder in your research, please cite:
@software{pypatchfinder,
title={PyPatchFinder: Advanced Document Field Extraction},
author={CATALOG Historic Records},
year={2024},
url={https://github.com/CATALOG-Historic-Records/pypatchfinder}
}