SmartReceipt: An AI-Powered Restaurant Spending Tracker

ECE 570 — Track 2: ProductPrototype
Anonymous Author(s)

SmartReceipt is a prototype system that lets a user upload a restaurant receipt image and automatically extracts:

• Vendor information
• Item names, prices, and quantities
• Beverage types (e.g., soda, alcohol, water)
• Gratuity amount (if present)

The system logs all parsed receipts into a CSV file and generates basic analytics plots for vendor type visits, total spending by vendor type, and beverage spending.

---

1. Repository Structure

.
├── SmartReceipt.py              # Main end-to-end pipeline (OCR + parsing + analytics)
├── parsed_receipts.csv          # Database for all input receipts for cumulative analysis
├── requirements.txt             # Python dependencies
├── README.md                    # This file
├── final_t5_base_receipt_parser/# fine-tuned T5-base checkpoint
└── data/
    └── README_DATA.md           # Notes / instructions for downloading the dataset

---

2. Installation & Setup

This project requires Python 3.9+ and the dependencies listed in requirements.txt.

1. Create and Activate a Virtual Environment

It is highly recommended to use a virtual environment.

Linux / macOS

python3 -m venv venv
source venv/bin/activate

Windows (PowerShell)

python -m venv venv
venv\Scripts\activate

2. Install Dependencies

All Python dependencies are listed in the requirements.txt file.

pip install -r requirements.txt

This installs:

• torch — Required for both DocTR and the T5 model
• python-doctr[torch] — OCR engine
• transformers, sentencepiece — T5 model parsing
• pandas, numpy — Data handling
• matplotlib — Plotting
• opencv-python — Image preprocessing
• ...and other utilities.

3. Installing Tkinter (Required for File Selection GUI)

Tkinter is used for the file dialog (Tk().withdraw() + filedialog.askopenfilename).
Install it if missing:

Ubuntu / Debian

sudo apt-get update
sudo apt-get install python3-tk

Fedora

sudo dnf install python3-tkinter

macOS

• Included with system Python
• For Homebrew Python:

  brew install python-tk

Windows

• Comes bundled with standard Python installers
• If missing, reinstall Python from https://www.python.org/downloads/windows/

---

3. Model Setup

The fine-tuned parsing model is included in the models/ folder.
Ensure the directory structure matches the repository tree above.

---

4. Running the Program

Run the main script:

python SmartReceipt.py

The Program Workflow

1. Opens a file dialog window
2. User selects a receipt image (.jpg, .jpeg, .png, .tiff, .bmp)
3. Script performs:
   • DocTR OCR extraction
   • T5 receipt parsing
   • Extraction (items, prices, quantities, beverage types, gratuity)
4. Appends parsed data to parsed_receipts.csv
5. Generates analytics plots:
   • Vendor type distribution (pie chart)
   • Spending by vendor type (bar chart)
   • Beverage spending (bar chart)

---

5. Output Files

• parsed_receipts.csv: Cumulative log of parsed receipts
• Matplotlib window: Visual analytics popup
• Console output: Parsed metadata fields and progress logs

---

Code Authorship Statement

All code contained in this folder was written entirely by the project author for the SmartReceipt system.
No external source code, templates, or previously published implementations were copied or adapted.
Any third-party libraries used (e.g., DocTR, Transformers, PyTorch, OpenCV) are referenced only through their official public APIs.

This codebase represents original work created specifically for this ECE 570 course project.