ARCS: Another Review Classification System

ARCS is a modular Python command-line tool that analyzes customer reviews from CSV files and classifies them as positive, negative, or optionally neutral, based on a probability confidence threshold. It can also filter reviews by relevance to a specific topic using embedding similarity.

✨ Features

• 📊 Powerful Analysis: Process customer reviews from CSV files with high accuracy
• 🎯 Flexible Classification: Categorize reviews as positive, negative, or neutral based on configurable confidence thresholds
• 🧠 Multiple Models:
  • 🤖 BERT-based model using HuggingFace Transformers
  • 🔤 Simple keyword-based model for testing or lightweight applications
• 📁 Organized Output: Generate multiple CSV files with detailed classification results
• 🧩 Extensible Design: Easily add new sentiment classification models
• 🔍 Relevance Filtering: Filter reviews based on semantic relevance to a specific topic
• 📏 Embedding Similarity: Use sentence embeddings to compute similarity between reviews and topics

🚀 Installation

1. Clone the repository:

   git clone https://github.com/yourusername/ARCS.git
   cd ARCS

2. Create a virtual environment:

   python -m venv .venv

3. Activate the virtual environment:

   • Windows:

     .venv\Scripts\activate

   • macOS/Linux:

     source .venv/bin/activate

4. Install dependencies:

   pip install -r requirements.txt

📋 Usage

python main.py --input reviews.csv --threshold 0.75 --model bert --output-dir outputs --include-neutral

Command Line Arguments

Argument	Description	Default
--input	Path to input CSV file containing reviews	Required
--threshold	Confidence threshold for classification	0.75
--model	Model to use for sentiment classification (choices: "bert", "dummy")	"bert"
--output-dir	Directory to save output files	"outputs"
--include-neutral	Include neutral/uncertain reviews in output	False
--review-column	Name of the column containing review text	"review_text"
--topic	Topic for relevance filtering (enables relevance filtering if provided)	None
--relevance-threshold	Threshold for topic relevance filtering	0.4
--include-irrelevant-in-output	Include irrelevant reviews in output files (marked as relevant=False)	False

📊 Output Files

ARCS generates the following output files:

File	Description
all_reviews_with_confidence.csv	All reviews with sentiment label, confidence score, and relevance information
positive_reviews.csv	Reviews classified as positive with confidence ≥ threshold
negative_reviews.csv	Reviews classified as negative with confidence ≥ threshold
neutral_or_uncertain_reviews.csv	Reviews with confidence < threshold (only if --include-neutral is specified)
irrelevant_reviews.csv	Reviews filtered out due to low relevance to the topic (only if --topic is specified)

📂 Project Structure

│
├── main.py                 # CLI entry point, parses args, launches pipeline
├── config.py               # Global config values
├── requirements.txt        # Dependencies
├── example_reviews.csv     # Example CSV file with reviews
│
├── models/
│   ├── __init__.py
│   ├── sentiment_base.py   # Abstract base class for sentiment models
│   ├── bert_sentiment.py   # BERT-based sentiment classifier
│   └── dummy_sentiment.py  # Simple baseline classifier for testing
│
├── processing/
│   ├── __init__.py
│   ├── classifier_engine.py # Core logic for classification
│   └── io_handler.py        # CSV read/write utility functions
│
├── relevance/
│   ├── __init__.py
│   └── relevance_scorer.py  # Core logic for relevance filtering
│
├── tests/
│   ├── README.md           # Test documentation
│   ├── test_combined.py    # Test script for both relevance and sentiment
│   ├── test_relevance.py   # Test script for relevance filtering
│   ├── test_classifier.py  # Test script for sentiment classification
│   ├── example_diverse_reviews.csv # Sample diverse reviews for testing
│   ├── run_test.bat        # Windows batch file to run tests
│   ├── run_test.sh         # Shell script to run tests
│   └── outputs/            # Test output files go here
│
├── utils/
│   ├── __init__.py
│   └── text_utils.py        # Text cleaning and preprocessing
│
└── outputs/                 # Output CSV files go here

🧩 Extending with New Models

ARCS is designed to be easily extended with new sentiment models:

1. Create a new file in the models/ directory (e.g., my_model_sentiment.py)
2. Implement a class that inherits from SentimentModel
3. Implement the required methods: load(), predict(), and predict_batch()
4. Add the model to the choices in main.py
5. Add model configuration in config.py if needed

🔍 Examples

# Classify reviews using BERT model with default threshold
python main.py --input customer_reviews.csv

# Classify reviews using the dummy model with a custom threshold
python main.py --input customer_reviews.csv --model dummy --threshold 0.6

# Include neutral/uncertain reviews in the output
python main.py --input customer_reviews.csv --include-neutral

# Filter reviews by relevance to a specific topic before classification
python main.py --input customer_reviews.csv --topic "customer service" --relevance-threshold 0.5

# Filter reviews by relevance but include irrelevant reviews in output (marked as relevant=False)
python main.py --input customer_reviews.csv --topic "product quality" --include-irrelevant-in-output

🧪 Testing

The project includes several test scripts in the tests/ directory:

# Run the sentiment classification test
python tests/test_classifier.py

# Run the relevance filtering test
python tests/test_relevance.py

# Run the combined test (relevance filtering + sentiment classification)
python tests/test_combined.py

You can also use the provided batch/shell scripts to run the combined test:

• Windows: Double-click tests/run_test.bat
• Linux/Mac: Run ./tests/run_test.sh (make it executable first with chmod +x tests/run_test.sh)

For more details on testing, see the tests/README.md file.

🎨 User Interface

ARCS features a user-friendly command-line interface with improved readability:

• 🏷️ Clear Section Headers: Distinct headers and subheaders to organize output
• 🚥 Color-Coded Information: Colored output to highlight important information (with fallbacks for terminals without color support)
• 📏 Progress Indicators: Visual indicators for long-running tasks
• 📊 Formatted Statistics: Neatly formatted statistics with percentages
• 🔍 Structured Results: Clearly organized results for easy interpretation
• 🖥️ Cross-Platform Support: Works on Windows, macOS, and Linux terminals

Install the optional colorama package for better color support on Windows:

pip install colorama

📚 Documentation

ARCS includes comprehensive documentation in the docs/ directory:

• Usage Guides: Installation, CLI options, input/output formats, and examples
• Component Documentation: Architecture, models, relevance filtering, and UI
• Development Guides: Project structure, adding new models, and testing

To access the documentation, browse the Markdown files in the docs/ directory, starting with docs/README.md for an overview.

You can also view the documentation by navigating to the GitHub repository's docs folder, which renders the Markdown files with proper formatting.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.