Consolidate eval scripts into one

.gitignore

@@ -10,6 +10,7 @@ ENV/
 build/
 dist/
 *.egg-info/
+.venv
 
 # Data (never commit large datasets)
 data/raw/*

COMPREHENSIVE_DEMO.py

@@ -228,7 +228,7 @@ def demo_evaluation_framework():
     print_section("DEMO 5: Evaluation Framework")
 
     print("The evaluation framework can be run with:")
-    print("  python experiments/kavya_evaluation_framework.py")
+    print("  python experiments/run_evaluation.py --eval-set path/to/eval_set.json")
     print("\nIt provides:")
     print("  ✅ WER/CER calculation")
     print("  ✅ Error analysis")

README.md

@@ -77,9 +77,8 @@ Adaptive-Self-Learning-Agentic-AI-System/
 │   ├── test_agent.py             # Test agent functionality
 │   ├── test_api.py               # Test API endpoints
 │   ├── test_data_management.py   # Test data management
-│   ├── kavya_evaluation_framework.py  # Comprehensive evaluation
+│   ├── run_evaluation.py         # Evaluate dataset on baseline + improved models
 │   ├── evaluate_models.py        # Model evaluation
-│   ├── run_benchmark.py          # Performance benchmarking
 │   ├── visualize_evaluation_results.py # Generate charts
 │   └── example_usage.py          # Usage examples
 │
@@ -267,27 +266,25 @@ system.record_training_performance(
 report = system.generate_comprehensive_report()
 ```
 
-### 4. Evaluation Framework (`experiments/kavya_evaluation_framework.py`)
+### 4. Evaluation (`experiments/run_evaluation.py`)
 
-Comprehensive evaluation with metrics and visualization.
+Evaluate any dataset (audio + ground-truth references) on the baseline and on improved (fine-tuned) models if they exist.
 
 **Features:**
-- WER/CER calculation
-- Error analysis
-- Performance benchmarking
-- Visualization generation
+- WER/CER for baseline (Whisper) and all fine-tuned Wav2Vec2 versions
+- Optional latency/throughput benchmark (`--benchmark`)
 
 **Usage:**
-```python
-from experiments.kavya_evaluation_framework import EvaluationFramework
+```bash
+# Eval set file (JSON/JSONL/CSV with audio_path + reference)
+python experiments/run_evaluation.py --eval-set path/to/eval_set.json
 
-framework = EvaluationFramework(model_name="whisper")
-results = framework.run_comprehensive_evaluation(
-    eval_datasets=["data/processed/test_dataset"],
-    output_report=True
-)
+# Or audio directory + refs file
+python experiments/run_evaluation.py --audio-dir data/recordings_for_test --refs path/to/refs.json
 ```
 
+See **`docs/EVALUATION_SUMMARY.md`** for full options and examples.
+
 ## 🌐 Running the System
 
 ### 1. Baseline API (Simple Transcription)
@@ -342,37 +339,24 @@ curl -X POST "http://localhost:8000/transcribe" \
   -F "file=@data/test_audio/test_1.wav"
 ```
 
-### 3. Evaluation & Benchmarking
+### 3. Evaluation
 
-#### Run Comprehensive Evaluation
+#### Run evaluation (baseline + improved models)
 ```bash
-cd experiments
-python kavya_evaluation_framework.py
+python experiments/run_evaluation.py --eval-set path/to/eval_set.json
 ```
 
-Output:
-- `evaluation_outputs/evaluation_report.json` - Detailed results
-- `evaluation_outputs/evaluation_summary.json` - Summary metrics
-- `docs/EVALUATION_SUMMARY.md` - Human-readable report
-- `evaluation_outputs/visualizations/` - Charts and graphs
-
-#### Run Benchmark Tests
+With latency/throughput benchmark:
 ```bash
-python experiments/run_benchmark.py
+python experiments/run_evaluation.py --eval-set path/to/eval_set.json --benchmark
 ```
 
-Output:
-- `evaluation_outputs/benchmark_report.json` - Performance metrics
-
-#### Visualize Results
-```bash
-python experiments/visualize_evaluation_results.py
-```
+Output (in `experiments/evaluation_outputs/`):
+- `evaluation_report.json` - Baseline and improved-model WER/CER
+- `evaluation_report.txt` - Short summary
+- `benchmark_report.json` - If `--benchmark` was used
 
-Generates:
-- WER/CER comparison charts
-- Error distribution histograms
-- Comprehensive dashboards
+See **`docs/EVALUATION_SUMMARY.md`** for input format (eval set, audio-dir + refs, or --gold-from-llm) and all options.
 
 ### 4. Testing Components
 
@@ -630,25 +614,15 @@ if user_correction:
     data_system.add_correction(case_id, user_correction)
 ```
 
-### Workflow 2: Model Evaluation & Comparison
+### Workflow 2: Model Evaluation
 
-```python
-from experiments.kavya_evaluation_framework import EvaluationFramework
-
-# Evaluate baseline model
-framework = EvaluationFramework(model_name="whisper")
-results = framework.run_comprehensive_evaluation(
-    eval_datasets=["data/processed/test_dataset"]
-)
-
-# Generate visualizations
-framework.generate_visualizations()
-
-# Get metrics
-print(f"WER: {results['overall_metrics']['mean_wer']:.4f}")
-print(f"CER: {results['overall_metrics']['mean_cer']:.4f}")
+```bash
+# Run evaluation on a dataset (eval set = JSON/JSONL/CSV with audio_path + reference)
+python experiments/run_evaluation.py --eval-set path/to/eval_set.json
 ```
 
+Then read `experiments/evaluation_outputs/evaluation_report.json` for baseline and improved-model WER/CER. See `docs/EVALUATION_SUMMARY.md` for details.
+
 ### Workflow 3: Fine-tuning Pipeline
 
 ```python