From 3745fa70f4c272ff6e02af8887a19fae26b21e76 Mon Sep 17 00:00:00 2001 From: Akshaya4225 Date: Sat, 28 Mar 2026 11:23:41 +0530 Subject: [PATCH] Enhance project_AutoEIT.md with evaluation strategy Added evaluation strategy and dataset considerations for AutoEIT project. --- _gsocprojects/2026/project_AutoEIT.md | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/_gsocprojects/2026/project_AutoEIT.md b/_gsocprojects/2026/project_AutoEIT.md index 2b898713..95b8a2d9 100644 --- a/_gsocprojects/2026/project_AutoEIT.md +++ b/_gsocprojects/2026/project_AutoEIT.md @@ -6,5 +6,27 @@ description: | AutoEIT is an applied machine learning project focused on automating the scoring of the Elicited Imitation Task (EIT), a research tool used to measure global language proficiency. The EIT is widely respected and is available for free in several languages, but the current workflow requires manual audio transcription and human scoring—slow, tedious, and error‑prone. This project aims to build an end‑to‑end system that will: Process raw audio files, perform accurate voice‑to‑text transcription, and automatically evaluate responses using a standardized scoring rubric. --- +## Evaluation Strategy + +To ensure the effectiveness of the AutoEIT system, the following evaluation approach can be used: + +- **Transcription Accuracy** + - Measure Word Error Rate (WER) to evaluate speech-to-text performance + +- **Scoring Accuracy** + - Compare automated scores with human-evaluated scores + +- **Robustness** + - Test system performance on noisy and diverse audio inputs + +- **Consistency** + - Ensure the system produces stable and repeatable results across multiple runs + + --- + ## Dataset Considerations + +- Use multilingual speech datasets +- Include diverse accents and speaking speeds +- Ensure balanced dataset for fair evaluation {% include gsoc_project.ext %}