Multilingual Voice Mobile Assistant 🎤📱

A sophisticated AI-powered multilingual voice assistant for mobile phone sales, featuring real-time speech-to-text, multilingual response generation, and text-to-speech capabilities.

🌟 Features

🗣️ Voice Interaction

• Real-time Speech-to-Text in 15+ languages
• Multilingual Text-to-Speech output with Murf AI integration
• Voice-controlled mobile shopping experience
• Browser-based voice recognition

🌍 Multilingual Support

• English (Default)
• Hindi (हिंदी) - hi-IN-kabir voice
• Korean (한국어) - ko-KR-gyeong voice
• Japanese (日本語) - ja-JP-kenji voice
• Chinese (中文) - zh-CN-tao voice
• Spanish (Español) - es-ES-elvira voice
• 10+ additional languages with native voice outputs

🤖 AI Capabilities

• Gemini AI-powered conversational agent
• Smart product search with Pinecone vector database
• Personalized deal generation based on conversation context
• Natural language understanding for mobile sales

🛍️ E-commerce Features

• Mobile product catalog with detailed specifications
• Intelligent product recommendations
• Dynamic discount generation
• Visual product displays with images
• Advanced negotiation capabilities that create personalized shopping experiences

🙏 Acknowledgments

This project would not have been possible without the following amazing technologies and services:

🤖 AI & Machine Learning

• Google Gemini AI - For providing powerful multilingual language capabilities and intelligent conversational AI
• Murf AI - For high-quality, natural-sounding text-to-speech services across multiple languages
• Pinecone - For robust vector database infrastructure enabling intelligent product search

🛠️ Frameworks & Libraries

• LangChain - For the comprehensive AI agent framework that powers our conversational workflows
• FastAPI - For the high-performance, modern web framework that makes our API development efficient
• React - For the declarative and component-based frontend library
• Vite - For the fast build tool and development server

🗣️ Speech Technologies

• Web Speech API - For browser-based speech recognition capabilities
• Google Cloud Speech-to-Text - For accurate speech recognition technology

📊 Data & Infrastructure

• Pydantic - For data validation and settings management using Python type annotations
• Uvicorn - For the lightning-fast ASGI server implementation
• Hugging Face - For embeddings models and transformer resources

Environment Variables

To run this project, you will need to add the following environment variables to your .env file

MURF_API_KEY

GOOGLE_API_KEY

PINECONE_API_KEY

MURF_VOICE_ID (optional - defaults to en-US-natalie)

PINECONE_INDEX_NAME (optional - defaults to mobile-phones)

Installation

Clone the repository

git clone https://github.com/codewhyofficial/building-on-murf-challenge.git
cd building-on-murf-challenge

Backend setup

cd backend
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt

Frontend setup

cd ../frontend
npm install

Project Structure

multilingual-voice-assistant/
├── backend/                    # FastAPI Backend Application
│   ├── agent.py               # AI Agent Workflows
│   ├── tools.py                 # LangChain Tools
│   ├── vector_store.py          # Pinecone Integration
│   ├── schemas.py               # Pydantic Models
│   ├── main.py                # FastAPI Application Entry Point
│   ├── config.py              # Configuration Settings
│   ├── requirements.txt       # Python Dependencies
│   └── .env                   # Environment Variables
│
├── frontend/                  # React Frontend Application
│   ├── src/
│   │   ├── components/        # React Components
│   │   ├── hooks/             # Custom React Hooks
│   │   ├── utils/             # Utility Functions
│   │   ├── App.jsx            # Main App Component
│   │   └── main.jsx           # Application Entry Point
│   ├── package.json
│   ├── vite.config.js
│   └── index.html
│
├── data/                      # Sample Data Files
├── .gitignore
├── Dockerfile
├── docker-compose.yml
└── README.md

🤝 Contributing

We welcome contributions from the community! Whether you're fixing bugs, adding new features, or improving documentation, your help is appreciated.