A sophisticated multimodal AI-powered healthcare assistant that provides preliminary health analysis based on text descriptions, medical images, and audio recordings. Built with OpenAI's latest models and deployed on Hugging Face Spaces.
Try the Healthcare AI Assistant
This is a demonstration tool for educational purposes only. It does not provide medical advice and should not be used as a substitute for professional medical consultation, diagnosis, or treatment. Always seek advice from qualified healthcare professionals.
- Text Input: Describe symptoms in natural language
- Image Analysis: Upload medical photos (rashes, wounds, swelling, etc.)
- Audio Input: Voice descriptions of symptoms using speech-to-text
- Audio-Only Mode: Complete analysis from voice input alone
- Patient Information: Name, age, and gender integration
- Personalized Responses: AI addresses patients by name
- Demographics-Aware: Age and gender-relevant medical insights
- Contextual Analysis: Tailored recommendations based on patient profile
- GPT-4: Comprehensive health analysis and recommendations
- GPT-4 Vision: Medical image interpretation
- Whisper AI: High-accuracy speech transcription
- Real-time Processing: Instant analysis and feedback
- Gradio 4.0+: Modern web interface with healthcare theme
- HTML/CSS: Custom styling for medical application aesthetics
- JavaScript: Interactive elements and real-time updates
- Python 3.11+: Core application language
- OpenAI API v1.0+: Advanced AI models integration
- GPT-4: Text analysis and medical reasoning
- GPT-4 Vision: Medical image analysis
- Whisper: Speech-to-text transcription
- PIL (Pillow): Image processing and format conversion
- Librosa: Audio processing and analysis
- NumPy: Numerical computing and array operations
- Hugging Face Spaces: Cloud deployment platform
- Git: Version control and collaboration
- Virtual Environments: Isolated Python environments
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ User Input │ │ Gradio UI │ │ OpenAI APIs │
│ │ │ │ │ │
│ • Text │───▶│ • Interface │───▶│ • GPT-4 │
│ • Image │ │ • Validation │ │ • GPT-4 Vision │
│ • Audio │ │ • Processing │ │ • Whisper │
│ • Demographics │ │ │ │ │
└─────────────────┘ └──────────────────┘ └─────────────────┘
│ │ │
│ ▼ │
│ ┌──────────────────┐ │
│ │ Data Ingestion │ │
│ │ │ │
│ │ • Text Parser │ │
└──────────────│ • Image Handler │◀───────────┘
│ • Audio Processor│
│ │
└──────────────────┘
│
▼
┌──────────────────┐
│ AI Analysis & │
│ Response Gen. │
│ │
│ • Medical Insights│
│ • Personalization│
│ • Recommendations│
└──────────────────┘
healthcare-ai-assistant/
├── app.py # Main Gradio application
├── requirements.txt # Python dependencies
├── .env.example # Environment variables template
├── .gitignore # Git ignore rules
├── LICENSE # MIT License
├── README.md # This file
└── src/
├── openai_integration.py # OpenAI API client
├── ingestion.py # Multimodal data processing
└── preprocess.py # Data preprocessing utilities
- Python 3.11 or higher
- OpenAI API key
- Git
- Clone the repository
git clone https://github.com/raiigauravv/heathcare-ai-assistant.git
cd heathcare-ai-assistant- Create virtual environment
python -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate- Install dependencies
pip install -r requirements.txt- Set up environment variables
cp .env.example .env
# Edit .env and add your OpenAI API key- Run the application
python app.pyThe application will be available at http://localhost:7860
Create a .env file in the root directory:
OPENAI_API_KEY=your_openai_api_key_here- Images: JPG, JPEG, PNG, BMP, TIFF, WEBP
- Audio: MP3, WAV, M4A, FLAC, OGG
Patient: "John Doe"
Age: "25"
Gender: "Male"
Symptoms: "I have been experiencing severe headaches for the past 3 days, along with sensitivity to light and nausea."
Patient: "Jane Smith"
Age: "35"
Gender: "Female"
Symptoms: "Strange rash appeared on my arm yesterday"
Image: [Upload photo of rash]
Patient: "Mike Johnson"
Age: "45"
Gender: "Male"
Audio: [Record voice describing symptoms]
- No Data Storage: Patient information is not stored permanently
- Secure API Calls: All communications encrypted via HTTPS
- Environment Variables: Sensitive keys stored securely
- Open Source: Full transparency of code and processes
- Fork the repository
- Create a feature branch (
git checkout -b feature/amazing-feature) - Commit your changes (
git commit -m 'Add amazing feature') - Push to the branch (
git push origin feature/amazing-feature) - Open a Pull Request
This project is licensed under the MIT License - see the LICENSE file for details.
IMPORTANT: This application is for educational and demonstration purposes only. It:
- ❌ Does NOT provide medical diagnosis
- ❌ Does NOT replace professional medical advice
- ❌ Should NOT be used for emergency medical situations
- ✅ Provides general health information only
- ✅ Encourages consultation with healthcare professionals
In case of medical emergency, contact emergency services immediately.
- OpenAI for providing advanced AI models
- Hugging Face for the deployment platform
- Gradio for the excellent UI framework
- Python Community for the amazing libraries
For questions, issues, or suggestions:
- GitHub Issues: Create an issue
- Live Demo: Try it out
⭐ If you find this project helpful, please consider giving it a star on GitHub!