Portkey's AI Gateway streamlines working with 100+ models via one fast API. Load balance, fallback between providers, plug-in middleware, and more. Battle-tested on 100B+ tokens.
Portkey's AI Gateway 🚀 - A TypeScript gateway to streamline working with 100+ models
✅ Universal API - Connect to 100+ models with OpenAI's API signature
✅ Load Balancing - Distribute requests across multiple API keys and providers to prevent bottlenecks
✅ Fallbacks - Seamlessly failover across providers and models to improve resilience
✅ Retries - Automatic exponential backoff retry logic to handle errors
✅ Middleware - Plug in custom middleware functions
✅ Battle-tested on over 100 billion tokens
The gateway is open source with a tiny footprint. It provides capabilities to build performant and reliable large-scale AI applications.
👉 Universal API - Implement your application once and easily switch between 100+ models and providers without code changes 💡
🔁 Load Balancing - Prevent bottlenecks and ensure high availability by distributing requests across multiple API keys and providers ⚖️
💥 Fallbacks - Build resilient applications that can failover across providers and models when errors occur 📉
♻️ Retries - Handle temporary errors automatically with exponential backoff retry logic built-in 🔄
- 👷🏽♀️ Builders: Visarg Desai, Rohit Agarwal, Vrushank Vyas
- 👩🏽💼 Builders on LinkedIn: https://www.linkedin.com/in/1rohitagarwal/, https://www.linkedin.com/in/visarg-desai/, https://www.linkedin.com/in/vrushank-vyas/
- 👩🏽🏭 Builders on X: https://twitter.com/jumbld
- 👩🏽💻 Contributors: 5
- 💫 GitHub Stars: 699
- 🍴 Forks: 44
- 👁️ Watch: 7
- 🪪 License: MIT
- 🔗 Links: Below 👇🏽
- GitHub Repository: https://github.com/Portkey-AI/gateway
- Official Website: https://portkey.ai/features/ai-gateway
- LinkedIn Page: https://www.linkedin.com/company/portkey-ai/
- X Page: https://twitter.com/PortkeyAI
- Profile in The AI Engineer: https://github.com/theaiengineer/awesome-opensource-ai-engineering/blob/main/libraries/gateway/README.md
🧙🏽 Follow The AI Engineer for more about Gateway and daily insights tailored to AI engineers. Subscribe to our newsletter. We are the AI community for hackers!
♻️ Repost this to help Gateway become more popular. Support AI Open-Source Libraries!