LMQL brings programming to LLM interaction. Write Python code interwoven with prompts. Add constraints, types & control flow. Supports beam search & other advanced decoding. Makes prompting modular, expressive & portable across LLMs.
LMQL is a programming language for Large Language Models. As large language models (LLMs) like GPT-3 gain more capabilities, effectively leveraging them requires moving beyond simplistic prompting. LMQL offers a programming model tailored for control and coordination of LLM interaction.
🐍 Native Python Syntax - Write code mixing traditional logic and LLM prompts for tight integration.
⚙️ Modular Prompting - Break prompts into reusable components with variables. Build libraries of prompt modules.
🤖 Advanced Decoding - Leverage techniques like beam search to deeply explore reasoning chains.
⛓️ Robust Constraints - Control LLM responses with token lengths, data types, regexes and more.
🔁 Optimizing Runtime - Speculative execution, tree caching and other optimizations accelerate prompting.
📡 Async Support - Scale to hundreds of parallel queries with async and batching.
🌈 Multi-Backend - Write once, target OpenAI, Cohere, Anthropic, etc. with same code.
So whether you need more control, better constraints, modular reuse, optimized performance or backend portability, LMQL brings the power of programming to LLM interaction. I invite you to check out the examples, documentation and playground to see how LMQL can boost your LLM applications.
- 🧱 Modularity - Break large prompts into reusable, configurable components. Build libraries of prompt modules.
- ⛓️ Constraints - Better control over LLM responses with token lengths, data types, regexes, etc. Critical for safety.
- 🔎 Decoding - Sophisticated decoding algorithms like beam search to deeply explore reasoning chains.
- ⚡️ Performance - Runtime optimizations like speculation, caching, batching accelerate prompting.
- 🚚 Portability - Write once, target multiple LLM backends (OpenAI, Cohere, Anthropic, etc.)
- 🛡️ Safety - Integrations with tools like ChainSafe for applying safety constraints.
- 💻 Extensibility - Call arbitrary Python functions during generation to augment capabilities.
- 🎹 Orchestration - Coordinate and sequence the prompting of multiple models.
So in summary, LMQL gives AI engineers far more control over LLM prompting through constraints, decoding techniques, performance improvements, and backend portability. This makes integration safer while expanding what's possible.
- 👷🏽♀️ Builders: Luca Beurer-Kellner, Marc Fischer, Charles Duffy, Matthew Dupree
- 👩🏽💼 Builders on LinkedIn: https://www.linkedin.com/in/luca-beurer-kellner-0b345616a/, https://www.linkedin.com/in/marc-fischer-b047571b1/, https://www.linkedin.com/in/charlesduffy/, https://www.linkedin.com/in/4onen/
- 👩🏽🏭 Builders on X: https://twitter.com/lbeurerkellner, https://twitter.com/marc_r_fischer, https://twitter.com/charlesdyfis, https://twitter.com/4onen
- 👩🏽💻 Contributors: 29
- 💫 GitHub Stars: 2.7k
- 🍴 Forks: 147
- 👁️ Watch: 21
- 🪪 License: Apache-2.0
- 🔗 Links: Below 👇🏽
- GitHub Repository: https://github.com/eth-sri/lmql
- Official Website: https://lmql.ai
- X Page: https://twitter.com/lmqllang
- Profile in The AI Engineer: https://github.com/theaiengineer/awesome-opensource-ai-engineering/blob/main/libraries/lmql.md
🧙🏽 Follow The AI Engineer for more about lmql and daily insights tailored to AI engineers. Subscribe to our newsletter. We are the AI community for hackers!
♻️ Repost this to help lmql become more popular. Support AI Open-Source Libraries!