An Introduction to LMQL: The Bridge Between SQL and Large Language Models

SQL, the Structured Query Language, is a cornerstone of database management, enabling efficient data storage, retrieval, and manipulation. Its widespread adoption stems from its simplicity and effectiveness in handling vast datasets. However, the evolving data landscape introduces new challenges.

The rise of artificial intelligence and large language models (LLMs) presents powerful tools, but interacting with them can be cumbersome. This is where LMQL steps in.

Developed by the SRI Lab at ETH Zürich, LMQL acts as a bridge between developers and LLMs. It brings the structured querying power of SQL to the world of language models, streamlining interactions and enhancing efficiency.

This tutorial covers:

• What is LMQL?
• Why use LMQL?
• Setting up LMQL
• Practical LMQL applications
• LMQL limitations
• Best practices

What is LMQL?

LMQL, or Language Models Query Language, is a novel programming language designed for LLMs. It combines declarative SQL-like features with an imperative scripting syntax, offering a more structured approach to information extraction and response generation from LLMs.

Importantly, LMQL extends Python, adding new functionalities and expanding its capabilities. This allows developers to craft natural language prompts incorporating text and code, increasing query flexibility and expressiveness. As its creators state, LMQL seamlessly integrates LLM interaction into program code, moving beyond traditional templating. It was introduced in a research paper, "Prompting is Programming: A Query Language for Large Language Models," as a solution for "Language Model Prompting" (LMP).

LLMs excel at tasks like question answering and code generation, generating logical sequences based on input probabilities. LMP leverages this by using language instructions or examples to trigger tasks. Advanced techniques even allow interactions between users, the model, and external tools.

The challenge lies in achieving optimal performance or tailoring LLMs for specific tasks, often requiring complex, task-specific programs that may still depend on ad-hoc interactions. LMQL addresses this by providing an intuitive blend of text prompting and scripting, enabling users to define constraints on LLM output.

Why Use LMQL?

While modern LLMs can be prompted conceptually, maximizing their potential and adapting to new models requires deep understanding of their inner workings and vendor-specific tools. Tasks like limiting output to specific words or phrases can be complex due to tokenization. Furthermore, using LLMs, whether locally or via APIs, is expensive due to their size.

LMQL mitigates these issues. It reduces LLM calls by leveraging predefined behaviors and search constraints. It also simplifies prompting techniques that often involve iterative communication between user and model or specialized interfaces. LMQL's constraint capabilities are crucial for production environments, ensuring predictable and processable output. For instance, in sentiment analysis, LMQL ensures consistent output like "positive," "negative," or "neutral," rather than more verbose, less easily parsed responses. Human-readable constraints replace the need to work with model tokens directly.

Setting Up LMQL

LMQL can be installed locally or accessed via its online Playground IDE. Local installation is necessary for self-hosted models using Transformers or llama.cpp.

Installation and Environment Setup

Local installation is simple:

pip install lmql

For GPU support with PyTorch >= 1.11:

pip install lmql[hf]

Using a virtual environment is recommended.

Three ways to run LMQL programs exist:

1. Playground: lmql playground launches a browser-based IDE (requires Node.js). Access via https://www.php.cn/link/4a914e5c38172ae9b61780ffbd0b2f90 if not automatically launched.
2. Command-line: lmql run executes local .lmql files.
3. Python Integration: Import lmql and use lmql.run or the @lmql.query decorator.

When using local Transformer models in the Playground or command line, launch the LMQL Inference API using lmql serve-model.

Understanding LMQL Syntax

An LMQL program has five key parts:

• Query: The primary communication method between user and LLM. Uses [varname] for generated text and {varname} for variable retrieval.
• Decoder: Specifies the decoding algorithm (e.g., beam search). Can be defined within the query or externally (in Python).
• Model: LMQL supports various models (OpenAI, llama.cpp, HuggingFace Transformers). Models are loaded using lmql.model(...), and passed to the query either externally or using a from clause.
• Constraints: Control LLM output using various constraints (stopping phrases, data types, character/token length, regex, custom constraints).
• Distribution: Defines the output format and structure.

LMQL Limitations and Community Support

LMQL's relative newness leads to a small community and less comprehensive documentation. Limitations with the OpenAI API also restrict full utilization with certain models like ChatGPT. However, ongoing development promises improvements.

Conclusion

LMQL offers a powerful, SQL-inspired approach to interacting with LLMs. Its Python integration and constraint capabilities make it a valuable tool for various applications. For further learning, explore resources on LlamaIndex, ChatGPT alternatives, LLM training with PyTorch, LangChain, and the Cohere API.

The above is the detailed content of An Introduction to LMQL: The Bridge Between SQL and Large Language Models. For more information, please follow other related articles on the PHP Chinese website!