Skip to content

Commit a79bae0

Browse files
RobinPicardrlouf
authored andcommitted
Create the migration guide to v1
1 parent dacd7ed commit a79bae0

File tree

1 file changed

+244
-0
lines changed

1 file changed

+244
-0
lines changed
+244
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,244 @@
1+
# Outlines 1.0 migration guide
2+
3+
Outlines 1.0 introduces some breaking changes that affect the way you use the library. You are likely concerned by all of the following sections, so please read this document carefully until the end.
4+
5+
This guide will help you migrate your code to the new version.
6+
7+
All previous functionality will be supported until Outlines version 1.5.0, but a warning message will be displayed to remind you to migrate your code and provide instructions to help you do so. Please migrate your code to the v1 as soon as possible.
8+
9+
## Removed or modified features
10+
- [Generate functions](#generate-functions)
11+
- [Models](#models)
12+
- [Samplers](#samplers)
13+
- [Functions](#functions)
14+
- [Text generation return types](#text-generation-return-types)
15+
- [Inference arguments](#inference-arguments)
16+
17+
### Generate functions
18+
19+
The whole `generate` module has been removed. That includes the functions `generate.cfg`, `generate.choice`, `generate.format`,`generate.fsm`, `generate.json`, `generate.regex` and `generate.text`.
20+
21+
You should replace these functions by the `Generator` object along with the right output type as an argument (on top of the model). The output type can either be a python type or be an object from the `outlines.types` module. You can find more information about the output types in the [output types](TODO: link to the documentatoin on output types) guide.
22+
23+
Associated v1 output types for each deprecated function:
24+
- `generate.cfg` -> `outlines.types.CFG`
25+
- `generate.choice` -> `typing.Literal` or `typing.Union`
26+
- `generate.format` -> native python types (`str`, `int` etc.)
27+
- `generate.fsm` -> `outlines.types.FSM`
28+
- `generate.json` -> `pydantic.BaseModel`, `typing.TypedDict`, `dataclasses.dataclass`, `genson.schema.SchemaBuilder` or `outlines.types.JsonSchema`
29+
- `generate.regex` -> `outlines.types.Regex`
30+
- `generate.text` -> no output type (`None`)
31+
32+
For instance, instead of:
33+
34+
```python
35+
from outlines import generate
36+
37+
model = ...
38+
generator = generate.choice(model, ["foo", "bar"])
39+
```
40+
41+
You should now use:
42+
43+
```python
44+
from typing import Literal
45+
from outlines import Generator
46+
47+
model = ...
48+
generator = Generator(model, Literal["foo", "bar"])
49+
```
50+
51+
### Models
52+
53+
The model classes found in the `outlines.models` module are maintained but there are a few important changes to be aware of.
54+
55+
The functions used to created a model have been replaced by equivalent functions named with a `from_` prefix. The function `outlines.models.transformers` has been replaced by `outlines.from_transformers` for instance. On top of this change of name, the arguments have been modified. You should refer to the [models documentation](TODO: link to the documentation on models) for more details, but the overall idea is that you now need to provide a model/client instance from the inference library the Outlines model is wrapping.
56+
57+
For instance, instead of:
58+
59+
```python
60+
from outlines import models
61+
62+
model = models.llamacpp(
63+
repo_id="M4-ai/TinyMistral-248M-v2-Instruct-GGUF",
64+
filename="TinyMistral-248M-v2-Instruct.Q4_K_M.gguf",
65+
)
66+
```
67+
68+
You should now do:
69+
70+
```python
71+
from llama_cpp import Llama
72+
from outlines import from_llamacpp
73+
74+
llamacpp_model = Llama.from_pretrained(
75+
repo_id="M4-ai/TinyMistral-248M-v2-Instruct-GGUF",
76+
filename="TinyMistral-248M-v2-Instruct.Q4_K_M.gguf",
77+
)
78+
model = from_llamacpp(llamacpp_model)
79+
```
80+
81+
The `load_lora` methods that are present on the `VLLM` and `LlamaCpp` models have been removed. You should now handle lora loading through the `Llama` instance in the case of the `LlamaCpp` model or provide it as a keyword argument when calling the model in the case of the `VLLM` model.
82+
83+
For instance, instead of:
84+
85+
```python
86+
from outlines import from_vllm
87+
from vllm import LLM
88+
89+
model = from_vllm(
90+
LLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
91+
)
92+
model.load_lora("path/to/lora/file")
93+
94+
response = model("foo")
95+
```
96+
97+
You should now do:
98+
99+
```python
100+
from outlines import from_vllm
101+
from vllm import LLM
102+
from vllm.lora.request import LoRARequest
103+
104+
model = from_vllm(
105+
LLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
106+
)
107+
lora_request = LoRARequest("path/to/lora/file", 1, "path/to/lora/file")
108+
109+
response = model("foo", lora_request=lora_request)
110+
```
111+
112+
The `ExLlamav2` model has been removed without replacement. This inference library is not fully compatible with Outlines, so we decided to remove it. You can still use it until final deprecation, but we recommend you to migrate to a different inference library right now.
113+
114+
### Samplers
115+
116+
The `outlines.samplers` module has been removed without replacement. You should now use the arguments of the inference library model to control the sampling. Depending on the model you use, this could be done at initialization or when calling the model to generate text (so when calling the outlines model or a generator).
117+
118+
For instance, instead of:
119+
120+
```python
121+
from outlines import generate
122+
123+
model = <transformers_model>
124+
125+
generator = generate.text(model, samplers.beam_search(2))
126+
response = generator("foo")
127+
```
128+
129+
You should now do:
130+
131+
```python
132+
from outlines import Generator
133+
134+
model = <transformers_model>
135+
136+
generator = Generator(model)
137+
response = generator("foo", num_beams=2)
138+
```
139+
140+
### Functions
141+
142+
The `outlines.function` module has been removed. It is replaced by the `outlines.applications` module. An `Application` serves a similar purpose as a `Function`: it encapsulates a prompt template and an output type. A difference is that can `Application` is not instantiated with a model name. Instead, you should provide a model instance along with the prompt when calling it.
143+
144+
For instance, instead of:
145+
146+
```python
147+
from outlines import Function
148+
149+
prompt_template = ...
150+
output_type = ...
151+
152+
fn = Function(
153+
prompt_template,
154+
output_type,
155+
"hf-internal-testing/tiny-random-GPTJForCausalLM",
156+
)
157+
158+
result = fn("foo")
159+
```
160+
161+
You should now do:
162+
163+
```python
164+
from outlines import Application
165+
166+
prompt_template = ...
167+
output_type = ...
168+
169+
application = Application(
170+
prompt_template,
171+
output_type,
172+
)
173+
174+
model = ...
175+
176+
result = application(model, "foo")
177+
```
178+
179+
### Text generation return types
180+
181+
In the previous version of Outlines, the return type of the generators depended on the output type provided. For instance, if you passed a Pydantic model to the `generate.json` function, the return type was a Pydantic model instance. In the v1, the return type of a generator is always a `str`, the raw text generated by the model. You are responsible for parsing the text into the desired format.
182+
183+
For instance, instead of:
184+
185+
```python
186+
from pydantic import BaseModel
187+
from outlines import generate
188+
189+
class Foo(BaseModel):
190+
bar: str
191+
192+
model = ...
193+
194+
generator = generate.json(model, Foo)
195+
196+
result = generator("foo")
197+
print(result.bar)
198+
```
199+
200+
You should now do:
201+
202+
```python
203+
from pydantic import BaseModel
204+
from outlines import Generator
205+
206+
class Foo(BaseModel):
207+
bar: str
208+
209+
model = ...
210+
211+
generator = Generator(model, Foo)
212+
213+
result = generator("foo")
214+
result = Foo.model_validate_json(result) # parse the text into the Pydantic model instance
215+
print(result.bar)
216+
```
217+
218+
### Inference arguments
219+
220+
In the previous version of Outlines, some of the inference arguments were standardized across the models and were provided as positional arguments to the generator or through the sampling params dictionary. Additionally, various default values were added by outlines to the inference library models. This is no longer the case. You should refer to the documentation of the inference library you use to find the right arguments for your use case and pass them as keyword arguments to the outlines generator when calling it.
221+
222+
For instance, instead of:
223+
224+
```python
225+
from outlines import generate
226+
227+
model = <transformers_model>
228+
229+
generator = generate.text(model)
230+
231+
result = generator("foo", 256, ".", 10) # 256 tokens, stop at "." and seed 10
232+
```
233+
234+
You should now do:
235+
236+
```python
237+
from outlines import Generator
238+
239+
model = <transformers_model>
240+
241+
generator = Generator(model)
242+
243+
result = generator("foo", max_new_tokens=256, stop_strings=".", seed=10)
244+
```

0 commit comments

Comments
 (0)