|
| 1 | +# Outlines 1.0 migration guide |
| 2 | + |
| 3 | +Outlines 1.0 introduces some breaking changes that affect the way you use the library. You are likely concerned by all of the following sections, so please read this document carefully until the end. |
| 4 | + |
| 5 | +This guide will help you migrate your code to the new version. |
| 6 | + |
| 7 | +All previous functionality will be supported until Outlines version 1.5.0, but a warning message will be displayed to remind you to migrate your code and provide instructions to help you do so. Please migrate your code to the v1 as soon as possible. |
| 8 | + |
| 9 | +## Removed or modified features |
| 10 | +- [Generate functions](#generate-functions) |
| 11 | +- [Models](#models) |
| 12 | +- [Samplers](#samplers) |
| 13 | +- [Functions](#functions) |
| 14 | +- [Text generation return types](#text-generation-return-types) |
| 15 | +- [Inference arguments](#inference-arguments) |
| 16 | + |
| 17 | +### Generate functions |
| 18 | + |
| 19 | +The whole `generate` module has been removed. That includes the functions `generate.cfg`, `generate.choice`, `generate.format`,`generate.fsm`, `generate.json`, `generate.regex` and `generate.text`. |
| 20 | + |
| 21 | +You should replace these functions by the `Generator` object along with the right output type as an argument (on top of the model). The output type can either be a python type or be an object from the `outlines.types` module. You can find more information about the output types in the [output types](TODO: link to the documentatoin on output types) guide. |
| 22 | + |
| 23 | +Associated v1 output types for each deprecated function: |
| 24 | +- `generate.cfg` -> `outlines.types.CFG` |
| 25 | +- `generate.choice` -> `typing.Literal` or `typing.Union` |
| 26 | +- `generate.format` -> native python types (`str`, `int` etc.) |
| 27 | +- `generate.fsm` -> `outlines.types.FSM` |
| 28 | +- `generate.json` -> `pydantic.BaseModel`, `typing.TypedDict`, `dataclasses.dataclass`, `genson.schema.SchemaBuilder` or `outlines.types.JsonSchema` |
| 29 | +- `generate.regex` -> `outlines.types.Regex` |
| 30 | +- `generate.text` -> no output type (`None`) |
| 31 | + |
| 32 | +For instance, instead of: |
| 33 | + |
| 34 | +```python |
| 35 | +from outlines import generate |
| 36 | + |
| 37 | +model = ... |
| 38 | +generator = generate.choice(model, ["foo", "bar"]) |
| 39 | +``` |
| 40 | + |
| 41 | +You should now use: |
| 42 | + |
| 43 | +```python |
| 44 | +from typing import Literal |
| 45 | +from outlines import Generator |
| 46 | + |
| 47 | +model = ... |
| 48 | +generator = Generator(model, Literal["foo", "bar"]) |
| 49 | +``` |
| 50 | + |
| 51 | +### Models |
| 52 | + |
| 53 | +The model classes found in the `outlines.models` module are maintained but there are a few important changes to be aware of. |
| 54 | + |
| 55 | +The functions used to created a model have been replaced by equivalent functions named with a `from_` prefix. The function `outlines.models.transformers` has been replaced by `outlines.from_transformers` for instance. On top of this change of name, the arguments have been modified. You should refer to the [models documentation](TODO: link to the documentation on models) for more details, but the overall idea is that you now need to provide a model/client instance from the inference library the Outlines model is wrapping. |
| 56 | + |
| 57 | +For instance, instead of: |
| 58 | + |
| 59 | +```python |
| 60 | +from outlines import models |
| 61 | + |
| 62 | +model = models.llamacpp( |
| 63 | + repo_id="M4-ai/TinyMistral-248M-v2-Instruct-GGUF", |
| 64 | + filename="TinyMistral-248M-v2-Instruct.Q4_K_M.gguf", |
| 65 | +) |
| 66 | +``` |
| 67 | + |
| 68 | +You should now do: |
| 69 | + |
| 70 | +```python |
| 71 | +from llama_cpp import Llama |
| 72 | +from outlines import from_llamacpp |
| 73 | + |
| 74 | +llamacpp_model = Llama.from_pretrained( |
| 75 | + repo_id="M4-ai/TinyMistral-248M-v2-Instruct-GGUF", |
| 76 | + filename="TinyMistral-248M-v2-Instruct.Q4_K_M.gguf", |
| 77 | +) |
| 78 | +model = from_llamacpp(llamacpp_model) |
| 79 | +``` |
| 80 | + |
| 81 | +The `load_lora` methods that are present on the `VLLM` and `LlamaCpp` models have been removed. You should now handle lora loading through the `Llama` instance in the case of the `LlamaCpp` model or provide it as a keyword argument when calling the model in the case of the `VLLM` model. |
| 82 | + |
| 83 | +For instance, instead of: |
| 84 | + |
| 85 | +```python |
| 86 | +from outlines import from_vllm |
| 87 | +from vllm import LLM |
| 88 | + |
| 89 | +model = from_vllm( |
| 90 | + LLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct") |
| 91 | +) |
| 92 | +model.load_lora("path/to/lora/file") |
| 93 | + |
| 94 | +response = model("foo") |
| 95 | +``` |
| 96 | + |
| 97 | +You should now do: |
| 98 | + |
| 99 | +```python |
| 100 | +from outlines import from_vllm |
| 101 | +from vllm import LLM |
| 102 | +from vllm.lora.request import LoRARequest |
| 103 | + |
| 104 | +model = from_vllm( |
| 105 | + LLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct") |
| 106 | +) |
| 107 | +lora_request = LoRARequest("path/to/lora/file", 1, "path/to/lora/file") |
| 108 | + |
| 109 | +response = model("foo", lora_request=lora_request) |
| 110 | +``` |
| 111 | + |
| 112 | +The `ExLlamav2` model has been removed without replacement. This inference library is not fully compatible with Outlines, so we decided to remove it. You can still use it until final deprecation, but we recommend you to migrate to a different inference library right now. |
| 113 | + |
| 114 | +### Samplers |
| 115 | + |
| 116 | +The `outlines.samplers` module has been removed without replacement. You should now use the arguments of the inference library model to control the sampling. Depending on the model you use, this could be done at initialization or when calling the model to generate text (so when calling the outlines model or a generator). |
| 117 | + |
| 118 | +For instance, instead of: |
| 119 | + |
| 120 | +```python |
| 121 | +from outlines import generate |
| 122 | + |
| 123 | +model = <transformers_model> |
| 124 | + |
| 125 | +generator = generate.text(model, samplers.beam_search(2)) |
| 126 | +response = generator("foo") |
| 127 | +``` |
| 128 | + |
| 129 | +You should now do: |
| 130 | + |
| 131 | +```python |
| 132 | +from outlines import Generator |
| 133 | + |
| 134 | +model = <transformers_model> |
| 135 | + |
| 136 | +generator = Generator(model) |
| 137 | +response = generator("foo", num_beams=2) |
| 138 | +``` |
| 139 | + |
| 140 | +### Functions |
| 141 | + |
| 142 | +The `outlines.function` module has been removed. It is replaced by the `outlines.applications` module. An `Application` serves a similar purpose as a `Function`: it encapsulates a prompt template and an output type. A difference is that can `Application` is not instantiated with a model name. Instead, you should provide a model instance along with the prompt when calling it. |
| 143 | + |
| 144 | +For instance, instead of: |
| 145 | + |
| 146 | +```python |
| 147 | +from outlines import Function |
| 148 | + |
| 149 | +prompt_template = ... |
| 150 | +output_type = ... |
| 151 | + |
| 152 | +fn = Function( |
| 153 | + prompt_template, |
| 154 | + output_type, |
| 155 | + "hf-internal-testing/tiny-random-GPTJForCausalLM", |
| 156 | +) |
| 157 | + |
| 158 | +result = fn("foo") |
| 159 | +``` |
| 160 | + |
| 161 | +You should now do: |
| 162 | + |
| 163 | +```python |
| 164 | +from outlines import Application |
| 165 | + |
| 166 | +prompt_template = ... |
| 167 | +output_type = ... |
| 168 | + |
| 169 | +application = Application( |
| 170 | + prompt_template, |
| 171 | + output_type, |
| 172 | +) |
| 173 | + |
| 174 | +model = ... |
| 175 | + |
| 176 | +result = application(model, "foo") |
| 177 | +``` |
| 178 | + |
| 179 | +### Text generation return types |
| 180 | + |
| 181 | +In the previous version of Outlines, the return type of the generators depended on the output type provided. For instance, if you passed a Pydantic model to the `generate.json` function, the return type was a Pydantic model instance. In the v1, the return type of a generator is always a `str`, the raw text generated by the model. You are responsible for parsing the text into the desired format. |
| 182 | + |
| 183 | +For instance, instead of: |
| 184 | + |
| 185 | +```python |
| 186 | +from pydantic import BaseModel |
| 187 | +from outlines import generate |
| 188 | + |
| 189 | +class Foo(BaseModel): |
| 190 | + bar: str |
| 191 | + |
| 192 | +model = ... |
| 193 | + |
| 194 | +generator = generate.json(model, Foo) |
| 195 | + |
| 196 | +result = generator("foo") |
| 197 | +print(result.bar) |
| 198 | +``` |
| 199 | + |
| 200 | +You should now do: |
| 201 | + |
| 202 | +```python |
| 203 | +from pydantic import BaseModel |
| 204 | +from outlines import Generator |
| 205 | + |
| 206 | +class Foo(BaseModel): |
| 207 | + bar: str |
| 208 | + |
| 209 | +model = ... |
| 210 | + |
| 211 | +generator = Generator(model, Foo) |
| 212 | + |
| 213 | +result = generator("foo") |
| 214 | +result = Foo.model_validate_json(result) # parse the text into the Pydantic model instance |
| 215 | +print(result.bar) |
| 216 | +``` |
| 217 | + |
| 218 | +### Inference arguments |
| 219 | + |
| 220 | +In the previous version of Outlines, some of the inference arguments were standardized across the models and were provided as positional arguments to the generator or through the sampling params dictionary. Additionally, various default values were added by outlines to the inference library models. This is no longer the case. You should refer to the documentation of the inference library you use to find the right arguments for your use case and pass them as keyword arguments to the outlines generator when calling it. |
| 221 | + |
| 222 | +For instance, instead of: |
| 223 | + |
| 224 | +```python |
| 225 | +from outlines import generate |
| 226 | + |
| 227 | +model = <transformers_model> |
| 228 | + |
| 229 | +generator = generate.text(model) |
| 230 | + |
| 231 | +result = generator("foo", 256, ".", 10) # 256 tokens, stop at "." and seed 10 |
| 232 | +``` |
| 233 | + |
| 234 | +You should now do: |
| 235 | + |
| 236 | +```python |
| 237 | +from outlines import Generator |
| 238 | + |
| 239 | +model = <transformers_model> |
| 240 | + |
| 241 | +generator = Generator(model) |
| 242 | + |
| 243 | +result = generator("foo", max_new_tokens=256, stop_strings=".", seed=10) |
| 244 | +``` |
0 commit comments