Skip to content

Commit bf9f860

Browse files
authored
docs: add a 10.0-to-10.1 upgrade page and document the 10.1 LSN and 96-document changes (#782)
* docs(migration): add a 10.0-to-10.1 upgrade page and mark 10.1 changes in the V10 guide The page covers each behavior change since v10.0.0 by task, and a unit test runs its snippets in order against respx, checking every quoted error. The V10 guide again describes 10.0.0, with version notes where 10.1.0 differs; its path-parameter section moves to the new page. * docs(models): bound the read-your-writes recipe and note the 10.1 LSN change The ResponseInfo example now gives its search loop a deadline, and a versionchanged note says why the 10.0.0 fetch-polling recipe never exits. A test runs the example both to completion and to its deadline. * docs(documents): state the 96-document cap for indexes that embed a field upsert, batch_upsert and update now name the server's 96-documents-per-request limit for indexes with embed or sparse_embed fields, and batch_upsert no longer calls an oversize batch_size safe on those indexes. * test(docs): drop the db-control 9.x parametrization that collected nothing The section's 9.x-shaped calls became working deprecated sugar in #500/#501, so the selector found no block and the test skipped as [NOTSET]. The guided errors it meant to cover are pinned in client/test_indexes_create.py and its async twin. * no-mistakes(review): Fix 10.0.0 db_data shim note; restore non-URL type-guard list * no-mistakes(ci): The Sphinx docs build failed at its "Check published docs URLs still resolve" step. That step requires every built page to be listed in `docs/published-urls.txt`, and the new 10.0-to-10.1 upgrade page (`migration/v10-1-upgrade.html`) wasn't in the list. I added that one line, in sorted position with the other `migration/` entries. No other new page was missing. Locally, `make -C docs html` built successfully, and `uv run --no-sync python scripts/check_docs_urls.py` passed: "ok: all 236 published URLs still resolve in this build
1 parent 04e7125 commit bf9f860

10 files changed

Lines changed: 610 additions & 108 deletions

File tree

‎docs/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ Pinecone Python SDK
2525
:maxdepth: 1
2626
:caption: Migration & FAQ
2727

28+
migration/v10-1-upgrade
2829
migration/v10-migration
2930
migration/v10-grpc-partial-failures
3031
migration/v9-migration

‎docs/migration/v10-1-upgrade.md‎

Lines changed: 315 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,315 @@
1+
# Upgrading from 10.0 to 10.1
2+
3+
10.1 removes nothing and deprecates nothing. Most of what changed is when a
4+
mistake gets reported: calls that 10.0.0 sent to the server as given, or
5+
quietly reshaped, now raise before a request goes out. One change can stop
6+
working code without raising: a loop that polls `documents.fetch` to wait for
7+
a write never exits. If you wait for writes to become visible, start with
8+
[Waiting for a write to become visible](#v10-1-read-your-writes).
9+
10+
Coming from `9.x`? Read [Migrating to V10](v10-migration.md) first, then this
11+
page.
12+
13+
The snippets run in order against one client. `movies` is a document index
14+
whose schema declares `title` as a `string` field with full-text search:
15+
16+
```python
17+
from pinecone import Pinecone
18+
19+
pc = Pinecone(api_key="your-api-key")
20+
index = pc.index("movies")
21+
```
22+
23+
## Contents
24+
25+
- [Waiting for a write to become visible](#v10-1-read-your-writes)
26+
- [Documents API arguments are checked before sending](#v10-1-documents-arguments)
27+
- [Timeouts must be positive](#v10-1-timeouts)
28+
- [Names and IDs must be strings](#v10-1-path-parameters)
29+
- [Creating a client](#v10-1-clients)
30+
- [Bulk document upserts](#v10-1-bulk-upsert)
31+
- [Summary](#v10-1-summary)
32+
33+
(v10-1-read-your-writes)=
34+
## Waiting for a write to become visible
35+
36+
10.1.0 reads the log-position headers the Documents API sends. A documents
37+
upsert, update or delete response now carries
38+
`response_info.lsn_committed`, and a documents search response carries
39+
`response_info.lsn_reconciled`. In 10.0.0 both were always `None` on the
40+
Documents API.
41+
42+
That breaks the read-your-writes loop the 10.0.0 `ResponseInfo` reference
43+
showed:
44+
45+
```python
46+
# 10.0.0 recipe: never exits on 10.1.0
47+
import time
48+
49+
written = index.documents.upsert(
50+
namespace="movies-en", documents=[{"_id": "movie-42", "title": "Arrival"}]
51+
)
52+
target = written.response_info.lsn_committed
53+
54+
while True:
55+
fetched = index.documents.fetch(namespace="movies-en", ids=["movie-42"])
56+
if target is None or fetched.response_info.is_reconciled(target):
57+
break
58+
time.sleep(0.5)
59+
```
60+
61+
On 10.0.0, `target` was `None`, so the loop exited at once without waiting
62+
for anything. On 10.1.0, `target` is a number, but a fetch response never
63+
reports a reconciled position, so `is_reconciled(target)` stays `False` and
64+
the loop runs forever. Nothing raises or warns.
65+
66+
Poll `documents.search` instead, and give the loop a deadline:
67+
68+
```python
69+
import time
70+
71+
from pinecone import TextQuery
72+
73+
written = index.documents.upsert(
74+
namespace="movies-en", documents=[{"_id": "movie-42", "title": "Arrival"}]
75+
)
76+
target = written.response_info.lsn_committed
77+
78+
deadline = time.monotonic() + 30
79+
while target is not None: # None: caught-up cannot be confirmed
80+
found = index.documents.search(
81+
namespace="movies-en",
82+
top_k=1,
83+
score_by=[TextQuery(query="arrival", fields=["title"])],
84+
)
85+
if found.response_info.is_reconciled(target):
86+
break
87+
if time.monotonic() > deadline:
88+
raise TimeoutError(f"search had not reached LSN {target} after 30s")
89+
time.sleep(0.5)
90+
```
91+
92+
Fetch and search can see a new write at different times, so a fetch that
93+
returns the document does not mean a search will find it. After
94+
`documents.batch_upsert`, use `result.response_info.lsn_committed` as the
95+
target; it is the highest position any of the batch's requests reached.
96+
97+
(v10-1-documents-arguments)=
98+
## Documents API arguments are checked before sending
99+
100+
A dict `score_by` clause with a key its type does not define now raises
101+
`PineconeValueError`. 10.0.0 dropped the key and sent the rest, so an option
102+
such as `boost` silently did nothing:
103+
104+
```python
105+
index.documents.search(
106+
namespace="movies-en",
107+
top_k=5,
108+
score_by=[{"type": "text", "fields": ["title"], "query": "arrival", "boost": 2}],
109+
)
110+
# PineconeValueError: Object contains unknown field `boost`
111+
```
112+
113+
A bare `str` where a list of names belongs now raises `PineconeTypeError`.
114+
10.0.0 split `ids`, `include_fields` and `remove_fields` into one entry per
115+
character, so `ids="movie-42"` deleted the IDs `"m"`, `"o"`, `"v"` and so on.
116+
A clause's `fields` given as one string, which 10.0.0 sent as is, is refused
117+
the same way:
118+
119+
```python
120+
index.documents.delete(namespace="movies-en", ids="movie-42")
121+
# PineconeTypeError: ids must be a sequence of strings, not a single str; pass ['movie-42'] to name one entry.
122+
123+
TextQuery(query="arrival", fields="title")
124+
# PineconeTypeError: 'fields' must be a list of field names, not a single str; pass ['title'].
125+
```
126+
127+
Three more wrong-type arguments now raise `PineconeTypeError` with a message
128+
that names the fix:
129+
130+
| Call | 10.0.0 | 10.1.0 |
131+
| --- | --- | --- |
132+
| `score_by=TextQuery(...)`, not in a list | `TypeError: 'TextQuery' object is not iterable` | `PineconeTypeError` saying to wrap the clause in a list |
133+
| `score_by=[clause, 5]`, an item that is not a clause or a dict | sent to the server, or a misleading `PineconeValueError` about combining clauses | `PineconeTypeError` naming the item and the clause classes |
134+
| `top_k` or `limit` that is not an `int`, such as `"5"` or `5.0` | a bare `TypeError`, or sent to the server | `PineconeTypeError` |
135+
136+
`PineconeTypeError` is a `TypeError`, so an `except TypeError` around these
137+
calls still catches them.
138+
139+
(v10-1-timeouts)=
140+
## Timeouts must be positive
141+
142+
A `timeout` or `total_timeout` of zero, a negative number, or NaN now raises
143+
`PineconeValueError` before anything is sent. This covers the client-level
144+
`timeout` on `Pinecone` and `AsyncPinecone`, the per-call `timeout` on every
145+
Documents API method, and `total_timeout` on `documents.batch_upsert`,
146+
`upsert(batch_size=...)` and `upsert_from_dataframe`. 10.0.0 passed these
147+
values through: `timeout=0` reached the HTTP layer, and `total_timeout=0`
148+
returned a result with every item failed and `timed_out` set.
149+
150+
```python
151+
index.documents.upsert(
152+
namespace="movies-en", documents=[{"_id": "movie-42", "title": "Arrival"}], timeout=0
153+
)
154+
# PineconeValueError: timeout must be a positive number of seconds, got 0
155+
```
156+
157+
`None` still means the default (no deadline, for `total_timeout`).
158+
159+
The wait `timeout` on `pc.indexes.create()`, `create_for_model()` and
160+
`delete()` changed too. A negative value other than `-1` now raises
161+
`PineconeValueError` before the request. 10.0.0 sent the request, so the
162+
index was created or deleted anyway, and then raised `PineconeTimeoutError`
163+
at the first check that did not find the operation finished:
164+
165+
```python
166+
pc.indexes.create(
167+
name="movies-v2",
168+
schema={"fields": {"title": {"type": "string", "full_text_search": {}}}},
169+
timeout=-5,
170+
)
171+
# PineconeValueError: timeout must be None (wait indefinitely), -1 (do not wait), or a non-negative number of seconds, got -5. Pass -1 to skip waiting.
172+
```
173+
174+
Pass `-1` to return without waiting, `None` to wait as long as it takes, or a
175+
positive number of seconds.
176+
177+
(v10-1-path-parameters)=
178+
## Names and IDs must be strings
179+
180+
A name or ID the SDK puts into a URL (an index name, a backup ID, an assistant
181+
name or file ID, an admin resource ID) must be a `str`. Anything else now
182+
raises `PineconeTypeError` naming the parameter and the type it got, before
183+
any request:
184+
185+
```python
186+
import uuid
187+
188+
pc.backups.list(index_name=1234)
189+
# PineconeTypeError: index_name must be a string, got int
190+
191+
pc.assistants.describe(name=uuid.uuid4())
192+
# PineconeTypeError: name must be a string, got UUID
193+
194+
pc.indexes.describe(None)
195+
# PineconeTypeError: name must be a string, got NoneType
196+
```
197+
198+
What each case did in 10.0.0:
199+
200+
- `backups.list` and the assistant methods converted the value with `str()`
201+
and sent it, asking for `GET /indexes/1234/backups`, a resource that cannot
202+
exist.
203+
- Elsewhere a non-string escaped as a bare `AttributeError`
204+
(`'int' object has no attribute 'strip'`) or a `TypeError` from the URL
205+
encoder.
206+
- `None` raised `PineconeValueError: name must be a non-empty string`. An
207+
`except PineconeValueError` (or `except ValueError`) written for that no
208+
longer catches it; catch `PineconeTypeError` as well. An empty string still
209+
raises `PineconeValueError`.
210+
211+
The same check covers twelve arguments that never reach the URL, which move
212+
from `AttributeError` in 10.0.0 to `PineconeTypeError`:
213+
`create_index_from_backup(name=)`, `indexes.create_for_model(cloud=, region=)`,
214+
`collections.create(source=)`, `backup_schedules.create(name=)`,
215+
`inference.rerank(query=)`, and, on the `Admin` client,
216+
`api_keys.create(name=)`, `invites.create(email=)`, `projects.create(name=)`,
217+
`projects.describe_by_name(name=)`, `role_bindings.create(principal_id=)` and
218+
`service_accounts.create(name=)`.
219+
220+
Convert at the call site, with `str(uuid_value)` or `str(numeric_id)`. Two
221+
cases keep their documented behavior: an optional path parameter still
222+
accepts `None`, so `pc.backups.list(index_name=None)` lists backups across
223+
every index, and `describe_import()` and `cancel_import()` still accept an
224+
`int` ID and convert it.
225+
226+
(v10-1-clients)=
227+
## Creating a client
228+
229+
An API key that cannot be sent as an HTTP header, such as one with a trailing
230+
newline read from a file, now raises `PineconeValueError` when you create the
231+
client. The message never repeats the key. 10.0.0 accepted the key, and the
232+
first request failed inside the HTTP library with a message that could
233+
include it:
234+
235+
```python
236+
api_key = "your-api-key\n" # as read from a file
237+
Pinecone(api_key=api_key)
238+
# PineconeValueError: The API key (api_key or PINECONE_API_KEY) is not a valid HTTP header value: it is blank or whitespace-only, has leading or trailing whitespace (such as a newline read from a file), or contains a control or non-ASCII character. Strip the key, for example api_key=key.strip().
239+
240+
pc = Pinecone(api_key=api_key.strip())
241+
```
242+
243+
`GrpcIndex` still accepts leading and trailing spaces and tabs, which gRPC can
244+
send, and refuses the rest.
245+
246+
An index host given with `http://` now emits a `RuntimeWarning`, because the
247+
API key and every request travel unencrypted. Loopback and private IP
248+
addresses, `localhost`, and `*.localhost` are exempt, so a local emulator
249+
stays quiet. Pass the host with `https://`, or with no scheme:
250+
251+
```python
252+
pc.index(host="http://movies-abc123.svc.aped-4627-b74a.pinecone.io")
253+
# RuntimeWarning: The index host http://movies-abc123.svc.aped-4627-b74a.pinecone.io uses http, so the API key and every request and response body travel unencrypted.
254+
```
255+
256+
`additional_headers` now replaces an SDK header whatever its capitalization.
257+
In 10.0.0 only the exact spelling, such as `"X-Pinecone-Api-Version"`,
258+
replaced it; any other spelling was sent alongside the SDK's own header, so
259+
the server received two values. This client now sends one
260+
`X-Pinecone-Api-Version`, `2025-10`:
261+
262+
```python
263+
pinned = Pinecone(
264+
api_key="your-api-key", additional_headers={"x-pinecone-api-version": "2025-10"}
265+
)
266+
```
267+
268+
`Pinecone()` and `AsyncPinecone()` called with a keyword they do not take
269+
raise `PineconeTypeError` instead of a plain `TypeError`. It is a subclass of
270+
`TypeError`, so existing handlers still catch it.
271+
272+
(v10-1-bulk-upsert)=
273+
## Bulk document upserts
274+
275+
When the server rejects a `documents.batch_upsert` batch as too large
276+
(HTTP 413), 10.1.0 halves the batch and resends each half, down to single
277+
documents. 10.0.0 recorded the whole batch as failed. Code that retried
278+
`result.failed_items` with a smaller `batch_size` can drop that step; what
279+
remains in `failed_items` after a 413 is a single document over the server's
280+
size cap, which no resend fixes. One batch can now land in part, so several
281+
`result.errors` rows can share a `batch_index`.
282+
283+
A 413 from any call now raises `PayloadTooLargeError` rather than a plain
284+
`ApiError`. It is a subclass of `ApiError`, so `except ApiError` still
285+
catches it.
286+
287+
On an index with an `embed` or `sparse_embed` field, the server takes at most
288+
96 documents per upsert or update request. The default `batch_size` of 50
289+
fits; keep any value you pass at 96 or below:
290+
291+
```python
292+
index.documents.batch_upsert(
293+
namespace="movies-en",
294+
documents=[{"_id": f"movie-{i}", "title": f"Movie {i}"} for i in range(500)],
295+
batch_size=96,
296+
)
297+
```
298+
299+
(v10-1-summary)=
300+
## Summary
301+
302+
| Change | Before (10.0.0) | Now (10.1.0) |
303+
| --- | --- | --- |
304+
| Polling `documents.fetch` until `is_reconciled(target)` | exits at once (`lsn_committed` was `None`) | never exits; poll `documents.search` with a deadline |
305+
| Unknown key in a dict `score_by` clause | dropped | `PineconeValueError` |
306+
| Bare `str` for `ids`, `include_fields` or `remove_fields` | split into characters | `PineconeTypeError` |
307+
| Bare `str` for a clause's `fields` | sent as one string | `PineconeTypeError` |
308+
| `timeout` / `total_timeout` of zero, negative or NaN | passed through | `PineconeValueError` |
309+
| Negative wait `timeout` other than `-1` on create, create-for-model, delete | request sent, then `PineconeTimeoutError` | `PineconeValueError`, nothing sent |
310+
| Non-`str` or `None` name or ID | coerced, `AttributeError`, `TypeError` or `PineconeValueError` | `PineconeTypeError` |
311+
| API key with whitespace or control characters | first request fails | `PineconeValueError` at construction |
312+
| Public index host with `http://` | silent | `RuntimeWarning` |
313+
| `additional_headers` in a different capitalization | sent alongside the SDK's header | replaces it |
314+
| 413 in `documents.batch_upsert` | whole batch failed | halved and resent |
315+
| 413 from any call | `ApiError` | `PayloadTooLargeError` (an `ApiError`) |

0 commit comments

Comments
 (0)