Repository navigation
Commit 92806ee
committed
test(inference): tolerate model non-determinism in bare-string embed test
test_embed_bare_string_auto_wrapped_{rest,async} called embed() twice
with the same input (once as a bare string, once wrapped in a list)
and asserted bit-exact equality of the returned values. The
multilingual-e5-large model isn't bit-deterministic across replicas —
two consecutive calls regularly drift in the 3rd–4th decimal. That
flake just took down release-readiness on run 25746418051.
The SDK contract being verified here is that the bare-string path
doesn't transform the input differently from the single-element-list
path, not that the model itself is deterministic. Switch the value
comparison to pytest.approx(abs=1e-3), matching the existing
convention from test_query_namespaces_tie_breaking_rest. Length
check is retained so a shape-changing regression still fails.1 parent 17390dc commit 92806ee
2 files changed
Lines changed: 14 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
234 | 234 | | |
235 | 235 | | |
236 | 236 | | |
237 | | - | |
238 | | - | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
239 | 244 | | |
240 | 245 | | |
241 | 246 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
190 | 190 | | |
191 | 191 | | |
192 | 192 | | |
193 | | - | |
194 | | - | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
195 | 200 | | |
196 | 201 | | |
197 | 202 | | |
| |||
0 commit comments