Repository navigation
Commit f3d4581
committed
Copy instead of failing when a planned move can't start
A consuming save decides which buffers move before it writes anything, and checks each
source again when its turn comes. Until now, a refusal at that point raised, after the rest
of the output was already written, even though the data was intact and a copy would have
worked. It also took the file lock only at that point, so a file system without `flock`
support passed every check while planning and then failed the save.
Now:
- The lock is tried while planning, before the other checks. A file system without locks
is a reason to copy, shown in the warning. A file that another save is moving still
raises.
- If a buffer can't move any more when its turn comes, for example because it gained a
second name, it is copied into its place, counted in the warning, and synced before the
rename. A source whose size changed still raises.
- The writer reports which buffers it has started to move, and the cleanup uses that list
instead of guessing from the sizes of the source files. A save that loses a shared buffer
to another save no longer keeps its temporary file or claims that it holds the data.
- The output directory is synced after the temporary file is created and after the rename,
so both names survive a crash. This is skipped where a directory can't be opened.
- The output directory is found with `os.path.dirname`, as `move_from` does, so a path
through `link/..` is checked against the directory the file is really created in.
- Cadence's own `save_pte_program` now calls `prog.save`, so it raises on a failed write
like the shared helper does.
Test plan: `exir/_serialize/test/test_cord.py` and `exir/program/test/test_program.py` on
Linux x86_64. New tests cover a file system without locks, a source that gains a second name
after planning, two saves that share a buffer, a first truncation that fails, and the
directory syncs; each fails when its fix is removed. Another new test covers a source on
another file system at the save level, a case that already copied. The `os.path.dirname`
change has no test of its own.1 parent 905c4bc commit f3d4581
5 files changed
Lines changed: 311 additions & 72 deletions
File tree
- backends/cadence/aot
- exir
- _serialize
- test
- program
- test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
290 | 290 | | |
291 | 291 | | |
292 | 292 | | |
293 | | - | |
294 | | - | |
295 | | - | |
296 | | - | |
297 | | - | |
298 | | - | |
| 293 | + | |
299 | 294 | | |
300 | 295 | | |
301 | 296 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| 15 | + | |
15 | 16 | | |
16 | 17 | | |
17 | 18 | | |
| |||
162 | 163 | | |
163 | 164 | | |
164 | 165 | | |
165 | | - | |
166 | | - | |
| 166 | + | |
| 167 | + | |
167 | 168 | | |
168 | 169 | | |
169 | 170 | | |
| |||
174 | 175 | | |
175 | 176 | | |
176 | 177 | | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
177 | 195 | | |
178 | 196 | | |
179 | 197 | | |
180 | 198 | | |
181 | | - | |
182 | 199 | | |
183 | 200 | | |
184 | 201 | | |
| |||
197 | 214 | | |
198 | 215 | | |
199 | 216 | | |
200 | | - | |
201 | | - | |
202 | | - | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
203 | 224 | | |
204 | 225 | | |
205 | | - | |
| 226 | + | |
206 | 227 | | |
207 | | - | |
208 | | - | |
209 | | - | |
210 | | - | |
211 | | - | |
212 | | - | |
213 | | - | |
214 | | - | |
215 | | - | |
216 | | - | |
| 228 | + | |
217 | 229 | | |
218 | 230 | | |
| 231 | + | |
219 | 232 | | |
220 | 233 | | |
221 | 234 | | |
222 | 235 | | |
223 | 236 | | |
224 | 237 | | |
225 | | - | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
226 | 245 | | |
227 | 246 | | |
228 | 247 | | |
| |||
243 | 262 | | |
244 | 263 | | |
245 | 264 | | |
246 | | - | |
247 | | - | |
248 | | - | |
249 | | - | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
250 | 272 | | |
251 | 273 | | |
252 | 274 | | |
| |||
257 | 279 | | |
258 | 280 | | |
259 | 281 | | |
260 | | - | |
| 282 | + | |
| 283 | + | |
261 | 284 | | |
| 285 | + | |
| 286 | + | |
262 | 287 | | |
263 | 288 | | |
264 | 289 | | |
| |||
329 | 354 | | |
330 | 355 | | |
331 | 356 | | |
332 | | - | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
333 | 362 | | |
334 | 363 | | |
335 | 364 | | |
336 | 365 | | |
337 | 366 | | |
338 | 367 | | |
339 | | - | |
340 | | - | |
341 | | - | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
342 | 375 | | |
343 | 376 | | |
344 | 377 | | |
| |||
350 | 383 | | |
351 | 384 | | |
352 | 385 | | |
353 | | - | |
| 386 | + | |
354 | 387 | | |
355 | 388 | | |
356 | 389 | | |
| |||
359 | 392 | | |
360 | 393 | | |
361 | 394 | | |
362 | | - | |
363 | | - | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
364 | 404 | | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
365 | 408 | | |
366 | | - | |
| 409 | + | |
367 | 410 | | |
368 | | - | |
| 411 | + | |
369 | 412 | | |
370 | 413 | | |
371 | 414 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
18 | 19 | | |
19 | 20 | | |
20 | 21 | | |
| |||
624 | 625 | | |
625 | 626 | | |
626 | 627 | | |
627 | | - | |
| 628 | + | |
628 | 629 | | |
629 | 630 | | |
630 | | - | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
631 | 645 | | |
632 | 646 | | |
633 | | - | |
634 | | - | |
635 | | - | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
636 | 654 | | |
637 | 655 | | |
638 | 656 | | |
| |||
0 commit comments