You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 306a9de
Browse filesBrowse the repository at this point in the historyBrowse files
fix(executorch): move compile_and_save's engines into the program instead of copying them
`compile_and_save` writes each engine to a temporary file as soon as it is built. The ExecuTorch
save then copies that file into the `.pte` (or the `.ptd`), and the file is deleted only when the
call returns. So while the program is written, every engine is on disk twice. For a model whose
engine is tens of gigabytes, the copy needs that much more free disk, and an export can fail on a
device that has room for the result.
The backend now hands each of these engine files to ExecuTorch as data that owns the file
(`FileBackedData(path, cleanup=True)`); it is deleted after the save anyway. Inside
`compile_and_save`, the save then writes the program and the `.ptd` with `consume=True`, so
ExecuTorch moves each engine into the program: it copies it back to front and truncates the file as
it goes, and each engine is on disk once. ExecuTorch does this only when the temporary directory,
where the engine files are, is on the same file system as the program; otherwise it copies.
With `generate_etrecord`, the save copies the engines instead. The ETRecord reads the engines after
the program is written, so the save order stays the same as before, and the program is kept when
the ETRecord can't be written.
Every ExecuTorch save now passes `consume` (as `False` outside `compile_and_save`), so this needs
the `consume` argument from pytorch/executorch#25099. It lands after that change, together with an
ExecuTorch pin that has it, which also raises the minimum ExecuTorch version.
Test plan:
- `tests/py/dynamo/executorch/test_api.py`, CPU, with the lowering stubbed:
- New `test_compile_and_save_moves_the_engines`: inside `compile_and_save`'s offload block, the
program and the `.ptd` are written with `consume=True`. It fails without this change.
- New `test_compile_and_save_keeps_the_program_when_the_etrecord_fails`: in the same block with
an ETRecord, the program and the `.ptd` are written first, without `consume`, and the `.pte`
is kept when the ETRecord save raises.
- New `test_save_executorch_keeps_the_program_when_the_etrecord_fails`: the same for a plain
save.
- New `test_save_executorch_does_not_consume`: a plain save writes without `consume`.
- `tests/py/dynamo/executorch/test_compile_and_save.py` on an A100:
- The streaming test now also checks that each engine reaches ExecuTorch as data that owns its
file.
- New `test_engines_move_into_the_program_without_a_second_copy` checks that each engine file
is empty after the program is written.
- New `test_an_etrecord_makes_the_save_copy_the_engines`: with an ETRecord, the engine files
keep their bytes, and both the `.pte` and the ETRecord are written.
0 commit comments