You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 3cf1702
Browse filesBrowse the repository at this point in the historyBrowse files
fix(executorch): move compile_and_save's engines into the program instead of copying them
`compile_and_save` writes each engine to a temporary file as soon as it is built. The ExecuTorch save
then copies that file into the `.pte` (or the `.ptd`), and the file is deleted only when the call
returns. So while the program is written, every engine is on disk twice. For a model whose engine
is tens of gigabytes, the copy needs that much more free disk, and an export can fail on a device
that has room for the result.
The backend now hands each of these engine files to ExecuTorch as data that owns the file
(`FileBackedData(path, cleanup=True)`); it is deleted after the save anyway. Inside
`compile_and_save`, the save then writes the program with `consume=True`, so ExecuTorch moves each
engine into the program: it copies it back to front and truncates the file as it goes, and each
engine is on disk once. ExecuTorch does this only when the temporary directory, where the engine
files are, is on the same file system as the program; otherwise it copies. The save checks that
ExecuTorch's `write_to_file` takes `consume`; an ExecuTorch without it keeps copying, as before.
After the move the engine file is empty, while the compiled module still maps it. Reading the
mapping then is a bus error. The ETRecord reads the engines, so when the save consumes, the
ETRecord is saved before the program, and an ETRecord that can't be saved stops the save before
the program is written. Every other save keeps the old order: the program is written first, so it
is kept even when the ETRecord can't be saved.
Test plan:
- `tests/py/dynamo/executorch/test_api.py`, CPU, with the lowering stubbed:
- New `test_compile_and_save_saves_the_etrecord_before_the_engines_move` checks the calls inside
`compile_and_save`'s offload block: the ETRecord first, then the program with `consume=True`.
It fails without this change. The `.ptd` write consumes too.
- New `test_compile_and_save_copies_with_an_executorch_that_cannot_consume`: with an ExecuTorch
whose writes take no `consume`, the same save copies, in the old order.
- New `test_save_executorch_keeps_the_program_when_the_etrecord_fails`: a plain save whose
ETRecord save raises still writes the `.pte` first, without `consume`.
- `tests/py/dynamo/executorch/test_compile_and_save.py` on an A100:
- New `test_engines_move_into_the_program_without_a_second_copy` checks that each engine file
is empty after the program is written. It skips, with a reason, when ExecuTorch can only copy.
With an ExecuTorch that can move and without this change, the files keep their full size.
- New `test_an_etrecord_is_saved_from_the_engines_before_they_move`. With the move but the old
order, the process dies with a bus error.
0 commit comments