Repository navigation
Commit 61129f4
committed
Serialize a constant left on the GPU from a host copy
When a module is still on the GPU at export, as in flows that compile
it there first such as Torch-TensorRT, a constant that no delegate
takes stays in the program with its storage on the GPU. That happens
when forward() returns a buffer, alone or next to a computed result,
or when a delegate reads a constant from outside it. to_executorch()
then raised an InternalError (and crashed before #24202).
A program stores constants as host bytes whatever device they came
from. So when only delegates, copies between host and device memory,
or the method output read such a constant, the emitter now reads it
from a host copy instead of raising. The constant keeps its place and
device information in the program, so the runtime treats it as it
would the same constant exported from the CPU.
The InternalError stays when an operator outside any delegate reads
the constant. That operator was traced on the GPU but runs on a CPU
kernel, which may not match (#21545): a GPU batch norm lowered with no
backend, for example, gets statistics of a shape the CPU kernel
rejects, and fails on every call. Storage with no data, such as
"meta", still raises too.
Test plan: CUDA tests export a GPU module that returns (x * 2, buffer)
with the CudaPartitioner, and a GPU module whose constant a delegate
reads from outside it. Both run, match eager, and fail without the
change. Another test checks that a GPU Linear lowered with no backend
still raises, and the "meta" test now also covers a buffer that only the
output reads.1 parent aa83485 commit 61129f4
4 files changed
Lines changed: 112 additions & 23 deletions
File tree
- backends/cuda/tests
- exir/emit
- test
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
23 | 28 | | |
| 29 | + | |
24 | 30 | | |
25 | 31 | | |
26 | 32 | | |
| |||
1088 | 1094 | | |
1089 | 1095 | | |
1090 | 1096 | | |
| 1097 | + | |
| 1098 | + | |
| 1099 | + | |
| 1100 | + | |
| 1101 | + | |
| 1102 | + | |
| 1103 | + | |
| 1104 | + | |
| 1105 | + | |
| 1106 | + | |
| 1107 | + | |
| 1108 | + | |
| 1109 | + | |
| 1110 | + | |
| 1111 | + | |
| 1112 | + | |
| 1113 | + | |
| 1114 | + | |
| 1115 | + | |
| 1116 | + | |
| 1117 | + | |
| 1118 | + | |
| 1119 | + | |
| 1120 | + | |
| 1121 | + | |
| 1122 | + | |
| 1123 | + | |
| 1124 | + | |
| 1125 | + | |
| 1126 | + | |
| 1127 | + | |
| 1128 | + | |
| 1129 | + | |
| 1130 | + | |
| 1131 | + | |
| 1132 | + | |
| 1133 | + | |
1091 | 1134 | | |
1092 | 1135 | | |
1093 | 1136 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
122 | 122 | | |
123 | 123 | | |
124 | 124 | | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
125 | 145 | | |
126 | 146 | | |
127 | 147 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
214 | 214 | | |
215 | 215 | | |
216 | 216 | | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
217 | 228 | | |
218 | 229 | | |
219 | 230 | | |
| |||
467 | 478 | | |
468 | 479 | | |
469 | 480 | | |
470 | | - | |
471 | | - | |
472 | | - | |
473 | | - | |
474 | | - | |
475 | | - | |
476 | | - | |
477 | | - | |
478 | | - | |
479 | | - | |
480 | | - | |
481 | | - | |
482 | | - | |
483 | | - | |
484 | | - | |
485 | | - | |
| 481 | + | |
| 482 | + | |
| 483 | + | |
| 484 | + | |
| 485 | + | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
486 | 499 | | |
487 | | - | |
| 500 | + | |
488 | 501 | | |
489 | 502 | | |
490 | 503 | | |
491 | 504 | | |
492 | | - | |
| 505 | + | |
493 | 506 | | |
494 | 507 | | |
495 | 508 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1858 | 1858 | | |
1859 | 1859 | | |
1860 | 1860 | | |
1861 | | - | |
1862 | | - | |
1863 | | - | |
1864 | | - | |
| 1861 | + | |
| 1862 | + | |
| 1863 | + | |
| 1864 | + | |
| 1865 | + | |
| 1866 | + | |
| 1867 | + | |
| 1868 | + | |
| 1869 | + | |
| 1870 | + | |
| 1871 | + | |
| 1872 | + | |
| 1873 | + | |
| 1874 | + | |
| 1875 | + | |
| 1876 | + | |
| 1877 | + | |
1865 | 1878 | | |
1866 | 1879 | | |
1867 | 1880 | | |
| |||
0 commit comments