Repository navigation
Commit 1d55f9c
authored
Add a Native CPU engine backed by
Summary:
Add `CPUEngineHost` and `CPUEngineContext` using the existing CPU backend's provider selection, memory planning, and execution. Consume standard Native preprocessing output by materializing supported views and functionalizing supported in-place activations during engine compilation. The engine integrates with `Model`/`Session`, shares package constants across methods, writes inputs once into dedicated buffers outside the activation arena so they persist across repeated execution, and converts host dtypes with checked integer narrowing. Support static, unquantized graphs with contiguous constants and public I/O; reject unsupported aliasing and mutation semantics. `current_execution_context` moves from `CPUBackend.cpp` into `cpu_runtime` so `CPUBackend` and the engine read the threadpool the same way, and the unused `ExecutionContext::avx2_fma` flag, which no provider reads, is removed. `tensor_bytes` no longer checks dtypes; `CPUBackend` keeps its own check for the FP32, integer, and boolean dtypes it supports.
Alias handling lives in the shared `functionalize_aliases` pass (`runtime/graph/utils/Aliases.h`), which also hosts the `alias_root`/`has_mutated_alias_use` helpers that Vulkan's `MaterializeViewCopies` now uses instead of its local copies. A mutation is functionalized only when every live alias reaches the mutated value through `view`s alone with the same shape, so the decision does not depend on activation dim-order hints. `as_strided` accepts operands that index their root's storage in order, including the output of a functionalized mutation. The pass has direct graph-level unit tests in `aliases_test`.
Add CMake libraries and installed targets for the CPU runtime, providers, and engine. `EXECUTORCH_BUILD_NATIVE_CPU` requires XNNPACK and optimized kernels. Register optimized kernels wherever available and portable kernels only for operators absent from the optimized list. Shared builds resolve CPU lookups through `executorch_shared`. Include the CPU engine integration suite in CTest.
Authored with OpenAI Codex and Claude Code.
Reviewed By: digantdesai
Differential Revision: D125540986
Pull Request resolved: #23935CPUPlan (#23935)1 parent 6a2784e commit 1d55f9c
31 files changed
Lines changed: 2848 additions & 86 deletions
File tree
- backends
- cpu
- runtime
- test
- native
- runtime
- cpu
- graph
- utils
- vulkan
- passes
- test
- runtime
- cpu
- graph
- xnnpack
- tools/cmake/preset
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
9 | | - | |
10 | 9 | | |
11 | 10 | | |
12 | 11 | | |
| |||
32 | 31 | | |
33 | 32 | | |
34 | 33 | | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | 34 | | |
50 | 35 | | |
51 | 36 | | |
| |||
78 | 63 | | |
79 | 64 | | |
80 | 65 | | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
81 | 79 | | |
82 | 80 | | |
83 | 81 | | |
| |||
95 | 93 | | |
96 | 94 | | |
97 | 95 | | |
98 | | - | |
| 96 | + | |
99 | 97 | | |
100 | 98 | | |
101 | 99 | | |
| |||
219 | 217 | | |
220 | 218 | | |
221 | 219 | | |
222 | | - | |
| 220 | + | |
223 | 221 | | |
224 | 222 | | |
225 | 223 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
145 | 145 | | |
146 | 146 | | |
147 | 147 | | |
148 | | - | |
149 | 148 | | |
150 | 149 | | |
151 | 150 | | |
| |||
609 | 608 | | |
610 | 609 | | |
611 | 610 | | |
612 | | - | |
613 | | - | |
614 | | - | |
| 611 | + | |
| 612 | + | |
615 | 613 | | |
616 | 614 | | |
617 | 615 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
9 | 10 | | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
13 | 14 | | |
14 | 15 | | |
15 | 16 | | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
16 | 27 | | |
17 | 28 | | |
18 | 29 | | |
| |||
91 | 102 | | |
92 | 103 | | |
93 | 104 | | |
94 | | - | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
| 105 | + | |
102 | 106 | | |
103 | | - | |
| 107 | + | |
104 | 108 | | |
105 | 109 | | |
106 | 110 | | |
| |||
114 | 118 | | |
115 | 119 | | |
116 | 120 | | |
117 | | - | |
| 121 | + | |
118 | 122 | | |
119 | 123 | | |
120 | 124 | | |
121 | | - | |
| 125 | + | |
122 | 126 | | |
123 | 127 | | |
124 | 128 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
67 | | - | |
68 | 67 | | |
69 | 68 | | |
70 | 69 | | |
71 | 70 | | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
72 | 76 | | |
73 | 77 | | |
74 | 78 | | |
| |||
135 | 139 | | |
136 | 140 | | |
137 | 141 | | |
138 | | - | |
| 142 | + | |
139 | 143 | | |
140 | 144 | | |
141 | 145 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
66 | 66 | | |
67 | 67 | | |
68 | 68 | | |
69 | | - | |
| 69 | + | |
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
22 | 23 | | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
26 | 27 | | |
27 | 28 | | |
28 | 29 | | |
29 | | - | |
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
21 | 25 | | |
22 | 26 | | |
23 | 27 | | |
| |||
32 | 36 | | |
33 | 37 | | |
34 | 38 | | |
| 39 | + | |
35 | 40 | | |
36 | 41 | | |
37 | 42 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| 41 | + | |
41 | 42 | | |
42 | 43 | | |
43 | 44 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
0 commit comments