Skip to content

Commit 14b5e1d

Browse files
Make Neuron adapter selection version-aware (#24160) (#24160)
Summary: ## Problem A delegated NeuroPilot program did not identify the Neuron adapter ABI used to compile it. Runtime loading therefore relied on legacy soname order and could select an incompatible adapter. A failed compiled-network load could then surface as a usable-looking null delegate instead of an initialization error. ## Solution - Add an optional `neuron-adapter-major` compile spec and preserve it in the serialized ExecuTorch program. - Validate the spec during export as one raw ASCII decimal value from 1 through 255. - Select a versioned adapter first, verify every fallback alias with `Neuron_getVersion`, and reject a different or unknown major. - Pin the first successful adapter ABI process-wide because shim entry points cache resolved function pointers. - Keep adapter selection synchronization alive for the process lifetime so static module teardown cannot lock a destroyed mutex. - Preserve legacy lookup for programs without the new label. - Fail initialization on malformed labels, incompatible adapters, null handles, and compiled-network load failures. - Map Neuron load statuses to specific ExecuTorch errors and stop combining non-bitmask status values. - Keep the optional buffer allocator lazy and absent-safe. This does not claim allocator ABI compatibility across adapter generations; the allocator is loaded only by paths that request persistent imports or shared weights. ## Compatibility Existing programs without `neuron-adapter-major` keep their previous lookup behavior. Existing exporters without `mlir-export-spec` keep the previous converter call. Explicitly labeled programs may use an unversioned alias only when that library reports the requested major. This change is backend infrastructure only. It does not contain an application, model, asset, SDK pin, SoC allowlist, or rollout policy. ## Authorship This diff was authored with Codex. Test Plan: - NeuropilotPreprocessTest: 5 tests passed, including malformed and duplicate label rejection plus a serialized-program round trip proving the label reaches delegate compile specs. - NeuronAdapterLibrarySelectorTest: 18 tests passed, including a shutdown-order regression proving the adapter mutex outlives a static module. - NeuronErrorTest and NeuronBackendInitTest: 6 tests passed. The init-path guardrails cover malformed labels before adapter configuration, runtime-major mismatch, compiled-network load error mapping, and delegate cleanup on load failure. - The Android MediaTek backend built successfully. - Device-runner validation: a program labeled with the matching adapter major loaded and ran; an intentionally mismatched label failed cleanly during initialization. - Fast lint passed; comprehensive lint completed without errors, with only host-platform skips for Android-only compilation units. Differential Revision: D124210809 Pulled By: alirezamonfared Co-authored-by: Ali Monfared <2058201+alirezamonfared@users.noreply.github.com>
1 parent 012fd14 commit 14b5e1d

15 files changed

Lines changed: 1212 additions & 48 deletions

‎backends/mediatek/BUCK‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,32 @@ load("@fbsource//xplat/executorch/build:runtime_wrapper.bzl", "runtime")
33

44
oncall("executorch")
55

6+
fbcode_target(_kind = runtime.python_library,
7+
name = "preprocess_sdk_unbound",
8+
srcs = [
9+
"preprocess.py"
10+
],
11+
labels = ["noautodeps"],
12+
visibility = ["PUBLIC"],
13+
deps = [
14+
"//executorch/exir/_serialize:lib",
15+
"//executorch/exir/backend:backend_api",
16+
"//executorch/exir/passes:lib",
17+
],
18+
)
19+
20+
fbcode_target(_kind = runtime.python_library,
21+
name = "partitioner_sdk_unbound",
22+
srcs = [
23+
"partitioner.py"
24+
],
25+
labels = ["noautodeps"],
26+
visibility = ["PUBLIC"],
27+
deps = [
28+
":preprocess_sdk_unbound",
29+
],
30+
)
31+
632
fbcode_target(_kind = runtime.python_library,
733
name = "preprocess",
834
srcs = [
@@ -27,3 +53,18 @@ fbcode_target(_kind = runtime.python_library,
2753
":preprocess",
2854
],
2955
)
56+
57+
fbcode_target(
58+
_kind = runtime.python_test,
59+
name = "preprocess_test",
60+
py_version = "3.12",
61+
srcs = [
62+
"test_preprocess.py",
63+
],
64+
deps = [
65+
":preprocess",
66+
"//executorch/exir:lib",
67+
"//executorch/exir/_serialize:lib",
68+
"//executorch/exir/backend:backend_api",
69+
],
70+
)

‎backends/mediatek/preprocess.py‎

Lines changed: 41 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,14 +21,41 @@
2121
)
2222
from executorch.exir.backend.compile_spec_schema import CompileSpec
2323

24-
SKIP_COMPILE_SPEC_KEYS = {"ImportForever"}
24+
MLIR_EXPORT_SPEC_KEY = "mlir-export-spec"
25+
NEURON_ADAPTER_MAJOR_KEY = "neuron-adapter-major"
26+
SKIP_COMPILE_SPEC_KEYS = {
27+
"ImportForever",
28+
MLIR_EXPORT_SPEC_KEY,
29+
NEURON_ADAPTER_MAJOR_KEY,
30+
}
2531
EXTRACT_SHARED_BLOB_KEY = "ExtractSharedBlobKey"
2632
HEADER_SIZE = 13
2733
HEADER_VERSION = 1
2834
REQUIRED_COMPILE_SPEC_KEYS = {"platform-config"}
2935
SUPPORTED_PLATFORM_CONFIGS = {"mt6989", "mt6991", "mt6993"}
3036

3137

38+
def _validate_neuron_adapter_major(module_compile_spec: List[CompileSpec]) -> None:
39+
adapter_major_values = [
40+
spec.value
41+
for spec in module_compile_spec
42+
if spec.key == NEURON_ADAPTER_MAJOR_KEY
43+
]
44+
if not adapter_major_values:
45+
return
46+
47+
value = adapter_major_values[0]
48+
if (
49+
not 1 <= len(value) <= 3
50+
or not all(ord("0") <= byte <= ord("9") for byte in value)
51+
or not 1 <= int(value) <= 255
52+
):
53+
raise ValueError(
54+
"neuron-adapter-major CompileSpec must be an ASCII decimal integer "
55+
"between 1 and 255."
56+
)
57+
58+
3259
def assert_default_dim_order(edge_graph_module: torch.fx.GraphModule) -> None:
3360
for node in edge_graph_module.graph.nodes:
3461
if node.op != "placeholder":
@@ -75,6 +102,7 @@ def preprocess(
75102
raise RuntimeError(
76103
"Unsupported duplicated keys in the CompileSpec settings."
77104
)
105+
_validate_neuron_adapter_major(module_compile_spec)
78106
if not REQUIRED_COMPILE_SPEC_KEYS.issubset(set(compile_spec_keys)):
79107
raise RuntimeError(
80108
"Following keys are required in the CompileSpec settings: {}."
@@ -132,8 +160,19 @@ def preprocess(
132160
converter.prepend_input_quantize_ops_indices = fp_input_indices
133161
converter.append_output_dequantize_ops = True
134162
converter.append_output_dequantize_ops_indices = fp_output_indices
163+
mlir_export_spec = next(
164+
(
165+
spec.value.decode("utf-8")
166+
for spec in module_compile_spec
167+
if spec.key == MLIR_EXPORT_SPEC_KEY
168+
),
169+
None,
170+
)
135171
with contextlib.redirect_stdout(None):
136-
mlir_str = converter.convert_to_mlir()
172+
if mlir_export_spec is None:
173+
mlir_str = converter.convert_to_mlir()
174+
else:
175+
mlir_str = converter.convert_to_mlir(mlir_export_spec=mlir_export_spec)
137176
model_bytes = mtk_neuron.compile(mlir_str, " ".join(compile_options))
138177

139178
num_inputs = len(input_names)

‎backends/mediatek/runtime/BUCK‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,56 @@ load("@fbsource//xplat/executorch/build:runtime_wrapper.bzl", "runtime")
1010

1111
oncall("executorch")
1212

13+
non_fbcode_target(
14+
_kind = runtime.cxx_test,
15+
name = "neuron_adapter_library_selector_test",
16+
srcs = [
17+
"test/NeuronAdapterLibrarySelectorTest.cpp",
18+
],
19+
headers = [
20+
"include/api/NeuronAdapterLibrarySelector.h",
21+
],
22+
include_directories = ["include"],
23+
deps = [
24+
"//executorch/runtime/backend:interface",
25+
],
26+
)
27+
28+
non_fbcode_target(
29+
_kind = runtime.cxx_test,
30+
name = "neuron_backend_init_test",
31+
srcs = [
32+
"test/NeuronBackendInitTest.cpp",
33+
],
34+
headers = [
35+
"include/api/NeuronAdapterLibrarySelector.h",
36+
"include/api/NeuronBackendInit.h",
37+
"include/api/NeuronError.h",
38+
],
39+
include_directories = ["include"],
40+
deps = [
41+
"fbsource//third-party/mediatek/neuropilot-express-sdk:runtime_api",
42+
"//executorch/runtime/backend:interface",
43+
"//executorch/runtime/core:core",
44+
],
45+
)
46+
47+
non_fbcode_target(
48+
_kind = runtime.cxx_test,
49+
name = "neuron_error_test",
50+
srcs = [
51+
"test/NeuronErrorTest.cpp",
52+
],
53+
headers = [
54+
"include/api/NeuronError.h",
55+
],
56+
include_directories = ["include"],
57+
deps = [
58+
"fbsource//third-party/mediatek/neuropilot-express-sdk:runtime_api",
59+
"//executorch/runtime/core:core",
60+
],
61+
)
62+
1363
non_fbcode_target(
1464
_kind = runtime.cxx_library,
1565
name = "neuron_backend",
@@ -26,6 +76,7 @@ non_fbcode_target(
2676
deps = [
2777
"fbsource//third-party/mediatek/neuropilot-express-sdk:runtime_api",
2878
"//executorch/runtime/backend:interface",
79+
"//executorch/runtime/core:core",
2980
"//executorch/runtime/core:memory_allocator",
3081
"//executorch/runtime/executor:pte_data_map",
3182
],

‎backends/mediatek/runtime/NeuronBackend.cpp‎

Lines changed: 43 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
#include "NeuronLog.h"
1212
#include "NeuronPayloadHeader.h"
1313
#include "api/NeuronAdapter.h"
14+
#include "api/NeuronBackendInit.h"
1415

1516
#include <executorch/runtime/executor/pte_data_map.h>
1617
#include "executorch/runtime/core/error.h"
@@ -45,6 +46,30 @@ Result<DelegateHandle*> NeuronBackend::init(
4546
BackendInitContext& context,
4647
FreeableBuffer* processed,
4748
ArrayRef<CompileSpec> compile_specs) const {
49+
const auto adapter_init = validateNeuronAdapterForInit(
50+
compile_specs,
51+
configureAdapterLibraryMajor,
52+
[](NeuronRuntimeVersion* version) { return Neuron_getVersion(version); });
53+
if (adapter_init.error != Error::Ok) {
54+
if (adapter_init.versionQueried) {
55+
LogError(
56+
"NeuronBackend",
57+
"Requested Neuron adapter major %d, runtime reported %u (error %d)",
58+
adapter_init.requestedMajor,
59+
static_cast<unsigned>(adapter_init.runtimeVersion.major),
60+
adapter_init.versionStatus);
61+
}
62+
return adapter_init.error;
63+
}
64+
if (adapter_init.requestedMajor > 0) {
65+
LogInfo(
66+
"NeuronBackend",
67+
"Using Neuron adapter version %u.%u.%u",
68+
static_cast<unsigned>(adapter_init.runtimeVersion.major),
69+
static_cast<unsigned>(adapter_init.runtimeVersion.minor),
70+
static_cast<unsigned>(adapter_init.runtimeVersion.patch));
71+
}
72+
4873
NeuronDelegateSetting setting;
4974
MemoryAllocator* runtime_allocator = context.get_runtime_allocator();
5075
NeuronExecuTorchDelegate* delegate =
@@ -65,6 +90,9 @@ Result<DelegateHandle*> NeuronBackend::init(
6590
"NeuronBackend",
6691
"IsImportForever Enable : %d",
6792
setting.mImportForever);
93+
} else if (
94+
std::strcmp(compile_spec.key, kNeuronAdapterMajorKey.data()) == 0) {
95+
continue;
6896
} else if (std::strcmp(compile_spec.key, kSharedWeightsKey) == 0) {
6997
setting.mSharedWeights = true;
7098
std::string shared_weights_key(
@@ -130,14 +158,27 @@ Result<DelegateHandle*> NeuronBackend::init(
130158
Payload.Header.DataLen,
131159
processed->size());
132160

133-
int res = delegate->LoadCompiledNetwork(Payload, setting);
134-
return res == NEURON_NO_ERROR ? delegate : nullptr;
161+
const auto network_load = loadCompiledNetworkForInit(
162+
[&]() { return delegate->LoadCompiledNetwork(Payload, setting); },
163+
[&]() { delegate->~NeuronExecuTorchDelegate(); });
164+
if (network_load.error != Error::Ok) {
165+
LogError(
166+
"NeuronBackend",
167+
"Failed to load compiled network: Neuron status %d mapped to %s",
168+
network_load.status,
169+
runtime::to_string(network_load.error));
170+
return network_load.error;
171+
}
172+
return delegate;
135173
}
136174

137175
Error NeuronBackend::execute(
138176
ET_UNUSED BackendExecutionContext& context,
139177
DelegateHandle* handle,
140178
Span<EValue*> args) const {
179+
if (handle == nullptr) {
180+
return Error::DelegateInvalidHandle;
181+
}
141182
NeuronExecuTorchDelegate* delegate =
142183
reinterpret_cast<NeuronExecuTorchDelegate*>(handle);
143184
return delegate->execute(context, args);

‎backends/mediatek/runtime/NeuronBufferAllocator.cpp‎

Lines changed: 22 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -49,41 +49,54 @@ void unloadLibrary() {
4949

5050
BufferAllocator& BufferAllocator::GetInstance() {
5151
static BufferAllocator instance;
52+
static bool libraryLoadAttempted = false;
53+
static bool allocatorCreateAttempted = false;
5254

53-
if (handle == nullptr) {
55+
std::scoped_lock Guard(instance.mMutex);
56+
if (!libraryLoadAttempted) {
57+
libraryLoadAttempted = true;
5458
loadLibrary();
55-
if (allocatorHandle == nullptr) {
56-
allocatorHandle = create_func();
57-
}
59+
}
60+
if (handle != nullptr && create_func != nullptr &&
61+
!allocatorCreateAttempted) {
62+
allocatorCreateAttempted = true;
63+
allocatorHandle = create_func();
5864
}
5965

6066
return instance;
6167
};
6268

6369
void* BufferAllocator::Allocate(size_t size) {
6470
std::scoped_lock Guard(mMutex);
71+
if (allocate_func == nullptr || allocatorHandle == nullptr) {
72+
return nullptr;
73+
}
6574
return allocate_func(allocatorHandle, size);
6675
}
6776

6877
bool BufferAllocator::RemoveBuffer(void* address) {
6978
std::scoped_lock Guard(mMutex);
79+
if (remove_func == nullptr || allocatorHandle == nullptr) {
80+
return false;
81+
}
7082
return remove_func(allocatorHandle, address);
7183
}
7284

7385
const MemoryUnit* BufferAllocator::Find(void* address) {
7486
std::scoped_lock Guard(mMutex);
87+
if (find_func == nullptr || allocatorHandle == nullptr) {
88+
return nullptr;
89+
}
7590
return static_cast<const MemoryUnit*>(find_func(allocatorHandle, address));
7691
}
7792

7893
void BufferAllocator::Clear() {
7994
std::scoped_lock Guard(mMutex);
80-
clear_func(allocatorHandle);
95+
if (clear_func != nullptr && allocatorHandle != nullptr) {
96+
clear_func(allocatorHandle);
97+
}
8198
}
8299

83100
} // namespace neuron
84101
} // namespace backends
85102
} // namespace executorch
86-
87-
namespace {
88-
static auto& singletonInstance = GET_NEURON_ALLOCATOR;
89-
} // namespace

0 commit comments

Comments
 (0)