Error occurred when loading ANE module (MPSGraphExecutable.mm:3543; appleneuralengine Code=53 "Program load failed — no memory") — FP16 asset aborts the process on ANE load instead of returning an error

[john-rocky]

What happens

Loading some FP16 .aimodel assets fails to load their ANE program, and MPSGraph then raises a failed assertion rather than returning an error or falling back. The process aborts. Nothing the caller writes can catch it.

Error = Error Domain=com.apple.appleneuralengine Code=53
  "createProgramInstanceForModel:...: Program load failed — no memory (transient; retry under lower
   memory pressure) (underlying=0x1)" UserInfo={_ANEErrorUnderlyingStatus=1, _ANEErrorLoadStage=4}

MPSGraphExecutable.mm:3543: failed assertion `Error occurred when loading ANE module:
  Error Domain=MPSGraph Code=-1 "MPSGraphExecutable_Project.h:510:: could not load module from
  MPSGraphPackage"'

The same asset loads and runs with SpecializationOptions.cpu_only(). So this is the ANE program specifically, and the abort is instead of a fallback.

The message says the condition is transient and suggests retrying. It is not transient here: it reproduces on every run, in a fresh process, on an idle machine with memory free. Two different graphs reproduce it, and one of them is 12 MB.

Environment

M4 Max, macOS 27.0 beta 6 (26A5416b), coreai-torch 0.4.2, coreai-core 1.0.0b2, Python 3.12.

Reproducer

import torch, coreai_torch, asyncio, inspect, numpy as np
from pathlib import Path
from coreai_torch import TorchConverter
from coreai.runtime import AIModel, NDArray, SpecializationOptions

class Net(torch.nn.Module):
    def __init__(self):
        super().__init__()
        ch = [3, 32, 64, 128, 128]
        self.blocks = torch.nn.Sequential(*[
            torch.nn.Sequential(torch.nn.Conv2d(ch[i], ch[i + 1], 3, stride=2, padding=1),
                                torch.nn.SiLU())
            for i in range(4)
        ])
        self.head = torch.nn.Conv2d(128, 80, 1)

    def forward(self, x):
        s = self.head(self.blocks(x)).flatten(2).transpose(1, 2).max(dim=-1)[0]
        return s.topk(300, dim=1)[0]

net = Net().eval().half()
ex = torch.rand(1, 3, 640, 640).half()
with torch.no_grad():
    ep = torch.export.export(net, (ex,)).run_decompositions(coreai_torch.get_decomp_table())
c = TorchConverter()
c.add_exported_program(ep, entrypoint_name="main", input_names=["image"], output_names=["out0"])
prog = c.to_coreai(); prog.optimize(); prog.save_asset(Path("fp16.aimodel"))

async def go(options=None):
    m = AIModel.load(Path("fp16.aimodel"), options) if options else AIModel.load(Path("fp16.aimodel"))
    m = await m if inspect.isawaitable(m) else m
    f = m.load_function("main")
    f = await f if inspect.isawaitable(f) else f
    o = f({"image": NDArray(np.random.rand(1, 3, 640, 640).astype(np.float16))})
    return await o if inspect.isawaitable(o) else o

asyncio.run(go(SpecializationOptions.cpu_only()))   # OK
asyncio.run(go())                                   # aborts

I hit this on two real graphs first — an FP16 YOLO26-pose export at 640 and a smaller detection export at 32 — before reducing it to the above. The FP32 versions of both load and run on the ANE without complaint, and several other FP16 exports of similar size do too, so it is not every FP16 asset.

What I am asking for

An error instead of an abort. A caller can retry, fall back to the GPU, or report the problem, but only if AIModel.load returns rather than aborting the process. As it stands, an application that ships an FP16 asset can be terminated by a runtime condition it cannot detect or handle.

Whether the ANE program should load at all is the second question, and the more useful one if the "no memory" diagnosis is wrong — the machine has memory, and the condition does not clear on retry.

Context: found while adding a format="coreai" export to Ultralytics (ultralytics/ultralytics#25926). That PR exports FP16 but does not load it back, for this reason.