Skip to content

Qualcomm AI Engine Direct - Enable LPAI cut plan feature - #25403

Open
shewu-quic wants to merge 1 commit into
pytorch:mainfrom
CodeLinaro:dev1/hutton/enable_dfcut
Open

shewu-quic wants to merge 1 commit into
pytorch:mainfrom
CodeLinaro:dev1/hutton/enable_dfcut

Conversation

@shewu-quic

@shewu-quic shewu-quic commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Depth-first cut is a model-build-time, memory-fitting mechanism for the LPAI backend: it splits one op's compute (and its working set) into spatial tiles so a graph that otherwise wouldn't fit into LPAI's on-chip memory can still compile. It's threaded through as lpai_custom_prepare_config_file on generate_lpai_compiler_spec, pointing at a JSON manifest whose cut_plan key references a second JSON file naming which exported-graph node(s) to cut and how many height/width tiles to split into.

Writing a cut plan requires knowing exact QNN graph node names, so this also adds dump_qnn_graph_path: a flag that additionally compiles and exports the graph as a .dlc via the Ir backend for inspection with qairt-dlc-info or QAIRT Visualizer, independent of whatever is already produced for the .pte.

  • Enables LPAI's depth-first cut plan feature by adding custom_prepare_config_file through to
    LpaiBackendCustomConfig (QNN_LPAI_BACKEND_CUSTOM_CFG_PREPARE_CFG), gated on QNN API >= 2.40 (QAIRT SDK 2.51+). Wires the option through QnnConfig/export_utils.py, the qc_compiler_spec.fbs/qc_schema.py serialization, and generate_lpai_compiler_spec.
  • Adds --dump_qnn_graph_path: when set, additionally exports the delegated graph as a QCIR DLC via the Ir backend to the given directory, alongside the normal context binary (works for online_prepare, offline, and FCB compile paths). Introduces QnnManager::CompileIrGraph/GetIrContextBinary, refactors the shared graph-build logic into QnnManager::BuildGraph, and exposes PyQnnManager::CompileDlc to Python.
  • Fixed some faied test cases in test_qnn_delegate.py
    • Use log_msg instead of log_msg.stdout.
    • SharedBuffer is not support by X86
    • Resolved the use-after-free bug affecting qnn logger in QnnDlcManager.cpp.

Test Plan

  • TestQNNQuantizedUtils.test_qnn_backend_dfcut (new, test_qnn_delegate.py): exports SimpleModel with a depth-first cut plan via --custom_prepare_config_file and verifies the delegated graph still produces matching output.
  • backends/qualcomm/tests/rework/lpai/feature/v6/test.py::test_depth_first_cut: valid vs. invalid cut-plan parametrization (invalid no-op cut is expected to raise).
  • Existing HTP rework feature tests (v68/v69/v73/v75/v79/v81) and LPAI v6 extended to cover dump_qnn_graph_path.
python backends/qualcomm/tests/test_qnn_delegate.py \
    TestQNNFloatingPointUtils.test_qnn_backend_dump_qnn_graph \
    TestQNNFloatingPointUtils.test_qnn_backend_dump_qnn_graph_multi_partition
\
    TestQNNFloatingPointUtils.test_qnn_backend_dump_qnn_graph_multi_method
\
    TestQNNQuantizedUtils.test_qnn_backend_dump_qnn_graph \
    TestQNNQuantizedUtils.test_qnn_backend_dump_qnn_graph_multi_partition
\
    TestQNNQuantizedUtils.test_qnn_backend_dump_qnn_graph_multi_method \
    --soc_model SM8850 --build_folder build-android ...

python backends/qualcomm/tests/test_qnn_delegate.py \
    TestQNNQuantizedUtils.test_qnn_backend_dfcut \
    --soc_model SM8850 --build_folder build-android --backend lpai ...

python -m pytest --import-mode=importlib \
    backends/qualcomm/tests/rework/htp/feature/v81/test.py -k
"dump_qnn_graph" \
    --soc_model SM8850 --build_folder build-android ...

python -m pytest --import-mode=importlib \
    backends/qualcomm/tests/rework/lpai/feature/v6/test.py -k
"depth_first_cut or dump_qnn_graph" \
    --soc_model SM8850 --build_folder build-android --backend lpai ...

Depth-first cut is a model-build-time, memory-fitting mechanism for the
LPAI backend: it splits one op's compute (and its working set) into
spatial tiles so a graph that otherwise wouldn't fit into LPAI's
on-chip memory can still compile. It's threaded through as
`lpai_custom_prepare_config_file` on `generate_lpai_compiler_spec`,
pointing at a JSON manifest whose `cut_plan` key references a second
JSON file naming which exported-graph node(s) to cut and how many
height/width tiles to split into.

Writing a cut plan requires knowing exact QNN graph node names, so this
also adds `dump_qnn_graph_path`: a flag that additionally compiles and
exports the graph as a `.dlc` via the Ir backend for inspection with
`qairt-dlc-info` or QAIRT Visualizer, independent of whatever is
already produced for the `.pte`.

Fixed some faied test cases in `test_qnn_delegate.py`
  - Use log_msg instead of log_msg.stdout.
  - SharedBuffer is not support by X86
  - Resolved the use-after-free bug affecting `qnn logger` in
    `QnnDlcManager.cpp`.

```
python backends/qualcomm/tests/test_qnn_delegate.py \
    TestQNNFloatingPointUtils.test_qnn_backend_dump_qnn_graph \
    TestQNNFloatingPointUtils.test_qnn_backend_dump_qnn_graph_multi_partition
\
    TestQNNFloatingPointUtils.test_qnn_backend_dump_qnn_graph_multi_method
\
    TestQNNQuantizedUtils.test_qnn_backend_dump_qnn_graph \
    TestQNNQuantizedUtils.test_qnn_backend_dump_qnn_graph_multi_partition
\
    TestQNNQuantizedUtils.test_qnn_backend_dump_qnn_graph_multi_method \
    --soc_model SM8850 --build_folder build-android ...

python backends/qualcomm/tests/test_qnn_delegate.py \
    TestQNNQuantizedUtils.test_qnn_backend_dfcut \
    --soc_model SM8850 --build_folder build-android --backend lpai ...

python -m pytest --import-mode=importlib \
    backends/qualcomm/tests/rework/htp/feature/v81/test.py -k
"dump_qnn_graph" \
    --soc_model SM8850 --build_folder build-android ...

python -m pytest --import-mode=importlib \
    backends/qualcomm/tests/rework/lpai/feature/v6/test.py -k
"depth_first_cut or dump_qnn_graph" \
    --soc_model SM8850 --build_folder build-android --backend lpai ...
```
@pytorch-bot

pytorch-bot Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25403

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 25 Awaiting Approval

As of commit 69ca293 with merge base f8907c1 (image):

AWAITING APPROVAL - The following workflows need approval before CI can run:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 21, 2026
@shewu-quic

Copy link
Copy Markdown
Contributor Author

@pytorchbot label "release notes: qualcomm"

@pytorch-bot pytorch-bot Bot added the release notes: qualcomm Changes to the Qualcomm backend delegate label Nov 21, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. release notes: qualcomm Changes to the Qualcomm backend delegate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant