Repository navigation
Qualcomm AI Engine Direct - Enable LPAI cut plan feature - #25403
Open
shewu-quic wants to merge 1 commit into
Open
shewu-quic wants to merge 1 commit into
shewu-quic wants to merge 1 commit into
Conversation
Depth-first cut is a model-build-time, memory-fitting mechanism for the
LPAI backend: it splits one op's compute (and its working set) into
spatial tiles so a graph that otherwise wouldn't fit into LPAI's
on-chip memory can still compile. It's threaded through as
`lpai_custom_prepare_config_file` on `generate_lpai_compiler_spec`,
pointing at a JSON manifest whose `cut_plan` key references a second
JSON file naming which exported-graph node(s) to cut and how many
height/width tiles to split into.
Writing a cut plan requires knowing exact QNN graph node names, so this
also adds `dump_qnn_graph_path`: a flag that additionally compiles and
exports the graph as a `.dlc` via the Ir backend for inspection with
`qairt-dlc-info` or QAIRT Visualizer, independent of whatever is
already produced for the `.pte`.
Fixed some faied test cases in `test_qnn_delegate.py`
- Use log_msg instead of log_msg.stdout.
- SharedBuffer is not support by X86
- Resolved the use-after-free bug affecting `qnn logger` in
`QnnDlcManager.cpp`.
```
python backends/qualcomm/tests/test_qnn_delegate.py \
TestQNNFloatingPointUtils.test_qnn_backend_dump_qnn_graph \
TestQNNFloatingPointUtils.test_qnn_backend_dump_qnn_graph_multi_partition
\
TestQNNFloatingPointUtils.test_qnn_backend_dump_qnn_graph_multi_method
\
TestQNNQuantizedUtils.test_qnn_backend_dump_qnn_graph \
TestQNNQuantizedUtils.test_qnn_backend_dump_qnn_graph_multi_partition
\
TestQNNQuantizedUtils.test_qnn_backend_dump_qnn_graph_multi_method \
--soc_model SM8850 --build_folder build-android ...
python backends/qualcomm/tests/test_qnn_delegate.py \
TestQNNQuantizedUtils.test_qnn_backend_dfcut \
--soc_model SM8850 --build_folder build-android --backend lpai ...
python -m pytest --import-mode=importlib \
backends/qualcomm/tests/rework/htp/feature/v81/test.py -k
"dump_qnn_graph" \
--soc_model SM8850 --build_folder build-android ...
python -m pytest --import-mode=importlib \
backends/qualcomm/tests/rework/lpai/feature/v6/test.py -k
"depth_first_cut or dump_qnn_graph" \
--soc_model SM8850 --build_folder build-android --backend lpai ...
```
Contributor
Author
|
@pytorchbot label "release notes: qualcomm" |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Depth-first cut is a model-build-time, memory-fitting mechanism for the LPAI backend: it splits one op's compute (and its working set) into spatial tiles so a graph that otherwise wouldn't fit into LPAI's on-chip memory can still compile. It's threaded through as
lpai_custom_prepare_config_fileongenerate_lpai_compiler_spec, pointing at a JSON manifest whosecut_plankey references a second JSON file naming which exported-graph node(s) to cut and how many height/width tiles to split into.Writing a cut plan requires knowing exact QNN graph node names, so this also adds
dump_qnn_graph_path: a flag that additionally compiles and exports the graph as a.dlcvia the Ir backend for inspection withqairt-dlc-infoor QAIRT Visualizer, independent of whatever is already produced for the.pte.custom_prepare_config_filethrough toLpaiBackendCustomConfig(QNN_LPAI_BACKEND_CUSTOM_CFG_PREPARE_CFG), gated on QNN API >= 2.40 (QAIRT SDK 2.51+). Wires the option throughQnnConfig/export_utils.py, theqc_compiler_spec.fbs/qc_schema.pyserialization, andgenerate_lpai_compiler_spec.--dump_qnn_graph_path: when set, additionally exports the delegated graph as a QCIR DLC via the Ir backend to the given directory, alongside the normal context binary (works foronline_prepare, offline, and FCB compile paths). IntroducesQnnManager::CompileIrGraph/GetIrContextBinary, refactors the shared graph-build logic intoQnnManager::BuildGraph, and exposesPyQnnManager::CompileDlcto Python.test_qnn_delegate.pyqnn loggerinQnnDlcManager.cpp.Test Plan
TestQNNQuantizedUtils.test_qnn_backend_dfcut(new,test_qnn_delegate.py): exportsSimpleModelwith a depth-first cut plan via--custom_prepare_config_fileand verifies the delegated graph still produces matching output.backends/qualcomm/tests/rework/lpai/feature/v6/test.py::test_depth_first_cut: valid vs. invalid cut-plan parametrization (invalid no-op cut is expected to raise).dump_qnn_graph_path.