Repository navigation
[executorch][native-ggml] Lower split and split_with_sizes - #25373
digantdesai wants to merge 1 commit into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25373
Note: Links to docs will display an error until the docs builds have been completed. ⏳ 3 Pending, 1 Unrelated FailureAs of commit 6c7234d with merge base 90bd617 ( BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
SS-JIA
left a comment
There was a problem hiding this comment.
Review automatically exported from Phabricator review in Meta.
JakeStevens
left a comment
There was a problem hiding this comment.
Diff landed internally; approving.
SS-JIA
left a comment
There was a problem hiding this comment.
Diff landed internally; approving.
…uts to F32 Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0) (oldest at bottom): * #25381 * #25380 * #25379 * #25378 * #25377 * #25376 * #25375 * #25374 * #25373 * #25372 * #25371 * #25370 * #25369 * __->__ #25368 layer_norm widens F16 and BF16 inputs, weight and bias to F32 and normalizes in F32 as eager does; set() rounds the result back. hardtanh (and relu6) widens BF16 to F32, since ggml-cpu clamps only F32 and F16; F16 stays one native ggml_clamp, as in llama.cpp. Rounding half node results at set() left these lowerings refusing (layer_norm) or aborting (BF16 hardtanh) on half inputs that F32-computed lowerings feed. Authored with assistance from Claude and Codex. Differential Revision: D133480458 Pull Request resolved: #25368
…cases Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0) (oldest at bottom): * #25381 * #25380 * #25379 * #25378 * #25377 * #25376 * #25375 * #25374 * #25373 * #25372 * #25371 * #25370 * __->__ #25369 * #25368 test_ggml_ops_cuda and test_ggml_ops_hip run as bundles of 40 cases, so their 165 cases take about 5 test actions instead of one action per case. The CPU variant is unchanged. Each unbundled case paid ~19 s of torch/transformers import for ~3 s of test, so the GPU suites ran past their CI time limit. Nothing hangs or fails. Authored with assistance from Claude and Codex. Differential Revision: D133480459 Pull Request resolved: #25369
Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.15.0) (oldest at bottom): * #25381 * #25380 * #25379 * #25378 * #25377 * #25376 * #25375 * #25374 * #25373 * #25372 * #25371 * __->__ #25370 * #25369 * #25368 run_hf_llm_ggml and ggml_ptn_runner take their engine flags (--device, --device_index, --cpu_threads, --max_cached_graphs, --print_graph_ops) from one GgmlEngineFlags library, and the README gains a table of the engine options that change numerics. Both runners then accept the same flags with the same meaning and help, and a later option is added and documented once. Authored with assistance from Claude and Codex. Differential Revision: D133480460 Pull Request resolved: #25370
Stack from ghstack (oldest at bottom):
Each piece of
splitandsplit_with_sizes(and their copy forms) is a view ofits slice, copied only when a consumer needs it contiguous; the piece count and
sizes are checked against the declared outputs, without overflow.
The ggml engine had no lowering for them, and
chunkand QKV splits exportthem.
Authored with assistance from Claude and Codex.
Differential Revision: D131676083