Skip to content

Use precision-matched quantized references in INT4 matmul tests #19323

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
digantdesai wants to merge 1 commit into
base: main
Choose a base branch
from
+327 −100

Use precision-matched quantized references in INT4 matmul tests

70ade20
Select commit
Loading
Failed to load commit list.
Open

Use precision-matched quantized references in INT4 matmul tests #19323

Use precision-matched quantized references in INT4 matmul tests
70ade20
Select commit
Loading
Failed to load commit list.
PyTorch Bot / Dr.CI completed May 7, 2026 in 0s

Dr.CI classification results

{"FAILED":[{"workflowId":25416869963,"workflowUniqueId":195669493,"id":74550053770,"runnerName":"i-08e7a6cee19c890bc","authorEmail":"digantdesai@meta.com","name":"Test CUDA Builds / unittest-cuda / linux-job","jobName":"unittest-cuda / linux-job","conclusion":"failure","completed_at":"2026-05-06T05:31:36.000000000Z","html_url":"https://github.com/pytorch/executorch/actions/runs/25416869963/job/74550053770","logUrl":"https://ossci-raw-job-status.s3.amazonaws.com/log/pytorch/executorch/74550053770","head_branch":"int4_matmul_cuda_test","pr_number":19323,"head_sha":"70ade20926c455706021ffc77d95c2797f979b70","head_sha_timestamp":"2026-05-06T03:20:50.000000000Z","failure_captures":["backends/cuda/tests/test_int4_matmul.py::TestDequantThenMatmul::test_prefill_short"],"failure_lines":["FAILED backends/cuda/tests/test_int4_matmul.py::TestDequantThenMatmul::test_prefill_short - AssertionError: False is not true : fused vs dequant M=64 [2048x2048]: max_abs_err=1.000000"],"failure_context":[],"time":"2026-05-06T04:42:18.000000000Z"},{"workflowId":25416869992,"workflowUniqueId":187910448,"id":74550053609,"runnerName":"i-00dc1cd7b9cd8966b","authorEmail":"digantdesai@meta.com","name":"Test CoreML Backend / test-coreml / test-backend-macos (coreml, models) / macos-job","jobName":"test-coreml / test-backend-macos (coreml, models) / macos-job","conclusion":"cancelled","completed_at":"2026-05-06T08:23:01.000000000Z","html_url":"https://github.com/pytorch/executorch/actions/runs/25416869992/job/74550053609","logUrl":"https://ossci-raw-job-status.s3.amazonaws.com/log/pytorch/executorch/74550053609","head_branch":"int4_matmul_cuda_test","pr_number":19323,"head_sha":"70ade20926c455706021ffc77d95c2797f979b70","head_sha_timestamp":"2026-05-06T03:20:50.000000000Z","failure_captures":["##[error]The operation was canceled."],"failure_lines":["##[error]The operation was canceled."],"failure_context":[],"time":"2026-05-06T04:42:18.000000000Z"},{"workflowId":25416870002,"workflowUniqueId":63454257,"id":74550056394,"runnerName":"i-01f18ef859af5de0c","authorEmail":"digantdesai@meta.com","name":"pull / unittest-editable / linux / linux-job","jobName":"unittest-editable / linux / linux-job","conclusion":"failure","completed_at":"2026-05-06T05:46:39.000000000Z","html_url":"https://github.com/pytorch/executorch/actions/runs/25416870002/job/74550056394","logUrl":"https://ossci-raw-job-status.s3.amazonaws.com/log/pytorch/executorch/74550056394","head_branch":"int4_matmul_cuda_test","pr_number":19323,"head_sha":"70ade20926c455706021ffc77d95c2797f979b70","head_sha_timestamp":"2026-05-06T03:20:50.000000000Z","failure_captures":["examples/models/test/test_export.py::ExportTest::test_mv3_export_to_executorch"],"failure_lines":["FAILED examples/models/test/test_export.py::ExportTest::test_mv3_export_to_executorch - AssertionError: False is not true"],"failure_context":[],"time":"2026-05-06T04:42:20.000000000Z"}],"FLAKY":[{"workflowId":25416870002,"workflowUniqueId":63454257,"id":74550057419,"runnerName":"i-01fb7ca596a19f989","authorEmail":"digantdesai@meta.com","name":"pull / test-qnn-testsuite-linux / test-backend-linux (qnn, models) / linux-job","jobName":"test-qnn-testsuite-linux / test-backend-linux (qnn, models) / linux-job","conclusion":"failure","completed_at":"2026-05-06T05:38:04.000000000Z","html_url":"https://github.com/pytorch/executorch/actions/runs/25416870002/job/74550057419","logUrl":"https://ossci-raw-job-status.s3.amazonaws.com/log/pytorch/executorch/74550057419","head_branch":"int4_matmul_cuda_test","pr_number":19323,"head_sha":"70ade20926c455706021ffc77d95c2797f979b70","head_sha_timestamp":"2026-05-06T03:20:50.000000000Z","failure_captures":[],"failure_lines":[],"failure_context":[],"time":"2026-05-06T04:42:20.000000000Z"}],"BROKEN_TRUNK":[{"workflowId":25416870002,"workflowUniqueId":63454257,"id":74550056670,"runnerName":"i-02eccfbb06b25d3e3","authorEmail":"digantdesai@meta.com","name":"pull / unittest / macos / macos-job","jobName":"unittest / macos / macos-job","conclusion":"cancelled","completed_at":"2026-05-06T07:23:38.000000000Z","html_url":"https://github.com/pytorch/executorch/actions/runs/25416870002/job/74550056670","logUrl":"https://ossci-raw-job-status.s3.amazonaws.com/log/pytorch/executorch/74550056670","head_branch":"int4_matmul_cuda_test","pr_number":19323,"head_sha":"70ade20926c455706021ffc77d95c2797f979b70","head_sha_timestamp":"2026-05-06T03:20:50.000000000Z","failure_captures":["##[error]The operation was canceled."],"failure_lines":["##[error]The operation was canceled."],"failure_context":[],"time":"2026-05-06T04:42:20.000000000Z"},{"workflowId":25416870002,"workflowUniqueId":63454257,"id":74550056443,"runnerName":"i-03ba928aa3373e2d2","authorEmail":"digantdesai@meta.com","name":"pull / unittest-editable / macos / macos-job","jobName":"unittest-editable / macos / macos-job","conclusion":"cancelled","completed_at":"2026-05-06T07:23:30.000000000Z","html_url":"https://github.com/pytorch/executorch/actions/runs/25416870002/job/74550056443","logUrl":"https://ossci-raw-job-status.s3.amazonaws.com/log/pytorch/executorch/74550056443","head_branch":"int4_matmul_cuda_test","pr_number":19323,"head_sha":"70ade20926c455706021ffc77d95c2797f979b70","head_sha_timestamp":"2026-05-06T03:20:50.000000000Z","failure_captures":["##[error]The operation was canceled."],"failure_lines":["##[error]The operation was canceled."],"failure_context":[],"time":"2026-05-06T04:42:20.000000000Z"},{"workflowId":25416869992,"workflowUniqueId":187910448,"id":74550053608,"runnerName":"i-0dddd8bfbc5f67b29","authorEmail":"digantdesai@meta.com","name":"Test CoreML Backend / test-coreml / test-backend-macos (coreml, operators) / macos-job","jobName":"test-coreml / test-backend-macos (coreml, operators) / macos-job","conclusion":"failure","completed_at":"2026-05-06T06:29:36.000000000Z","html_url":"https://github.com/pytorch/executorch/actions/runs/25416869992/job/74550053608","logUrl":"https://ossci-raw-job-status.s3.amazonaws.com/log/pytorch/executorch/74550053608","head_branch":"int4_matmul_cuda_test","pr_number":19323,"head_sha":"70ade20926c455706021ffc77d95c2797f979b70","head_sha_timestamp":"2026-05-06T03:20:50.000000000Z","failure_captures":["RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1"],"failure_lines":["RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1"],"failure_context":[],"time":"2026-05-06T04:42:18.000000000Z"},{"workflowId":25416869963,"workflowUniqueId":195669493,"id":74662057455,"runnerName":"i-01e2bd5a3c69ed423","authorEmail":"digantdesai@meta.com","name":"Test CUDA Builds / test-cuda-pybind (qwen3-0.6b, --quantize, Qwen-Qwen3-0.6B-cuda-quantized-int4-tile-packed) / linux-job","jobName":"test-cuda-pybind (qwen3-0.6b, --quantize, Qwen-Qwen3-0.6B-cuda-quantized-int4-tile-packed) / linux-job","conclusion":"cancelled","completed_at":"2026-05-06T19:00:45.000000000Z","html_url":"https://github.com/pytorch/executorch/actions/runs/25416869963/job/74662057455","logUrl":"https://ossci-raw-job-status.s3.amazonaws.com/log/pytorch/executorch/74662057455","head_branch":"int4_matmul_cuda_test","pr_number":19323,"head_sha":"70ade20926c455706021ffc77d95c2797f979b70","head_sha_timestamp":"2026-05-06T03:20:50.000000000Z","failure_captures":["##[error]The operation was canceled."],"failure_lines":["##[error]The operation was canceled."],"failure_context":[],"time":"2026-05-06T16:59:36.000000000Z"},{"workflowId":25416869963,"workflowUniqueId":195669493,"id":74662057641,"runnerName":"i-09b8da53ff9dd73c5","authorEmail":"digantdesai@meta.com","name":"Test CUDA Builds / test-cuda-pybind (qwen3-0.6b, Qwen-Qwen3-0.6B-cuda-non-quantized) / linux-job","jobName":"test-cuda-pybind (qwen3-0.6b, Qwen-Qwen3-0.6B-cuda-non-quantized) / linux-job","conclusion":"cancelled","completed_at":"2026-05-06T19:00:44.000000000Z","html_url":"https://github.com/pytorch/executorch/actions/runs/25416869963/job/74662057641","logUrl":"https://ossci-raw-job-status.s3.amazonaws.com/log/pytorch/executorch/74662057641","head_branch":"int4_matmul_cuda_test","pr_number":19323,"head_sha":"70ade20926c455706021ffc77d95c2797f979b70","head_sha_timestamp":"2026-05-06T03:20:50.000000000Z","failure_captures":["##[error]The operation was canceled."],"failure_lines":["##[error]The operation was canceled."],"failure_context":[],"time":"2026-05-06T16:59:36.000000000Z"}],"UNSTABLE":[],"AWAITING_APPROVAL":[]}

View more details on PyTorch Bot