You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[ET-VK][conv2d] Prune padding taps in tiled conv2d - #25391
Speed up the dilated 3x3 convolutions of the atrous spatial pyramid pooling (ASPP) head of DeepLab v3 (MobileNetV3-Large backbone) on Mali-G715 (Pixel 9 Pro XL, FP16) and Adreno (Galaxy S24, Adreno 750, FP16 and FP32), without regressing other conv2d paths. The three ASPP convolutions have input [1, 960, 33, 33], weight [256, 960, 3, 3], bias, ReLU, stride 1 and dilation = padding = 12, 24 or 36 (called T12, T24 and T36 below).
Rationale
What: Tiled direct convolutions skip kernel taps that read only padding. 2x2 tiles that read padding through the sampler border do it for the whole output ("global" pruning) or per output tile ("tile" pruning), on every GPU family and for grouped convolutions. 1x1 tiles do it per pixel with the same tile tap range code, which replaces their CLAMP_TAPS shader variant.
Why: With dilation and padding of 12 to 36 on a 33x33 input, most taps of most output pixels read the zero padding. For T36, only the center tap reads the input at all. Skipping those taps removes their weight loads and multiply-adds without changing the result: a skipped tap only ever adds zero.
Details
Tap traversal
Before: every 2x2 tile loops over all kernel taps.
After: global pruning loops over the taps that read the input for at least one output pixel; tile pruning loops over the taps that read the input for at least one pixel of the tile. Taps that read the input for some pixels and padding for others still sample the padding through the sampler border. The summation order of the remaining taps is unchanged.
1x1 tiles
Before: CLAMP_TAPS computes the taps that read inside the input for pixels whose kernel footprint crosses the input edge, with its own formula.
After: TAP_RANGE: tile computes the same range with the tile formula on a single-pixel tile. Pixels whose footprint lies inside the input still skip the range math. Every kept tap of a single pixel reads inside the input, so these tiles keep their unchecked input fetch and also prune on resized inputs. Kernel names and selection are unchanged.
Selection (pick_conv2d_tap_range in the new Conv2dTapRange.h)
Exact per-axis counts of the taps each mode keeps give the share of tap work each mode skips.
Tile pruning is used when the grid has at least kMinTileTapRangeInvocations (4096) 2x2-pixel x 8-channel tiles and tile pruning skips at least kMinTileTapRangeGainPercent (30%) more of the tap work than global pruning. Per-tile bounds cost about 11% more than global bounds when both skip the same taps; 30% lies between measured shapes that did not gain (0.22) and did (0.42). 4096 is the pointwise underfill threshold, for tiles of the same footprint.
Otherwise global pruning is used if it skips any tap; else no pruning.
T12 and T24 use tile pruning, T36 global pruning.
Both thresholds were measured with two-slice tiles on Mali-G715 and Adreno 750. Work is counted in 2x2-pixel x 8-channel units for one-slice tiles too.
Scope
2x2 tiles that read padding through the sampler border, grouped or not: conv2d_2x2x8_border and conv2d_2x2x4_border (texture weights), conv2d_2x2x8_border_buffer and conv2d_2x2x8_border_buffer_offset (Mali buffer weights). The one-slice conv2d_2x2x4_border covers Mali Float, grouped Half on Adreno and irregular shapes on AMD.
All GPU families.
Pruning only applies while the input is at its allocated extents, like the sampler border it relies on. A resized input selects the variants without tap pruning; restoring the allocated extents restores pruning.
Shaders
conv2d_tiled.yaml takes a TAP_RANGE parameter (none, global or tile) and adds _tap_range_global / _tap_range_tile variants of the four 2x2 tiles (Half, plus Float for texture weights). Variants with TAP_RANGE: none generate the same SPIR-V as before.
conv2d_1x1x4 and conv2d_1x1x4_buffer use TAP_RANGE: tile; the CLAMP_TAPS parameter is removed.
Tests
test/custom_ops:conv2d_tap_range_test checks the tap bounds and counts against brute force over strides, dilations, paddings and sizes, and the mode picked for ASPP-like and held-out shapes.
The conv2d CI collection adds global/tile Half and Float cases, a 17 -> 9 -> 17 resize, and stress shapes: stride 2, different width/height geometry with a 3x5 kernel, odd output slice counts (Mali texture weights), 30 output / 6 input channels, unit dilation with large padding (buffer weights with constant offsets), global pruning with odd slices, and grouped tile and global pruning (one-slice Half and two-slice Float tiles on Adreno) with a 33 -> 17 -> 33 resize.
This comment was automatically generated by Dr. CI and updates every 15 minutes.
meta-claBot
added
the
CLA Signed
This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
label
Nov 20, 2026
If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.
To add a label, you can comment to pytorchbot, for example @pytorchbot label "release notes: none"
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CLA SignedThis label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.meta-exported
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
Ulterior Motive
Speed up the dilated 3x3 convolutions of the atrous spatial pyramid pooling (ASPP) head of DeepLab v3 (MobileNetV3-Large backbone) on Mali-G715 (Pixel 9 Pro XL, FP16) and Adreno (Galaxy S24, Adreno 750, FP16 and FP32), without regressing other conv2d paths. The three ASPP convolutions have input
[1, 960, 33, 33], weight[256, 960, 3, 3], bias, ReLU, stride 1 and dilation = padding = 12, 24 or 36 (called T12, T24 and T36 below).Rationale
What: Tiled direct convolutions skip kernel taps that read only padding. 2x2 tiles that read padding through the sampler border do it for the whole output ("global" pruning) or per output tile ("tile" pruning), on every GPU family and for grouped convolutions. 1x1 tiles do it per pixel with the same tile tap range code, which replaces their
CLAMP_TAPSshader variant.Why: With dilation and padding of 12 to 36 on a 33x33 input, most taps of most output pixels read the zero padding. For T36, only the center tap reads the input at all. Skipping those taps removes their weight loads and multiply-adds without changing the result: a skipped tap only ever adds zero.
Details
CLAMP_TAPScomputes the taps that read inside the input for pixels whose kernel footprint crosses the input edge, with its own formula.TAP_RANGE: tilecomputes the same range with the tile formula on a single-pixel tile. Pixels whose footprint lies inside the input still skip the range math. Every kept tap of a single pixel reads inside the input, so these tiles keep their unchecked input fetch and also prune on resized inputs. Kernel names and selection are unchanged.pick_conv2d_tap_rangein the newConv2dTapRange.h)kMinTileTapRangeInvocations(4096) 2x2-pixel x 8-channel tiles and tile pruning skips at leastkMinTileTapRangeGainPercent(30%) more of the tap work than global pruning. Per-tile bounds cost about 11% more than global bounds when both skip the same taps; 30% lies between measured shapes that did not gain (0.22) and did (0.42). 4096 is the pointwise underfill threshold, for tiles of the same footprint.conv2d_2x2x8_borderandconv2d_2x2x4_border(texture weights),conv2d_2x2x8_border_bufferandconv2d_2x2x8_border_buffer_offset(Mali buffer weights). The one-sliceconv2d_2x2x4_bordercovers Mali Float, grouped Half on Adreno and irregular shapes on AMD.conv2d_tiled.yamltakes aTAP_RANGEparameter (none,globalortile) and adds_tap_range_global/_tap_range_tilevariants of the four 2x2 tiles (Half, plus Float for texture weights). Variants withTAP_RANGE: nonegenerate the same SPIR-V as before.conv2d_1x1x4andconv2d_1x1x4_bufferuseTAP_RANGE: tile; theCLAMP_TAPSparameter is removed.test/custom_ops:conv2d_tap_range_testchecks the tap bounds and counts against brute force over strides, dilations, paddings and sizes, and the mode picked for ASPP-like and held-out shapes.Differential Revision: D133523544