Skip to content

[ET-VK][conv2d] Prune padding taps in tiled conv2d - #25391

Open
SS-JIA wants to merge 1 commit into
gh/SS-JIA/743/basefrom
gh/SS-JIA/743/head
Open

SS-JIA wants to merge 1 commit into
gh/SS-JIA/743/basefrom
gh/SS-JIA/743/head

Conversation

@SS-JIA

@SS-JIA SS-JIA commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Stack from ghstack (oldest at bottom):

Ulterior Motive

Speed up the dilated 3x3 convolutions of the atrous spatial pyramid pooling (ASPP) head of DeepLab v3 (MobileNetV3-Large backbone) on Mali-G715 (Pixel 9 Pro XL, FP16) and Adreno (Galaxy S24, Adreno 750, FP16 and FP32), without regressing other conv2d paths. The three ASPP convolutions have input [1, 960, 33, 33], weight [256, 960, 3, 3], bias, ReLU, stride 1 and dilation = padding = 12, 24 or 36 (called T12, T24 and T36 below).

Rationale

What: Tiled direct convolutions skip kernel taps that read only padding. 2x2 tiles that read padding through the sampler border do it for the whole output ("global" pruning) or per output tile ("tile" pruning), on every GPU family and for grouped convolutions. 1x1 tiles do it per pixel with the same tile tap range code, which replaces their CLAMP_TAPS shader variant.

Why: With dilation and padding of 12 to 36 on a 33x33 input, most taps of most output pixels read the zero padding. For T36, only the center tap reads the input at all. Skipping those taps removes their weight loads and multiply-adds without changing the result: a skipped tap only ever adds zero.

Details

  • Tap traversal
    • Before: every 2x2 tile loops over all kernel taps.
    • After: global pruning loops over the taps that read the input for at least one output pixel; tile pruning loops over the taps that read the input for at least one pixel of the tile. Taps that read the input for some pixels and padding for others still sample the padding through the sampler border. The summation order of the remaining taps is unchanged.
  • 1x1 tiles
    • Before: CLAMP_TAPS computes the taps that read inside the input for pixels whose kernel footprint crosses the input edge, with its own formula.
    • After: TAP_RANGE: tile computes the same range with the tile formula on a single-pixel tile. Pixels whose footprint lies inside the input still skip the range math. Every kept tap of a single pixel reads inside the input, so these tiles keep their unchecked input fetch and also prune on resized inputs. Kernel names and selection are unchanged.
  • Selection (pick_conv2d_tap_range in the new Conv2dTapRange.h)
    • Exact per-axis counts of the taps each mode keeps give the share of tap work each mode skips.
    • Tile pruning is used when the grid has at least kMinTileTapRangeInvocations (4096) 2x2-pixel x 8-channel tiles and tile pruning skips at least kMinTileTapRangeGainPercent (30%) more of the tap work than global pruning. Per-tile bounds cost about 11% more than global bounds when both skip the same taps; 30% lies between measured shapes that did not gain (0.22) and did (0.42). 4096 is the pointwise underfill threshold, for tiles of the same footprint.
    • Otherwise global pruning is used if it skips any tap; else no pruning.
    • T12 and T24 use tile pruning, T36 global pruning.
    • Both thresholds were measured with two-slice tiles on Mali-G715 and Adreno 750. Work is counted in 2x2-pixel x 8-channel units for one-slice tiles too.
  • Scope
    • 2x2 tiles that read padding through the sampler border, grouped or not: conv2d_2x2x8_border and conv2d_2x2x4_border (texture weights), conv2d_2x2x8_border_buffer and conv2d_2x2x8_border_buffer_offset (Mali buffer weights). The one-slice conv2d_2x2x4_border covers Mali Float, grouped Half on Adreno and irregular shapes on AMD.
    • All GPU families.
    • Pruning only applies while the input is at its allocated extents, like the sampler border it relies on. A resized input selects the variants without tap pruning; restoring the allocated extents restores pruning.
  • Shaders
    • conv2d_tiled.yaml takes a TAP_RANGE parameter (none, global or tile) and adds _tap_range_global / _tap_range_tile variants of the four 2x2 tiles (Half, plus Float for texture weights). Variants with TAP_RANGE: none generate the same SPIR-V as before.
    • conv2d_1x1x4 and conv2d_1x1x4_buffer use TAP_RANGE: tile; the CLAMP_TAPS parameter is removed.
  • Tests
    • test/custom_ops:conv2d_tap_range_test checks the tap bounds and counts against brute force over strides, dilations, paddings and sizes, and the mode picked for ASPP-like and held-out shapes.
    • The conv2d CI collection adds global/tile Half and Float cases, a 17 -> 9 -> 17 resize, and stress shapes: stride 2, different width/height geometry with a 3x5 kernel, odd output slice counts (Mali texture weights), 30 output / 6 input channels, unit dilation with large padding (buffer weights with constant offsets), global pruning with odd slices, and grouped tile and global pruning (one-slice Half and two-slice Float tiles on Adreno) with a 33 -> 17 -> 33 resize.
// Width and height axes: input, output, kernel, stride, padding, dilation.
const Conv2dAxis axis{33, 33, 3, 1, 12, 12};
const Conv2dTapRange mode = pick_conv2d_tap_range({axis, axis}, /*output_channels=*/256);
// mode == Conv2dTapRange::Tile

Differential Revision: D133523544

[ghstack-poisoned]
@pytorch-bot

pytorch-bot Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/25391

Note: Links to docs will display an error until the docs builds have been completed.

⏳ 1 Pending, 1 Unrelated Failure

As of commit 7714e40 with merge base ad017ad (image):

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 20, 2026
@github-actions

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

This branch was previously deployed

1 inactive deployment
cadence — 7714e40b Deployed Nov 20, 2026 by SS-JIA via hifi-op-test / hifi4 #39764
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant