Skip to content

Commit 2312458

Browse files
ethansfngfacebook-github-bot
authored andcommitted
Fix generic im2row NHWC layout to match Python reference
Summary: The generic Cadence `im2row` kernel wrote its NHWC (`channel_last=True`) output in kernel-position-major order `[kp][c]`, but the operator contract — defined by the Python reference in `ref_implementations.py` via `torch.nn.functional.unfold` — is channel-major `[c][kp]`, i.e. column index `c*(kH*kW) + kh*kW + kw`. The conv-lowering pass `ReplaceConvWithIm2RowAndLinear` packs the matmul weights in the same `[c][kp]` order (permute `[OC,kH,kW,IC]` -> `[OC,IC,kH,kW]` -> `[OC,K]`), so the generic kernel's `[kp][c]` output was transposed relative to the weights. This rewrites the generic NHWC branch to write `[c][kp]` (per-channel scatter `data_col[i_col*channels_col + c*num_kp + kp]`) Differential Revision: D110508326
1 parent 8965e51 commit 2312458

1 file changed

Lines changed: 14 additions & 18 deletions

File tree

backends/cadence/generic/operators/op_im2row.cpp

Lines changed: 14 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,6 @@
88

99
#include <executorch/backends/cadence/generic/operators/op_im2row.h>
1010

11-
#include <algorithm>
12-
1311
#include <executorch/runtime/core/exec_aten/util/scalar_type_util.h>
1412

1513
#ifndef DISABLE_ALWAYS_INLINE
@@ -59,34 +57,32 @@ ALWAYS_INLINE void im2row_(
5957
// array of size (out_height * out_width) x channels_col
6058
const int32_t channels_col = channels * kernel_h * kernel_w;
6159

62-
// If the layout is NHWC, we can copy 'channels' worth of contiguous data
63-
// points when performing im2row.
60+
// If the layout is NHWC, the input data is contiguous per-pixel (H, W, C).
61+
// The output layout must match torch.nn.functional.unfold, which is [c][kp]:
62+
// output[c * num_kp + kp] for each output position.
6463
if (channels_last) {
64+
const int32_t num_kp = kernel_h * kernel_w;
6565
// Iterate over the output domain
6666
for (int _h = 0; _h < out_height; ++_h) {
6767
for (int _w = 0; _w < out_width; ++_w) {
6868
int32_t i_col = _h * out_width + _w;
69-
// Each point in the output domain is the result of applying a filter of
70-
// size kernel_h x kernel_w x channels on the input. But since channels
71-
// is contiguous, we will not explicitly have a loop for it.
7269
for (int _kh = 0; _kh < kernel_h; ++_kh) {
7370
int32_t h_im = _h * stride_h - pad_h + _kh * dilation_h;
7471
for (int _kw = 0; _kw < kernel_w; ++_kw) {
7572
int32_t w_im = _w * stride_w - pad_w + _kw * dilation_w;
73+
int32_t kp = _kh * kernel_w + _kw;
7674

77-
// h_im and w_im are the actual height and width coordinates of the
78-
// input tensor from where we need to copy 'channels' points.
79-
const T* __restrict__ slice_im =
80-
data_im + (h_im * width + w_im) * channels;
81-
T* __restrict__ slice_col = data_col + i_col * channels_col +
82-
(_kh * kernel_w + _kw) * channels;
83-
// If the coordinates were within the input domain, we copy
84-
// 'channels' contiguous values. Otherwise we will fill the output
85-
// with 0's.
8675
if (h_im >= 0 && w_im >= 0 && h_im < height && w_im < width) {
87-
memcpy(slice_col, slice_im, channels * sizeof(T));
76+
const T* __restrict__ pixel =
77+
data_im + (h_im * width + w_im) * channels;
78+
for (int _c = 0; _c < channels; ++_c) {
79+
data_col[i_col * channels_col + _c * num_kp + kp] = pixel[_c];
80+
}
8881
} else {
89-
std::fill_n(slice_col, channels, T(in_zero_point));
82+
for (int _c = 0; _c < channels; ++_c) {
83+
data_col[i_col * channels_col + _c * num_kp + kp] =
84+
static_cast<T>(in_zero_point);
85+
}
9086
}
9187
}
9288
}

0 commit comments

Comments
 (0)