brgemm: support arbitrary K on AMX

[ankalinin]

There are two problems regarding brgemm K value on AMX:

brgemm doesn't support K not divisible by vnni granularity
for K not divisible by tile width (32 or 64) the blocking by K dimension may be not optimal.
To get around this limitation brgemm primitives are forced to either transform the matrix A or call many small brgemm kernels with different tile configurations. Both ways leads to performance lost.
This PR implements support of arbitrary K in brgemm on AMX and implements corresponding updates in 1x1 convolutions and in matmul to use this new ability.

Performance update for convolutions:

Performance update for matmul

[ankalinin]

make test
disable device_gpu

[theComputeKid]

I restarted the AArch64 CI because the encountered failure is a known sporadic bug on the c7g.

[theComputeKid]

cc: @Radu2k

[ankalinin]

I restarted the AArch64 CI because the encountered failure is a known sporadic bug on the c7g.

Thanks!

[Radu2k]

Hi @ankalinin, the code looks good to me, but even if this is a minimal invasive AArch64 change we need to run a performance analysis to be on the safe side. Could you please let me know what benchdnn test/s did you run to get the benchmark numbers? We will run them for AArch64 and then should be good to go in if no major regressions show up.

[ankalinin]

Hi @ankalinin, the code looks good to me, but even if this is a minimal invasive AArch64 change we need to run a performance analysis to be on the safe side. Could you please let me know what benchdnn test/s did you run to get the benchmark numbers? We will run them for AArch64 and then should be good to go in if no major regressions show up.

Hi, @Radu2k. The changes were made in AArch64 code only to avoid compile errors. I don't expect any performance changes in AArch64 part.
What benchdnn testing do you usually use for performance testing?

Anyway, for cpu commands may be like this:
benchdnn --mode=P --matmul --batch=tests/benchdnn/inputs/matmul/harness_matmul_runtime_f32
benchdnn --mode=P --conv --batch=tests/benchdnn/inputs/conv/harness_conv_f32

[vpirogov]

We will run them for AArch64 and then should be good to go in if no major regressions show up.

@Radu2k, if you still plan to run performance validation considering @ankalinin's explanation please let us know when you plan to do that. I would really like to promote these changes by v3.7 code freeze, which is this Friday.

[tprimak]

@Radu2k the change renames a variable and adds initialization for another 2 variables. This not exactly a kind of change that requires a performance study

[Radu2k]

@ankalinin @vpirogov I have just finished running the performance checks and it looks fine, no regressions showed up.

@tprimak As mentioned above, even if it is a minimal invasive change, we run these lightweight performances checks for all PRs before approving.

[vpirogov]

Thanks, @Radu2k!