Search for first duplicate in vectorized unique

[AlexGuteniev]

🔍 Missing piece

Vectorized remove has carefully considered find before actual removal loop. But the similar adjacent_find was missed in unique.

The effect is at least performance. Find is faster as vectorization, and it also avoids extra writes. I doubt about whether they have correctness impact though.

I've added the missing piece in separately compiled code. This is not the same as in remove, where it is in the header. But unique has in-place search for duplicates in scalar part that is tightly coupled with the rest of it, so I thought it is better to avoid disrupting that.

🏁 Benchmark

The modified benchmark adds two regions without duplicates: at the beginning and at the end of the range. The beginning one is avoidable with adjacent_find.

I've ran the modified benchmark for the main before #5092, to have the updated non-vectorized baseline to compare against

⏱️ Benchmark results

Benchmark	Before vectorization	Before PR	After PR
u<alg_type::std_fn, std::uint8_t>	1181 ns	540 ns	217 ns
u<alg_type::std_fn, std::uint16_t>	1075 ns	649 ns	208 ns
u<alg_type::std_fn, std::uint32_t>	1204 ns	775 ns	285 ns
u<alg_type::std_fn, std::uint64_t>	1110 ns	1482 ns	551 ns
u<alg_type::rng, std::uint8_t>	1233 ns	540 ns	221 ns
u<alg_type::rng, std::uint16_t>	1059 ns	653 ns	207 ns
u<alg_type::rng, std::uint32_t>	1298 ns	785 ns	285 ns
u<alg_type::rng, std::uint64_t>	1216 ns	1490 ns	629 ns

💡 Results interpretation

The effect is way more than expected. in particular, the "Before PR" column has shown that vectorization performance has severely degraded after adding already-unique ranges parts to the input.

Apparently it is due to reading just written output. Before the first removed element we read back last element previously written along with newly read elements. The mix of cache data and store buffer data causes the CPU to stall until the store buffer entry reaches the cache. So when we skip to the first non-unique, it is no longer a problem.

[StephanTLavavej]

I'm mirroring this to the MSVC-internal repo - please notify me if any further changes are pushed.

[StephanTLavavej]

🏇 🏎️ 🚀