Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AVX2_decode UTF8 #30

Merged
merged 12 commits into from
Sep 5, 2024
Merged

AVX2_decode UTF8 #30

merged 12 commits into from
Sep 5, 2024

Conversation

Nick-Nuon
Copy link
Collaborator

@Nick-Nuon Nick-Nuon commented Aug 25, 2024

Just to give a quick weekly update:this is a draft for the AVX2 decoding,

the tests involving the Enron files and the DoomedPartial test are unfortunately still failing. I am still in debugging mode.

@Nick-Nuon Nick-Nuon changed the title AVX2_decode AVX2_decode UTF8 Aug 25, 2024
@lemire
Copy link
Member

lemire commented Aug 25, 2024

Excellent. After that, we move on to AVX-512 and we'll be ready to run the benchmarks and make the project public.

Be careful because you might hit conflicts with my ARM PR.

@lemire
Copy link
Member

lemire commented Aug 26, 2024

@Nick-Nuon You have trivial conflicts. :-)

@Nick-Nuon
Copy link
Collaborator Author

@Nick-Nuon You have trivial conflicts. :-)

Yeah the conflicts were easy to resolve. ^_^

With a bit of shoulder grease, the draft now passes all tests but unfortunately , it is still too slow :-/

but I am investigating this morning.

Method FileName Mean Error StdDev Speed (GB/s)
AVX2DecodingRealDataUTF8 data/dns/swedenzonebase.txt 50,437.8 us 1,665.09 us 432.42 us .70
AVX2DecodingRealDataWithAllocUTF8 data/dns/swedenzonebase.txt 45,560.1 us 2,263.17 us 587.74 us .77
AVX2DecodingRealDataUTF16 data/dns/swedenzonebase.txt 38,953.8 us 1,678.75 us 435.97 us .90
AVX2DecodingRealDataWithAllocUTF16 data/dns/swedenzonebase.txt 41,831.5 us 555.31 us 144.21 us .84
AVX2DecodingRealDataUTF8 data/email/ 1,299.6 us 31.17 us 8.09 us 1.52
AVX2DecodingRealDataWithAllocUTF8 data/email/ 1,782.1 us 95.50 us 24.80 us 1.11
AVX2DecodingRealDataUTF16 data/email/ 1,366.9 us 23.13 us 3.58 us 1.45
AVX2DecodingRealDataWithAllocUTF16 data/email/ 1,806.2 us 164.07 us 42.61 us 1.09
SSEDecodingRealDataUTF8 data/dns/swedenzonebase.txt 31,978.8 us 3,158.94 us 820.37 us 1.10
SSEDecodingRealDataWithAllocUTF8 data/dns/swedenzonebase.txt 25,265.5 us 1,751.40 us 271.03 us 1.39
SSEDecodingRealDataUTF16 data/dns/swedenzonebase.txt 22,564.2 us 1,545.31 us 401.31 us 1.56
SSEDecodingRealDataWithAllocUTF16 data/dns/swedenzonebase.txt 29,426.0 us 5,128.19 us 1,331.78 us 1.19
SSEDecodingRealDataUTF8 data/email/ 301.1 us 17.34 us 4.50 us 6.56
SSEDecodingRealDataWithAllocUTF8 data/email/ 686.9 us 45.09 us 11.71 us 2.88
SSEDecodingRealDataUTF16 data/email/ 324.5 us 14.13 us 3.67 us 6.09
SSEDecodingRealDataWithAllocUTF16 data/email/ 736.5 us 59.67 us 15.50 us 2.68
DotnetRuntimeSIMDBase64RealDataUTF8 data/dns/swedenzonebase.txt 26,602.4 us 1,103.33 us 170.74 us 1.32
DotnetRuntimeSIMDBase64RealDataWithAllocUTF8 data/dns/swedenzonebase.txt 21,243.5 us 1,008.06 us 261.79 us 1.65
DotnetRuntimeBase64RealDataUTF16 data/dns/swedenzonebase.txt 45,996.4 us 1,085.65 us 281.94 us .76
DotnetRuntimeSIMDBase64RealDataUTF8 data/email/ 421.7 us 15.92 us 4.13 us 4.69
DotnetRuntimeSIMDBase64RealDataWithAllocUTF8 data/email/ 789.0 us 100.67 us 26.14 us 2.51
DotnetRuntimeBase64RealDataUTF16 data/email/ 2,490.5 us 76.34 us 19.82 us .79

@lemire
Copy link
Member

lemire commented Sep 3, 2024

@Nick-Nuon I have a fix upcoming.

@lemire lemire mentioned this pull request Sep 4, 2024
@lemire lemire merged commit a408bd8 into main Sep 5, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants