Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement benchmarks for vector operations #3

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

sbaldu
Copy link
Contributor

@sbaldu sbaldu commented Feb 25, 2025

This PR implements benchmarks for the heterogeneous vector operations using Google Benchmarks.

@mdessole mdessole self-requested a review February 25, 2025 15:28
@mdessole mdessole self-assigned this Feb 25, 2025
Copy link
Contributor

@mdessole mdessole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your PR Simone, it is much appreciated!

I found a few typos regarding your benchmarking code. Moreover, I could not make it compile for any SYCL backend. Did you run tests with OneAPI or AdaptiveCPP? Maybe I am missing something here.

endif()

if (benchmarking)
Copy link
Contributor

@mdessole mdessole Feb 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I would require google benchmark here instead of relying on git modules
find_package(benchmark REQUIRED)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now I am using it as a submodule because I thought that it would make it easier to use on remote machine where maybe it isn't installed, but if you think it's better I can remove the submodule and just use the find package.

@sbaldu
Copy link
Contributor Author

sbaldu commented Feb 28, 2025

Thanks for your PR Simone, it is much appreciated!

I found a few typos regarding your benchmarking code. Moreover, I could not make it compile for any SYCL backend. Did you run tests with OneAPI or AdaptiveCPP? Maybe I am missing something here.

Actually I am having problems with sycl even when compiling the tests already present, so maybe it's a problem with my machine. I'm trying to sort it out, I'll let you know.

@mdessole
Copy link
Contributor

mdessole commented Feb 28, 2025

Thanks for your PR Simone, it is much appreciated!
I found a few typos regarding your benchmarking code. Moreover, I could not make it compile for any SYCL backend. Did you run tests with OneAPI or AdaptiveCPP? Maybe I am missing something here.

Actually I am having problems with sycl even when compiling the tests already present, so maybe it's a problem with my machine. I'm trying to sort it out, I'll let you know.

Maybe you'll find this preprint helpful, it contains compiling instruction for SYCL backends (see appendix A.2)

@sbaldu sbaldu force-pushed the feature-benchmarks branch 5 times, most recently from 20db72a to 79fc6c8 Compare February 28, 2025 14:10
@sbaldu
Copy link
Contributor Author

sbaldu commented Feb 28, 2025

Since you've merged #4 I've added benchmarks for the streamed versions of CUDA as well

@sbaldu
Copy link
Contributor Author

sbaldu commented Feb 28, 2025

Thanks for your PR Simone, it is much appreciated!
I found a few typos regarding your benchmarking code. Moreover, I could not make it compile for any SYCL backend. Did you run tests with OneAPI or AdaptiveCPP? Maybe I am missing something here.

Actually I am having problems with sycl even when compiling the tests already present, so maybe it's a problem with my machine. I'm trying to sort it out, I'll let you know.

Maybe you'll find this preprint helpful, it contains compiling instruction for SYCL backends (see appendix A.2)

I was compiling in the way described in the paper and in the README but it stil gives error:

/opt/intel/oneapi/compiler/2025.0/bin/compiler/clang-offload-bundler: error: '/home/sbaldu/Documents/Codes/genvectorx/build/CMakeFiles/SYCLMath.dir/src/MathUtil.cxx.o': No such file or directory
icpx: error: clang-offload-bundler command failed with exit code 1 (use -v to see invocation)
gmake[2]: *** [CMakeFiles/SYCLMath.dir/build.make:74: CMakeFiles/SYCLMath.dir/src/MathUtil.cxx.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:117: CMakeFiles/SYCLMath.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

On the other hand, I've tested the AdaptiveCPP backend and the benchmarks work correctly.

@mdessole
Copy link
Contributor

mdessole commented Mar 8, 2025

Thanks for your PR Simone, it is much appreciated!
I found a few typos regarding your benchmarking code. Moreover, I could not make it compile for any SYCL backend. Did you run tests with OneAPI or AdaptiveCPP? Maybe I am missing something here.

Actually I am having problems with sycl even when compiling the tests already present, so maybe it's a problem with my machine. I'm trying to sort it out, I'll let you know.

Maybe you'll find this preprint helpful, it contains compiling instruction for SYCL backends (see appendix A.2)

I was compiling in the way described in the paper and in the README but it stil gives error:

/opt/intel/oneapi/compiler/2025.0/bin/compiler/clang-offload-bundler: error: '/home/sbaldu/Documents/Codes/genvectorx/build/CMakeFiles/SYCLMath.dir/src/MathUtil.cxx.o': No such file or directory
icpx: error: clang-offload-bundler command failed with exit code 1 (use -v to see invocation)
gmake[2]: *** [CMakeFiles/SYCLMath.dir/build.make:74: CMakeFiles/SYCLMath.dir/src/MathUtil.cxx.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:117: CMakeFiles/SYCLMath.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

On the other hand, I've tested the AdaptiveCPP backend and the benchmarks work correctly.

I've also tested successfully with AdaptiveCpp. I think some changes to the cmake file are necessary to make it work with OneAPI, I am trying to figure it out.

@sbaldu
Copy link
Contributor Author

sbaldu commented Mar 9, 2025

Thanks for your PR Simone, it is much appreciated!
I found a few typos regarding your benchmarking code. Moreover, I could not make it compile for any SYCL backend. Did you run tests with OneAPI or AdaptiveCPP? Maybe I am missing something here.

Actually I am having problems with sycl even when compiling the tests already present, so maybe it's a problem with my machine. I'm trying to sort it out, I'll let you know.

Maybe you'll find this preprint helpful, it contains compiling instruction for SYCL backends (see appendix A.2)

I was compiling in the way described in the paper and in the README but it stil gives error:

/opt/intel/oneapi/compiler/2025.0/bin/compiler/clang-offload-bundler: error: '/home/sbaldu/Documents/Codes/genvectorx/build/CMakeFiles/SYCLMath.dir/src/MathUtil.cxx.o': No such file or directory
icpx: error: clang-offload-bundler command failed with exit code 1 (use -v to see invocation)
gmake[2]: *** [CMakeFiles/SYCLMath.dir/build.make:74: CMakeFiles/SYCLMath.dir/src/MathUtil.cxx.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:117: CMakeFiles/SYCLMath.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

On the other hand, I've tested the AdaptiveCPP backend and the benchmarks work correctly.

I've also tested successfully with AdaptiveCpp. I think some changes to the cmake file are necessary to make it work with OneAPI, I am trying to figure it out.

I will write a small CI workflow that automatically compiles and tests the code, in order to detect this kind of problems.

@sbaldu sbaldu force-pushed the feature-benchmarks branch from 79fc6c8 to 0347b22 Compare April 1, 2025 15:09
@sbaldu
Copy link
Contributor Author

sbaldu commented Apr 1, 2025

Hello. As a follow-up regarding this PR, I'm currently trying to figure out what is causing the problem with the oneapi backend, but since it's not caused on the changes made here, maybe this can be merged and then we can fix the backend in a separate PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants