HIP/GCC host-only warning

[wiersema1]

Summary

For HIP/GCC builds, the new/delete operators in Src/Base/AMReX_BaseFab.H gave me host-only warnings. This PR adds device-side operator overloads to resolve the issue.

I'd welcome a careful review of these changes to catch any unintended side effects since I'm not comfortably familiar with AMReX internals.

Additional background

The following is from compiling the ERF model.

[ 34%] Building CXX object Submodules/AMReX/Src/CMakeFiles/amrex_3d.dir/EB/AMReX_EBAmrUtil.cpp.o
[ 34%] Building CXX object Submodules/AMReX/Src/CMakeFiles/amrex_3d.dir/EB/AMReX_EB_utils.cpp.o
[ 34%] Building CXX object Submodules/AMReX/Src/CMakeFiles/amrex_3d.dir/EB/AMReX_EB_FluxRedistribute.cpp.o
In file included from /home/wiersema/Projects/ERF/Submodules/AMReX/Src/EB/AMReX_EBCellFlag.cpp:1:
In file included from /home/wiersema/Projects/ERF/Submodules/AMReX/Src/EB/AMReX_EBCellFlag.H:7:
/home/wiersema/Projects/ERF/Submodules/AMReX/Src/Base/AMReX_BaseFab.H:118:9: error: no matching 'operator new' function for non-allocating placement new expression; include
118 | new (ptr+i) T;
| ^~~
/home/wiersema/Projects/ERF/Submodules/AMReX/Src/Base/AMReX_GpuLaunchMacrosG.nolint.H:475:88: note: expanded from macro 'AMREX_HOST_DEVICE_FOR_1D'
475 | amrex::ParallelFor(amrex_i_n,[=] AMREX_GPU_DEVICE (amrex_i_inttype i) noexcept block);
| ^~~~~
/home/wiersema/Projects/ERF/Submodules/AMReX/Src/Base/AMReX_BaseFab.H:117:5: note: while substituting into a lambda expression here
117 | {
| ^
/home/wiersema/Projects/ERF/Submodules/AMReX/Src/Base/AMReX_BaseFab.H:1473:5: note: in instantiation of function template specialization 'amrex::placementNewamrex::EBCellFlag' requested here
1473 | placementNew(this->dptr, this->truesize);
| ^
/home/wiersema/Projects/ERF/Submodules/AMReX/Src/Base/AMReX_BaseFab.H:1501:5: note: in instantiation of member function 'amrex::BaseFabamrex::EBCellFlag::define' requested here
1501 | define();
| ^
/home/wiersema/Projects/ERF/Submodules/AMReX/Src/EB/AMReX_EBCellFlag.cpp:12:7: note: in instantiation of member function 'amrex::BaseFabamrex::EBCellFlag::BaseFab' requested here
12 | : BaseFab(b,n,ar)
| ^
1 error generated when compiling for gfx1201.
gmake[2]: *** [Submodules/AMReX/Src/CMakeFiles/amrex_3d.dir/build.make:1490: Submodules/AMReX/Src/CMakeFiles/amrex_3d.dir/EB/AMReX_EBCellFlag.cpp.o] Error 1
gmake[2]: *** Waiting for unfinished jobs....
gmake[1]: *** [CMakeFiles/Makefile2:439: Submodules/AMReX/Src/CMakeFiles/amrex_3d.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

My build environment includes

• ROCm 6.2.4 / HIP 6.2.41134 / AMD Clang 18.0.0
• GCC 13.3.0 (Ubuntu 24.04)
• CMake 3.28.3

Checklist

The proposed changes:

• fix a bug or incorrect behavior in AMReX
• add new capabilities to AMReX
• changes answers in the test suite to more than roundoff level
• are likely to significantly affect the results of downstream AMReX users
• include documentation in the code and/or rst files, if appropriate

[WeiqunZhang]

Could you tell me how to reproduce the issue with ERF? ERF does have a HIP CI that does not have the issue.

[WeiqunZhang]

It might also help if you can compile with make VERBOSE=ON so that we can see what compiler flags are used.

[WeiqunZhang]

I could not reproduce it on frontier.

[wiersema1]

@WeiqunZhang I found the issue and it was entirely on my end. In my build script I had the GCC C++ headers provided via CPLUS_INCLUDE_PATH rather than in the cmake CXX_FLAGS, which meant GCC's was prioritized over the ROCm . Thank you for your time and sorry to raise a non-issue!