Skip to content

Fixed locate particles #342

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 25 commits into from
Apr 14, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
9638100
Checked out new ParticleSpatialLayout into newest master
rsammann Mar 10, 2025
469a8fb
added initialization timer
rsammann Mar 10, 2025
6ae9f7c
added loadbalance threshold check, to short-circuit any loadbalancing…
rsammann Mar 10, 2025
bf241fa
remove int variables "sends" and "recvs" since these are not needed a…
rsammann Mar 10, 2025
471cf56
removed initialization int variables "sends" and "recvs"
rsammann Mar 10, 2025
fd72474
added Nvidia Profiling Section
rsammann Mar 11, 2025
410183a
removed nsight systems macro again
rsammann Mar 11, 2025
7259882
added CMake Nvidia Nsight Systems option
rsammann Mar 12, 2025
422d9d4
Changed profiler macro definition name
rsammann Mar 12, 2025
07d53b9
changed to newer CMake function, add_definitions is deprecated
rsammann Mar 12, 2025
cf12aec
Debugging why Profiler isn't working despite setting the macro in the…
rsammann Mar 12, 2025
be86b22
The macro doesnt seem to be the problem. Trying with old nvtxRangePus…
rsammann Mar 12, 2025
f3c8fad
typo
rsammann Mar 12, 2025
b1dcb52
normal range push with hardcoded define works, now trying with PUSH_R…
rsammann Mar 12, 2025
32a002f
Worked with hardcoded #define, now trying with CMakeLists.txt add_com…
rsammann Mar 12, 2025
ba589fb
Removed leading -D from add_compile_definitions(...) in CMakeLists.txt
rsammann Mar 12, 2025
410d671
more debugging of the profiler compiler option
rsammann Mar 12, 2025
6f0cb89
naming convention
rsammann Mar 13, 2025
b3c98c3
CMake option debugging
rsammann Mar 17, 2025
6358ece
Merge branch 'IPPL-framework:master' into fixed-locateParticles
rammann Mar 17, 2025
8b388f9
removed profiler section, working on this on a different branch
rsammann Mar 17, 2025
0e55022
rounded buffer allocation to next 2MiB
rsammann Mar 25, 2025
646904b
Merge branch 'IPPL-framework:master' into fixed-locateParticles
rammann Mar 31, 2025
b1752f9
removed buffer rounder for now
Apr 1, 2025
57e2d04
corrected typos for pull request
Apr 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions alpine/LandauDamping.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,9 @@ int main(int argc, char* argv[]) {
Inform msg2all(TestName, INFORM_ALL_NODES);

static IpplTimings::TimerRef mainTimer = IpplTimings::getTimer("total");
static IpplTimings::TimerRef initializeTimer = IpplTimings::getTimer("initialize");
IpplTimings::startTimer(mainTimer);
IpplTimings::startTimer(initializeTimer);

// Read input parameters, assign them to the corresponding memebers of manager
int arg = 1;
Expand Down Expand Up @@ -79,6 +81,8 @@ int main(int argc, char* argv[]) {
// Perform pre-run operations, including creating mesh, particles,...
manager.pre_run();

IpplTimings::stopTimer(initializeTimer);

manager.setTime(0.0);

msg << "Starting iterations ..." << endl;
Expand Down
2 changes: 1 addition & 1 deletion alpine/LoadBalancer.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ class LoadBalancer {
}

bool balance(size_type totalP, const unsigned int nstep) {
if (ippl::Comm->size() < 2) {
if (ippl::Comm->size() < 2 && loadbalancethreshold_m != 1.0) {
return false;
}
if (std::strcmp(TestName, "UniformPlasmaTest") == 0) {
Expand Down
27 changes: 23 additions & 4 deletions src/Particle/ParticleSpatialLayout.h
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

//
// Class ParticleSpatialLayout
// Particle layout based on spatial decomposition.
Expand Down Expand Up @@ -29,6 +30,9 @@
#include "Particle/ParticleLayout.h"
#include "Region/RegionLayout.h"

#include "Communicate/Window.h"
#include <vector>

namespace ippl {

/*!
Expand Down Expand Up @@ -76,8 +80,14 @@ namespace ippl {
RegionLayout_t rlayout_m;

//! The FieldLayout containing information on nearest neighbors
FieldLayout_t flayout_m;

FieldLayout_t& flayout_m;

// MPI RMA window for one-sided communication
mpi::rma::Window<mpi::rma::Active> window_m;

// Vector keeping track of the recieves from all ranks
std::vector<size_type> nRecvs_m;

//! Type of the Kokkos view containing the local regions.
using region_view_type = typename RegionLayout_t::view_type;
//! Type of a single Region object.
Expand All @@ -89,6 +99,14 @@ namespace ippl {
KOKKOS_INLINE_FUNCTION constexpr static bool positionInRegion(
const std::index_sequence<Idx...>&, const vector_type& pos, const region_type& region);

/*!
* Evaluates the total number of MPI ranks sharing the spatial nearest neighbors.
* @param neighbors structure containing, for every spatial direction, a list of
* MPI ranks IDs corresponding to the nearest neighbors of the current local domain section.
* @return The total number of the ranks.
*/
size_type getNeighborSize(const neighbor_list& neighbors) const;

public:
/*!
* For each particle in the bunch, determine the rank on which it should
Expand All @@ -101,8 +119,8 @@ namespace ippl {
* @return The total number of invalidated particles
*/
template <typename ParticleContainer>
size_type locateParticles(const ParticleContainer& pc, locate_type& ranks,
bool_type& invalid) const;
std::pair<size_type,size_type> locateParticles(const ParticleContainer& pc, locate_type& ranks,
bool_type& invalid, locate_type& nSends_dview, locate_type& sends_dview) const;

/*!
* @param rank we sent to
Expand All @@ -122,3 +140,4 @@ namespace ippl {
#include "Particle/ParticleSpatialLayout.hpp"

#endif

Loading