-
Notifications
You must be signed in to change notification settings - Fork 82
Open
Description
Hello, a user of our HPC system has encountered over-threading issues, and as this software is their script's only external dependency, and makes no other use of multiprocessing, it is highly likely to be the cause. multiprocessing.cpu_count(), as I have frequently noticed, does not return the correct value on HPC schedulers. If the desired number of cores is not to be passed in explicitly, it could be read from OMP_NUM_THREADS, which is reasonably idiomatic, as numpy.linalg does the same.
In AbstractSustain.py:
self.pool = pathos.multiprocessing.ProcessingPool() #pathos.multiprocessing.ParallelPool()
self.pool.ncpus = multiprocessing.cpu_count()might become:
self.pool = pathos.multiprocessing.ProcessingPool(
int(os.environ.get('OMP_NUM_THREADS', multiprocessing.cpu_count()))
)Metadata
Metadata
Assignees
Labels
No labels