Avoid duplicates
Bug Summary
I recently noticed in a parallel run that different cores started to process different time windows which eventually caused the whole computation to crash.
I found, that his was caused by corrupt data in the database (false sampling rate in the miniseeds, missing responses in the inventory). It triggered exceptions during the pre-processing routines which should be handled by the _generate_data()- generator method of the Correlator. However, the exception handlers involve a continue-statement which cause the generator to proceed to the next time window. While this is not a problem in a single-core run, it is on multiple cores. The core that tried to process the corrupt data proceeds to the next time window while the other cores have to finish their processing.
This can be generally prevented by replacing the continue-statements with definitions of empty streams. A preliminary fix is implemented in the dev-jojomale branch.
Code to Reproduce
Error Traceback
SeisMIC Version?
dev, dev-jojomale
Operating System?
Ubuntu
Python Version?
No response
Installation Method?
developer installation / from source / git checkout
Avoid duplicates
Bug Summary
I recently noticed in a parallel run that different cores started to process different time windows which eventually caused the whole computation to crash.
I found, that his was caused by corrupt data in the database (false sampling rate in the miniseeds, missing responses in the inventory). It triggered exceptions during the pre-processing routines which should be handled by the _generate_data()- generator method of the Correlator. However, the exception handlers involve a continue-statement which cause the generator to proceed to the next time window. While this is not a problem in a single-core run, it is on multiple cores. The core that tried to process the corrupt data proceeds to the next time window while the other cores have to finish their processing.
This can be generally prevented by replacing the continue-statements with definitions of empty streams. A preliminary fix is implemented in the dev-jojomale branch.
Code to Reproduce
Error Traceback
SeisMIC Version?
dev, dev-jojomale
Operating System?
Ubuntu
Python Version?
No response
Installation Method?
developer installation / from source / git checkout