Create a single tmp dir per test session #20
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previous behavior was to create a new tmp directory for each test case that required writing to the cloud bucket. That adds a lot of overhead, as removing the directory is a slow, asynchronous call. We have an exponential backoff and retry to delete, but there were around 600 temp directories that were not cleaned up due to failed attempts.
This change creates a single temp directory in the cloud bucket for the whole execution, and cleans it up at the end of the testing session. I increase the number of retries, to get one more chance to successfully delete before giving up.
Because we only have ONE exponential backoff/retry cycle for the whole test session, we trim about 15-30 seconds off the total execution time
In addition, since this changed so many test files, I figured it was an ok time to rename the overly-long
example_cloud_storage_options
fixture. Also cleans up the session-level fixtures.