Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance Curate disk space cleanups (Oct 2023) #2213

Open
eporter23 opened this issue Oct 4, 2023 · 5 comments
Open

Enhance Curate disk space cleanups (Oct 2023) #2213

eporter23 opened this issue Oct 4, 2023 · 5 comments
Assignees
Labels
Library Staff Ingest and Migration Utilities Middleware Flag issues for Middleware/DevOps teams Software Engineering Flag work for software engineering team

Comments

@eporter23
Copy link
Contributor

This ticket follows work done previously in #2091. While we have done work to try to clear out temp files related to Bulkrax imports, we still periodically run out of disk space on Curate which may occur because each time we deploy to production, a new /tmp/imports folder is tied to a specific release.

@eporter23 eporter23 added Library Staff Ingest and Migration Utilities Middleware Flag issues for Middleware/DevOps teams Software Engineering Flag work for software engineering team labels Oct 4, 2023
@bwatson78
Copy link
Contributor

PR made: #2214

@bwatson78
Copy link
Contributor

@eporter23 To test this, we need to confirm that any import created after 10/4/2023 320pm EST on either Test or Arch gets automagically deleted with @jcrompton42 AWX deletion script (probably with a much shorter "age-before-deletion" set).

@eporter23
Copy link
Contributor Author

@bwatson78 I'm going to re-run that importer from last week because I made some errors in the CSV, and the import failed.

@eporter23
Copy link
Contributor Author

Here's the path to the import I ran today: tmp/imports/74_20231010133738/Archive 3.zip

@eporter23
Copy link
Contributor Author

@jcrompton42 @bwatson78 I ran this import in Curate-test yesterday: tmp/imports/87_20231011202814/Archive.zip . Not sure if the AWX cleanup job runs on arch so I did this one in test. I don't have ssh access so I can't confirm, but I can monitor the dlp-automation channel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Library Staff Ingest and Migration Utilities Middleware Flag issues for Middleware/DevOps teams Software Engineering Flag work for software engineering team
Projects
None yet
Development

No branches or pull requests

3 participants