You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In many executors, it's going to be more performant to clean up job records after the job has been completed and the output has been collected. That being said, opting out of this behavior is important when debugging jobs, for instance. It would be great to provide a flag that allows job cleanup (at the discretion of the executor) by default but then be able to specify when you want a job to stick around (at the cost of performance).
The text was updated successfully, but these errors were encountered:
Can you clarify what you mean @claymcleod ? TES is fundamentally dealing with individual tasks and inputs and outputs. It’s my opinion that users, or engines would need to deal with cleanup the outputs. As for performance can you clarify. The job is done, so the only performance is storage.
I would also appreciate a clarification, @claymcleod. Apart from what @vsmalladi asked, I am also wondering what exactly you mean by "job records" and by "executors" - do you mean TES tasks or actual _executor_s as per the spec (of which there can be more than one per TES task).
In many executors, it's going to be more performant to clean up job records after the job has been completed and the output has been collected. That being said, opting out of this behavior is important when debugging jobs, for instance. It would be great to provide a flag that allows job cleanup (at the discretion of the executor) by default but then be able to specify when you want a job to stick around (at the cost of performance).
The text was updated successfully, but these errors were encountered: