handle process initialization failure #1556
Conversation
🦋 Changeset detectedLatest commit: 3a8217d The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
| @@ -19,7 +19,7 @@ | |||
| "process_job_launched", | |||
| ] | |||
|
|
|||
| MAX_CONCURRENT_INITIALIZATIONS = 1 | |||
| MAX_CONCURRENT_INITIALIZATIONS = 5 | |||
There was a problem hiding this comment.
I remember this was added to limit the amount of strain it causes on the instance. is this no longer an issue?
perhaps this should scale according to the # of CPU cores (num_cores / 2)
There was a problem hiding this comment.
The caused effect was spiky CPU pattern when new processes load. I don't see how we can avoid that, I think it is OK as long as it doesn't overload.
We can scale with the number of CPU cores, tho prewarm functions aren't necessarily CPU bound. So it is OK to have more, perhaps it could be a new option to the worker.
| try: | ||
| client.initialize() | ||
| except Exception: | ||
| return # initialization failed, exit |
There was a problem hiding this comment.
The log is on the worker side, after receiving the InitializeResponse
catch user initialization cb failures & allow concurrent proc initializations