best strategy to update job args (or other metadata) to send to subscription?

[vsoch]

Hi! Apologies for the many questions - I'm into a nice groove and am loving using River, and I've been searching code and discussions first before asking a question, to the best that I can.

I have most of what I want working, and the last step is that I want to be able to (after the worker is successful) have the subscription receive notification, but also receive a node assignment. For some context, this is replacing the Kubernetes scheduler, and the worker is getting the nodes to allocate, and we send back to the main schedule function as a list to send to the kubelet to ask to bind. The design thus far is here: converged-computing/fluxnetes#2 if you are interested.

I naively tried updating job.Args, knowing well that it likely wouldn't update the database (indeed it did not) and I'm wanting to ask what the best strategy would be for this. Ideally I don't want to make another event stream (or worker queue) just to deliver notice back - the events that are already delivered should be the single source of truth for the particular job. I was wondering if it might be possible to have something additional tied to the event to send back? I know there are statistics (and this is generally what we have to work with) and I'm wondering if something else could be exposed to allow the worker to add arbitrary metadata to send to subscribers.

Thank you!

[bgentry]

There’s some discussion on this in #164

My general advice is to treat metadata and especially job args as immutable, although there are exceptions to this if done safely. One thing I’d like to introduce is a general solution to the problem of recording a job’s “output”, whether kept in metadata or elsewhere. But nothing like that exists yet so you’d be on your own.

The biggest thing to make sure of is that you’re either updating a job that’s already running from within its own work function, or if the job isn’t running that you lock it and commit your changes quickly so as not to hold open a long running transaction that would make the job inaccessible.