Thoughts on the worker API

[lllama]

This is an expansion of some thoughts from Discord.

I’ve been writing an app that runs a couple of subprocesses and I was launching these as tasks using the worker API. The user would click a button and I would launch the tasks in the handler.

def on_button_pressed(self, event):
    self.run_worker(self.task1())
    self.run_worker(self.task2())

The above code looks like it would do what we want but in fact only task 2 ends up running. The reason is that exclusive defaults to True and tasks default to the default group, so the second task cancels the first. (I believe this is being corrected and exclusive now defaults to False. )

However, in my case, I would like the tasks to be exclusive in that I only want one instance of each subprocess to be running. If I set exclusive to True then I’m back to the original problem in that the second task cancels the first. I therefore need to change my code to the following:

def on_button_pressed(self, event):
    self.run_worker(self.task1(), exclusive=True, group=“task1”)
    self.run_worker(self.task2(), exclusive=True, group=“task2”)

My argument is that the need to specify groups for each task is not intuitive. I feel that the natural assumption would be that instances of a particular task are distinct from those of other tasks.

My other argument is that exclusive is also counter-intuitive. In my head, the word implies that the first instance would run until completion and further invocations would not be run. As it stands, subsequent invocations cancel the running instance. If the task did something like download a large file, one would probably want the first invocation to keep running rather than be restarted. At the moment there’s no easy way use the worker API to achieve this - I believe you’d have to set flags in your class or disable buttons.

To help with this I propose a tweak to the API. Rather than the exclusive parameter, have run_worker and the work decorator take a behaviour or modifier parameter that is one of the following:

• restartable
• dropping
• enqueued
• keep_latest

The restartable modifier would act in a similar way to exclusive: New invocations would cancel existing ones.

The dropping modifier would not invoke any new tasks if one was running.

The two bonus modifiers would have the following behaviours:

enqueued would keep a queue of invocations and only start the next when the running one has finished.

keep_latest would drop all future invocations save for the last. E.g. if three invocations are created, the first will run, the second will be dropped and the third will be enqueued to run when the first completes.

I’ve pinched these names and definitions from Ember Concurrency - it has a nice API that covers most use cases.

Having looked at the Textual code, I don’t think implementing this would be too tricky. If there’s any interest then I can start on a PR to see what it would look like.