-
Notifications
You must be signed in to change notification settings - Fork 0
Description
This issue describes how node migration should ultimately be implemented.
The current NodeRegistry simply maps names (strings) to Node subclasses. We can do way better by registering each class with a minimum and maximum version defined as ClassVars. That way our registry could look like
FooNodeV1: >=1.0.0 <2.0.0
FooNodeV2: >=2.0.0 <3.0.0
BarNode: None None
So when we load a node with data {"type": "Foo", "version": "1.2.3"}, it'll get properly dispatched to the old FooNodeV1 class. If the engine operator wants, they can continue supporting the old version and the new version simultaneously.
Once the engine operator is sick of having FooNodeV1 nodes in their workflows, they can deprecate them by simply modifying the FooNodeV1 class. The operator still keeps both classes around, but they replace FooNodeV1.run method with a NotImplementedError. Then, they add a .to_next_version(self, edges: Sequence[Edge]) -> tuple[Self, Sequence[Edge]] method on the class. Some magic part of our pipeline will check for this method, and if it exists it will automatically bump the FooNodeV1 instance to FooNodeV2 during deserialization while also following through with those edge replacements.
Finally, once all the FooNodeV1 instances are gone (from the database, etc.), the engine operator can finally delete that class.