AIP-7511 user defined @exit_handler Part 2/2

[talebzeghmi]

https://argo-server.int.stage-k8s.zg-aip.net/argo-ui/workflows/aip-playground-dev/helloflow-vwdpv?tab=workflow

example use:

from metaflow import FlowSpec, step, exit_handler
from metaflow.plugins.aip import exit_handler_resources, exit_handler_retry

# advanced likely a 10% case of customizing the retry or resources
@exit_handler_retry(times=1, minutes_between_retries=0)
@exit_handler_resources(cpu="600m", memory="600Mi")
def my_exit_handler(
    status: str,
    flow_parameters: dict,
    argo_workflow_run_name: str,
    metaflow_run_id: str,
    argo_ui_url: str,
    retries: int,
) -> None:
    if retries == 0:
        raise Exception("oopsie")


@exit_handler(func=my_exit_handler, on_success=True)
class HelloFlow(FlowSpec):
    @step
    def start(self):
        raise Exception("foo")
        self.next(self.end)

    @step
    def end(self):
        print("HelloFlow is all done.")

if __name__ == "__main__":
    HelloFlow()

python metaflow/tutorials/00-helloworld/helloworld.py aip run --notify --sqs-url-on-error foo

note: Not able to reuse the Metaflow @resources and @retry decorator

Because:

• it can only be defined on a step https://github.com/zillow/metaflow/blob/feature/aip/metaflow/decorators.py#L405-L406
  • This is a good feature, because it reduces confusion.
• we won't support every feature such as volume, gpu, etc.