Skip to content

Stateless model APIs #225

@PaulCCCCCCH

Description

@PaulCCCCCCH

I recently went through the workflow again and thought it might be better to implement our APIs in stateless manner, i.e., instead of tracking a current_model at the backend, we can instead pass in a model_id parameters in all model APIs - specifying which model to perform this operation on.

Pros:

  • Managing current_model state is tricky and error prone. Difficult to keep frontend in sync with the backend
  • current_model design has poor extensibiliy, and tightly couples with our current architecture.
  • We can only track one model at a time

Cons:

  • Given our current architecture which assumes user deploy the app in their local machine, it's difficult to run these APIs with different model (e.g., what if the user is making interleaving and frequent calls model-1 and model-2, does the back end pre-load them, or load just in time on demand?)

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions