Serve an MLflow Model with BentoML

This is a BentoML example project, which demonstrates how to serve and deploy an MLflow model with BentoML.

See here for a full list of BentoML example projects.

Install dependencies

git clone https://github.com/bentoml/BentoMLflow.git
cd BentoMLflow

# Recommend Python 3.11
pip install -r requirements.txt

Train and save a model

Save the model to the BentoML Model Store:

python3 save_model.py

Run the BentoML Service

We have defined a BentoML Service in service.py. Run bentoml serve in your project directory to start the Service.

$ bentoml serve .

2024-06-19T10:25:31+0000 [INFO] [cli] Starting production HTTP BentoServer from "service:IrisClassifier" listening on http://localhost:3000 (Press CTRL+C to quit)

The server is now active at http://localhost:3000. You can interact with it using the Swagger UI or in other different ways.

CURL

curl -X 'POST' \
    'http://localhost:3000/predict' \
    -H 'accept: application/json' \
    -H 'Content-Type: application/json' \
    -d '{
    "input_data": [
        [5.9, 3, 5.1, 1.8]
    ]
}'

Python client

import bentoml

with bentoml.SyncHTTPClient("http://localhost:3000") as client:
    result = client.predict(
        input_data=[
            [5.9, 3, 5.1, 1.8]
        ],
    )
    print(result)

For detailed explanations, see the BentoML documentation.

Deploy to BentoCloud

After the Service is ready, you can deploy the application to BentoCloud for better management and scalability. Sign up if you haven't got a BentoCloud account.

Make sure you have logged in to BentoCloud.

bentoml cloud login

Deploy it from the project directory.

bentoml deploy .

Once the application is up and running, you can access it via the exposed URL.

Note: For custom deployment in your own infrastructure, use BentoML to generate an OCI-compliant image.