Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lazy import of kedro fastapi modules and configs #23

Merged
merged 2 commits into from
Feb 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,19 @@

## [Unreleased]

### Added

- :sparkles: Include ["fastapi*/] dynamically in the config loader config patterns

### Fixed

- :bug: Catch fastapi extra dependecies errors, and make their imports lazy.

## [0.2.0] - 2024-02-12

### Added

- :sparkles: Introduce Kedro FastAPI Server: A framework for building production-ready REST APIs using Kedro, FastAPI and Gunicorn.
- :sparkles: Introduce Kedro FastAPI Server: A framework for building production-ready REST APIs using Kedro, FastAPI and Gunicorn ([#18](https://github.com/takikadiri/kedro-boot/pull/18))
- :sparkles: Support dataset factories ([#5](https://github.com/takikadiri/kedro-boot/pull/5))
- :sparkles: Allow declaring apps through project settings and CLIs ([#16](https://github.com/takikadiri/kedro-boot/pull/16))
- :sparkles: Adding `boot_project` and `boot_package` allowing standalone apps to boot a session ([#17](https://github.com/takikadiri/kedro-boot/pull/17))
Expand Down
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,17 +32,17 @@ pip install kedro-boot[fastapi]
Then you can serve your fastapi app with :

```
kedro boot fastapi --app path.to.your.fastapi.app <kedro_args>
kedro boot fastapi --app path.to.your.fastapi.app <kedro_run_args>
```

Your fastapi app objects will be mapped with kedro pipeline objects, and the run results will be injected into your KedroFastAPI object through [FastAPI dependency injection](https://fastapi.tiangolo.com/tutorial/dependencies/). Here is an illustration of the kedro <-> fastapi objects mapping:

![Kedro FastAPI objects mapping](.github/kedro_fastapi_mapping.PNG)

A default FastAPI app is used if no FastAPI app given. It would serve a single endpoint that run your selected pipeline
A default FastAPI app is used if no FastAPI app given. It would serve a single endpoint that run in background your selected pipeline

```
kedro boot fastapi
kedro boot fastapi <kedro_run_args>
```

These production-ready features would be natively included in your FastAPI apps:
Expand All @@ -51,7 +51,7 @@ These production-ready features would be natively included in your FastAPI apps:
- [Pyctuator](https://github.com/SolarEdgeTech/pyctuator) that report some service health metrology and application states. Usually used by service orchestrators (kubernetes) or monitoring to track service health and ensure it's high availability
- Multiple environments configurations, leveraging kedro's OmegaConfigLoader. ``["fastapi*/"]`` config pattern could be used to configure the web server. Configs could also be passed as CLI args (refer to --help)

You can refer to the spaceflights [Kedro FastAPI examples](examples/src/spaceflights_kedro_fastapi/app.py) that showcases serving multiples endpoints operations that are mapped to differents pipeline namespaces
You can learn more by testing the [spaceflights Kedro FastAPI example](examples/README.md#rest-api-with-kedro-fastapi-server) that showcases serving multiples endpoints operations that are mapped to differents pipeline namespaces


## Consuming Kedro pipelines through SDK
Expand Down Expand Up @@ -94,19 +94,19 @@ return {"__default__": spaceflights_pipelines}

In this example, all the namespaces and their namespaced datasets (inputs, outputs, parameters) would infer compilation specs and therefore would be exposed to the Application.

You can use kedro-viz to visualize the datasets that woulc be exposed to the kedro boot apps. In the figure below, we see clearly that ``inference.feature_store`` and ``inference.predictions`` will be exposed to the applicaton (the blue one).
You can use kedro-viz to visualize the datasets that woulc be exposed to the kedro boot apps. In the figure below, for the ``inference`` namespace, we see clearly that ``inference.feature_store`` and ``inference.predictions`` will be exposed to the applicaton (the blue one).

![pipeline_namespace](.github/pipeline_namespace.png)

Below are the different categories of datasets that forms the compiled catalog.
Below are the differents categories of datasets that forms the compiled catalog.

- Inputs: inputs datasets that are be injected by the app at iteration time.
- Outputs: outputs dataset that hold the run results.
- Parameters: parameters that are injected by the app at iteration time.
- Artifacts: artifacts datasets that are materialized (loaded as MemoryDataset) at startup time.
- Templates: template datasets that contains ${itertime_params: param_name}. Their attributes are interpolated at iteration time.

You can compile the catalog without actually using it in a Kedro Boot App. This is helpful for verifying if the expected artifacts datasets are correctly infered or if the template datasets are correctly detected.
You can compile the catalog without actually using it in a Kedro Boot App. This is helpful for verifying if the expected artifacts datasets are correctly infered or if the template datasets are correctly detected. Here is an example of the catalog compilation report for a pipeline that contains an ``inference`` namespace.

```
kedro boot compile
Expand Down Expand Up @@ -164,7 +164,7 @@ session = boot_project(
run_results = session.run(inputs={"your_dataset_name": your_data})
```

You can found a complete example of a steamlit app that serve an ML model in the [Kedro Boot Examples](examples) project. We invite you to test it to gain a better understanding of Kedro Boot's ``boot_project`` or ``boot_package`` interfaces
You can found a complete example of a steamlit app that serve an ML model in the [Kedro Boot Examples](examples/README.md#data-app-with-streamlit-standalone-mode) project. We invite you to test it to gain a better understanding of Kedro Boot's ``boot_project`` or ``boot_package`` interfaces

### Embedded mode : The application is embeded inside kedro project

Expand Down Expand Up @@ -212,7 +212,7 @@ kedro boot run <kedro_run_args>
kedro boot run --app path.to.your.KedroBootApp <kedro_run_args>
````

You can fnd two examples of using the embeded mode in the [Kedro Boot Examples](examples) project
You can find an example of a [Monte Carlo App embeded into a kedro project](examples/README.md#monte-carlo-simulation-embeded-mode)


## Why does Kedro Boot exist ?
Expand Down
2 changes: 2 additions & 0 deletions examples/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ data/**

# keep also the example dataset
!data/01_raw/*
!data/04_feature/*
!data/06_models/*


##########################
Expand Down
2 changes: 1 addition & 1 deletion examples/conf/base/fastapi.yml
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
server:
port: 8002
port: 8000
Loading
Loading