Service profiles for Docker Compose

[malthe]

Description

Docker Compose is arguably the most straight-forward way to stand up an Airflow system on a local machine.

It should be possible to tailor the deployment based on profiles.

We already have debug and flower, but it seems ideal to attach a profile to postgresql as well. Many users would want to use sqlite for local development and don't need redis or airflow-worker.

Use case/motivation

The main motivation is to reduce the deployment to the bare minimum to get started with local development of DAGs.

Basically, running just the scheduler, triggerer and webserver, against a sqlite database.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[potiuk]

I dont think we should provide a configurable Docker Compose beyond quick start.

Who is going to supprt problems that people are going to have wihen they start modifying Docker Compose and get errors there? You can look up how many of the users already assumeed this is "official" docker compose and raised issues that this and that does not work there. If we make it configurable and "deployable", we automaticaly assume the responsibility on supporting all kinds of configurations there.

I am strongly against going this route.

[potiuk]

It's much, much better to make airflow standalone "small-production-ready". That would be far easier to complete and super easy for "casual users" to start airflow without learning docker compose and images.