Skip to content

[WIP][TBD] Update how we handle dependencies and run environment for our Python examples #5890

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ _deps

# Python virtual environment:
**/venv*
**/.venv*
.python-version

# Python build artifacts:
Expand Down
169 changes: 169 additions & 0 deletions examples/python/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,173 @@
# Rerun Python examples

---

## Example dependency management

We need the following for a smooth example experience:
- Per-example dependency specification: different example must be able to have conflicting dependencies.
- Per-example Python version requirement: although the Rerun SDK has a well-defined Python compatibility range, specific example should be able to restrict it when required by their specific dependencies.
- Per-example isolated environment: it follows from what precedes that each example should be run in their own venv.
- Minimal overhead and as "standard" as possible.
- (Nice to have) Lockfile.

In addition, we should support both dev environment (install `rerun-sdk` from the working directory) and end-user environment (install `rerun-sdk` from PyPI).

---

## Using `uv`

Example: `human_pose_tracking`

### Tooling Installation

```shell
pipx install uv

# or

curl -LsSf https://astral.sh/uv/install.sh | sh
```

### Environment Setup

Must be manually created but `uv` has facilities for that:

```shell
cd example/python/human_pose_tracking
uv venv --python 3.11
source .venv/bin/activate
uv pip install -r requirements.txt

# to use local rerun-sdk (this triggers a maturin compilation)
uv pip install -e ../../../rerun_py
```

⚠️There don't seem to be a way to specify Python version requirement for `uv` yet, so the user must pick a compatible python version.

Note: since this is basically based on a (locked) requirement file, "standard" Python tooling will also work:

```shell
cd example/python/human_pose_tracking
python3.11 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -e ../../../rerun_py
```

In that sense, `uv` is only required for example authors (to update the `requirements.txt` file).

### Run the example

```shell
python main.py
```

### Create/update the lockfile

The direct dependencies are specified in `requirements.in`. From that, `uv` is able to generate a `requirements.txt` which essentially is a lockfile:

```shell
uv pip compile requirements.in -o requirements.txt
```

---

## Using `poetry` with `package-mode = false`

Example: `human_pose_tracking_poetry`

Note: `package-mode = false` is very nice for lone scripts that don't need to be "pip installed" to run, just like our examples.

### Tooling Installation

```shell
pipx install poetry

# or

curl -sSL https://install.python-poetry.org | python3 -
```

### Environment Setup

```shell
poetry install

# to use the local rerun-sdk (this triggers maturin compilation)
poetry run pip install -e ../../../rerun_py
```

This will do the following:
- Unless a venv is already active, it will create one using some python interpreter that it can find and is compatible with constraints listed in `pyproject.toml`.
- Install all dependencies.
- Create the `main` executable script as specified in the `pyproject.toml`.

Additional facilities provided by Poetry:
```shell
poetry run ${ARGS} # run stuff inside the default environment
poetry env use 3.8 # create another environment with the specified python version
poetry env list # list all available environments
poetry shell # activate the default environment
```

### Run the example

```shell
poetry run python main.py

# or

poetry shell # activate the venv
python main.py
```

### Create/update the lockfile

```shell
poetry update
```

---

## Using `hatch`

Example: `human_pose_tracking_hatch`

`hatch` is similar to `poetry`, but with wider scope and a richer plug-in ecosystem to support it.

### Tooling Installation

```shell
pipx install hatch
```

### Environment Setup

Basically nothing, `hatch` creates a venv on the fly.

To use the dev rerun-sdk:
```shell
hatch run pip install ../../../rerun_py
```

### Run the example

```shell
hatch run python main.py

# or

hatch shell
python main.py
```




---
_original content follows_

The simplest example is [`minimal`](minimal/main.py). You may want to start there!

Read more about our examples at <https://www.rerun.io/examples>.
Expand Down
1 change: 1 addition & 0 deletions examples/python/human_pose_tracking/.gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
/.venv
dataset/
8 changes: 8 additions & 0 deletions examples/python/human_pose_tracking/requirements.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# no 3.12 version yet (https://pypi.org/project/mediapipe/)
# 0.10.10 no longer supports the legacy Pose model: https://github.com/rerun-io/rerun/issues/5859
mediapipe==0.10.9 ; python_version <= '3.11'

numpy
opencv-python>4.6 # Avoid opencv-4.6 since it rotates images incorrectly (https://github.com/opencv/opencv/issues/22088)
requests>=2.31,<3
rerun-sdk
75 changes: 67 additions & 8 deletions examples/python/human_pose_tracking/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,8 +1,67 @@
# no 3.12 version yet (https://pypi.org/project/mediapipe/)
# 0.10.10 no longer supports the legacy Pose model: https://github.com/rerun-io/rerun/issues/5859
mediapipe==0.10.9 ; python_version <= '3.11'

numpy
opencv-python>4.6 # Avoid opencv-4.6 since it rotates images incorrectly (https://github.com/opencv/opencv/issues/22088)
requests>=2.31,<3
rerun-sdk
# This file was autogenerated by uv via the following command:
# uv pip compile requirements.in -o requirements.txt
absl-py==2.1.0
# via mediapipe
attrs==23.2.0
# via
# mediapipe
# rerun-sdk
certifi==2024.2.2
# via requests
cffi==1.16.0
# via sounddevice
charset-normalizer==3.3.2
# via requests
contourpy==1.2.1
# via matplotlib
cycler==0.12.1
# via matplotlib
flatbuffers==24.3.25
# via mediapipe
fonttools==4.51.0
# via matplotlib
idna==3.6
# via requests
kiwisolver==1.4.5
# via matplotlib
matplotlib==3.8.4
# via mediapipe
mediapipe==0.10.9
numpy==1.26.4
# via
# contourpy
# matplotlib
# mediapipe
# opencv-contrib-python
# opencv-python
# pyarrow
# rerun-sdk
opencv-contrib-python==4.9.0.80
# via mediapipe
opencv-python==4.9.0.80
packaging==24.0
# via matplotlib
pillow==10.3.0
# via
# matplotlib
# rerun-sdk
protobuf==3.20.3
# via mediapipe
pyarrow==15.0.2
# via rerun-sdk
pycparser==2.22
# via cffi
pyparsing==3.1.2
# via matplotlib
python-dateutil==2.9.0.post0
# via matplotlib
requests==2.31.0
rerun-sdk==0.15.0
six==1.16.0
# via python-dateutil
sounddevice==0.4.6
# via mediapipe
typing-extensions==4.11.0
# via rerun-sdk
urllib3==2.2.1
# via requests
1 change: 1 addition & 0 deletions examples/python/human_pose_tracking_hatch/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
dataset/
Loading
Loading