feat: Migrate dependency management to Poetry (#247)

**Signed-off-by:** Areeb Ahmed
[[email protected]](mailto:[email protected])

## Summary of Changes
**Fixes:** #154

- Replaced `requirements.txt` with `pyproject.toml` and `poetry.lock`
- Added all dependencies to `pyproject.toml` using Poetry
- Updated CI configuration to use Poetry for installing dependencies
- Updated `README.md` and `INSTALL.md` to reflect the new dependency
management process

Author: areebahmeddd

.github/workflows/pull-request.yml

@@ -1,4 +1,4 @@
-name: Pull Request checks
+name: Pull Request Checks
 
 on:
   pull_request:
@@ -9,46 +9,63 @@ concurrency:
 
 jobs:
   run-tests:
-    name: Run pytest tests
+    name: Run pytest Tests
     runs-on: ubuntu-latest
     env:
       POSTGRES_USER: folksonomy
       POSTGRES_PASSWORD: folksonomy
-      # important: "localhost" does not work on github ci, only 127.0.0.1
+      # Important: "localhost" does not work on GitHub CI, only 127.0.0.1
       POSTGRES_HOST: "127.0.0.1"
       POSTGRES_DATABASE: folksonomy
+
     steps:
-      - name: checkout project
+      - name: Checkout Project
         uses: actions/checkout@v4
         with:
           fetch-depth: 1
-      - name: create pgdata directory
+
+      - name: Create pgdata Directory
         run: mkdir -p /home/runner/work/folksonomy_api/folksonomy_api/data/pg/pgdata
-      - name: change permissions for pgdata directory
+
+      - name: Change Permissions for pgdata Directory
         run: sudo chmod -R 777 /home/runner/work/folksonomy_api/folksonomy_api/data/pg/pgdata
-      - name: install python
+
+      - name: Install Python
         uses: actions/[email protected]
         with:
-          python-version: '3.8'
-          cache: 'pip'
-      - name: start a postgresql docker container
+          python-version: '3.9'
+
+      - name: Install Poetry
+        uses: snok/install-poetry@v1
+        with:
+          version: latest
+          virtualenvs-create: true
+          virtualenvs-in-project: false
+
+      - name: Start PostgreSQL Docker Container
         run: ./start_postgres.sh &
-      - name: install packages in a virtual environment
-        run: |
-          python -m venv .venv && \
-          .venv/bin/pip install -r requirements.txt
-      - name: wait postgres to be ready
+
+      - name: Install Dependencies
+        run: poetry install --no-interaction
+
+      - name: Wait for PostgreSQL to be Ready
         timeout-minutes: 10
         run: |
-          while ! (nc -vz 127.0.0.1 5432);do echo "waiting postgres"; sleep 1;done
-      - name: initialize database
-        run: |
-          .venv/bin/python db-migration.py
-      - name: run tests
-        # use PYTHONASYNCIODEBUG to ensure all asyncio tasks are closed
+          while ! (nc -vz 127.0.0.1 5432); do
+            echo "Waiting for PostgreSQL"
+            sleep 1
+          done
+
+      - name: Initialize Database
+        run: poetry run python db-migration.py
+
+      - name: Run Tests
+        # Use PYTHONASYNCIODEBUG to ensure all asyncio tasks are closed
         run: |
-          PYTHONASYNCIODEBUG=1 .venv/bin/pytest -v --cov=folksonomy  --cov-report xml tests/ folksonomy/
-      - uses: codecov/[email protected]
+          PYTHONASYNCIODEBUG=1 poetry run pytest -v --cov=folksonomy --cov-report xml tests/ folksonomy/
+
+      - name: Upload Coverage Reports
+        uses: codecov/[email protected]
         with:
           token: ${{ secrets.CODECOV_TOKEN }}
           fail_ci_if_error: true

INSTALL.md

@@ -4,13 +4,16 @@
 
 - PostgreSQL 13 or newer
 - Python 3.9 or newer
-- Python modules in "requirements.txt"
+- Poetry (for dependency management)
 
 ## Setup on Debian 12 (Recommended Method)
 
 ```bash
 # Start with a fresh Debian 12 install, logged as root
-apt install git sudo postgresql python3-venv -y
+apt install git sudo postgresql python3-pip curl -y
+
+# Install Poetry
+curl -sSL https://install.python-poetry.org | python3 -
 
 # Create a user for the application (optional but recommended)
 adduser folksonomy
@@ -22,50 +25,41 @@ cd ~
 git clone https://github.com/openfoodfacts/folksonomy_api.git
 cd folksonomy_api
 
-# Create and activate Python virtual environment
-python3 -m venv folksonomy
-. ./folksonomy/bin/activate
-
-# Install required packages
-pip install -r requirements.txt
+# Install dependencies using Poetry
+poetry install
 
 # Set up PostgreSQL database
 sudo -i -u postgres createuser $USER
 sudo -i -u postgres createdb folksonomy -O $USER
 
-# Initialize the database using yoyo-migrations
-yoyo apply --database postgresql:///folksonomy
+# Initialize the database using yoyo-migrations (inside Poetry environment)
+poetry run yoyo apply --database postgresql:///folksonomy
 
 # Create local settings file
 cp local_settings_example.py local_settings.py
 # Edit local_settings.py to fit your environment if needed
 
 # Run the application
-uvicorn folksonomy.api:app --reload --host <your-ip-address>
+poetry run uvicorn folksonomy.api:app --reload --host <your-ip-address>
 ```
 
 ## Alternative Setup Methods
 
-### Using virtualenvwrapper
+### Using Poetry with an existing repository
 
 ```bash
 # Clone repo
 git clone https://github.com/openfoodfacts/folksonomy_api.git
 cd folksonomy_api
 
-# Required packages to setup a virtualenv (optional, but recommended)
-apt install python3-virtualenv virtualenv virtualenvwrapper
-
-# Create and switch to virtualenv
-# If mkvirtualenv command is not found, search for virtualenvwrapper.sh
-# (/usr/share/virtualenvwrapper/virtualenvwrapper.sh, or /usr/bin/virtualenvwrapper.sh, for example)
+# Install Poetry if not already installed
+curl -sSL https://install.python-poetry.org | python3 -
 
-# Add the path in your bash profile
-mkvirtualenv folksonomy -p /usr/bin/python3
-workon folksonomy
+# Install dependencies
+poetry install
 
-# Install
-pip install -r requirements.txt
+# Activate the virtual environment
+poetry shell
 ```
 
 ### PostgreSQL Setup Options
@@ -108,30 +102,30 @@ docker rmi ef6f102be0da
 
 ### Database Migration
 
-The recommended method now uses `yoyo-migrations`:
+Using Poetry with yoyo-migrations:
 
 ```bash
-yoyo apply --database postgresql:///folksonomy
+poetry run yoyo apply --database postgresql:///folksonomy
 ```
 
 Alternatively, you can use the original migration script:
 
 ```bash
 # At the end, launch database migration tool; it will initialize the db and/or update the database if there are migrations to apply
 # You can run it on a regular basis to apply new migrations
-python ./db-migration.py
+poetry run python ./db-migration.py
 ```
 
 ## Run locally
 
 ```bash
-uvicorn folksonomy.api:app --reload
+poetry run uvicorn folksonomy.api:app --reload
 ```
 
 or use `--host` if you want to make it available on your local network:
 
 ```bash
-uvicorn folksonomy.api:app --reload --host <your-ip-address>
+poetry run uvicorn folksonomy.api:app --reload --host <your-ip-address>
 ```
 
 ## Run with a local instance of Product Opener
@@ -148,7 +142,7 @@ To do so you can:
 This then should work:
 
 ```bash
-uvicorn folksonomy.api:app --host 127.0.0.1 --reload --port 8888
+poetry run uvicorn folksonomy.api:app --host 127.0.0.1 --reload --port 8888
 ```
 
 You can then access the API at http://api.folksonomy.openfoodfacts.localhost:8888/docs

README.md

@@ -20,7 +20,7 @@ PostgreSQL is used as the backend database.
 
 # Dev
 
-You should create unit tests for each new feature or API change (see [test_main.py](https://github.com/openfoodfacts/folksonomy_api/blob/main/tests/test_main.py)). 
+You should create unit tests for each new feature or API change (see [test_main.py](https://github.com/openfoodfacts/folksonomy_api/blob/main/tests/test_main.py)).
 To run tests just launch:
 ```bash
 PYTHONASYNCIODEBUG=1  pytest tests/ folksonomy/
@@ -41,8 +41,8 @@ FastAPI is based on [OpenAPI](https://github.com/OAI/OpenAPI-Specification) (pre
 
 # How to install Folksonomy Engine on your local machine
 
-1. Install Python 3.8+
-2. Install pip
+1. Install Python 3.9+
+2. Install Poetry
 3. Install PostgreSQL 13+
 4. Follow the steps in [INSTALL.md](https://github.com/openfoodfacts/folksonomy_api/blob/main/INSTALL.md) to install requirements and for creating a db user
 5. Make a copy of [local_settings_example.py](https://github.com/openfoodfacts/folksonomy_api/blob/main/local_settings_example.py) and rename it to *local_settings.py*

folksonomy/api.py

@@ -601,7 +601,7 @@ async def get_unique_values(response: Response,
                             k: str,
                             owner: str = '',
                             q: str = '',
-                            limit: int = '',
+                            limit: int = 50,
                             user: User = Depends(get_current_user)):
     """
     Get the unique values of a given property and the corresponding number of products
@@ -613,8 +613,7 @@ async def get_unique_values(response: Response,
     """
     check_owner_user(user, owner, allow_anonymous=True)
     k, _ = sanitize_data(k, None)
-    if not limit:
-        limit = 50
+
     if limit > 1000:
         limit = 1000
 
@@ -644,7 +643,7 @@ async def get_unique_values(response: Response,
 
     cur, timing = await db.db_exec(sql, params)
     out = await cur.fetchone()
-    data = out[0] if out and out[0] else []
+    data = out[0] if out and out[0] is not None else []
     return JSONResponse(status_code=200, content=data, headers={"x-pg-timing": timing})
 
 
@@ -655,4 +654,4 @@ async def pong(response: Response):
     """
     cur, timing = await db.db_exec("SELECT current_timestamp AT TIME ZONE 'GMT'",())
     pong = await cur.fetchone()
-    return {"ping": "pong @ %s" % pong[0]}
+    return {"ping": "pong @ %s" % pong[0]}