Skip to content

Commit e1b06d1

Browse files
AkuukisAkuukis
authored
ci(api): check generated openapi DEV-859 (#6119)
### 💭 Notes For development purposes let's commit the generated schema, so that we can build on top of that frontend typed API helpers, see #6083. To ensure that the committed schema is always up-to-date, let's add to CI a job that checks if it's generated. P.S. along the way simplified the script with a helper for gosu. ### 👀 Preview steps 1. see the new "openapi" job 2. 🟢 [on PR showcase commit [f4d9501](f4d9501) before updating schema] notice that CI fails 3. 🟢 [on PR latest commit [936c461](936c461) after updating schema] notice that CI succeeds 4. 🟢 run locally `./run.py -cf run --rm kpi ./scripts/generate_openapi_schemas.sh` from `kobo-install` and it succeeds to regenerate schema
1 parent 77fa17d commit e1b06d1

File tree

15 files changed

+680
-447
lines changed

15 files changed

+680
-447
lines changed

‎.github/filters.yml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,12 @@ biome:
1818
- '{package*.json,patches/*.patch}' # npm + postinstall
1919
- '.github/workflows/biome.yml' # ci
2020

21+
openapi:
22+
- '{kpi,kobo,hub}/**/*.!(md)' # backend
23+
- 'dependencies/**/*.!(md)' # pip
24+
- 'pyproject.toml' # (can affect build/tests)
25+
- '.github/workflows/openapi.yml' # ci
26+
2127
pytest:
2228
- '{kpi,kobo,hub}/**/*.!(md)' # backend
2329
- 'dependencies/**/*.!(md)' # pip
@@ -26,6 +32,7 @@ pytest:
2632

2733
npm-test:
2834
- '{jsapp,test,webpack,static,scripts}/**/*.!(md|py|sh|bash)' # frontend
35+
- 'static/openapi/*' # API
2936
- '{package*.json,patches/*.patch,scripts/copy_fonts.py}' # npm + postinstall
3037
- '{tsconfig.json,.swcrc,.babelrc*,.browserslistrc}' # compilers
3138
- '{.editorconfig,.stylelint*,.eslint*,coffeelint*}' # linters

‎.github/workflows/ci-main.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,9 @@ jobs:
1111
biome:
1212
uses: ./.github/workflows/biome.yml
1313

14+
openapi:
15+
uses: ./.github/workflows/openapi.yml
16+
1417
pytest:
1518
uses: ./.github/workflows/pytest.yml
1619

‎.github/workflows/ci-pr.yml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ jobs:
1717
all: ${{ steps.filter.outputs.all }}
1818
darker: ${{ steps.filter.outputs.darker }}
1919
biome: ${{ steps.filter.outputs.biome }}
20+
openapi: ${{ steps.filter.outputs.openapi }}
2021
pytest: ${{ steps.filter.outputs.pytest }}
2122
npm-test: ${{ steps.filter.outputs.npm-test }}
2223

@@ -30,6 +31,11 @@ jobs:
3031
if: needs.changes.outputs.all == 'true' || needs.changes.outputs.biome == 'true'
3132
uses: ./.github/workflows/biome.yml
3233

34+
openapi:
35+
needs: changes
36+
if: needs.changes.outputs.all == 'true' || needs.changes.outputs.openapi == 'true'
37+
uses: ./.github/workflows/openapi.yml
38+
3339
pytest:
3440
needs: changes
3541
if: needs.changes.outputs.all == 'true' || needs.changes.outputs.pytest == 'true'

‎.github/workflows/openapi.yml‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
2+
3+
name: openapi
4+
5+
on: workflow_call
6+
jobs:
7+
build:
8+
runs-on: ubuntu-24.04
9+
env:
10+
DATABASE_URL: postgis://kobo:kobo@localhost:5432/kpi_test
11+
DJANGO_LANGUAGE_CODES: "ar cs de-DE en es fr hi ku pl pt tr zh-hans"
12+
DJANGO_SECRET_KEY: notSecretJustForTestingYep
13+
DJANGO_SETTINGS_MODULE: kobo.settings.testing
14+
REDIS_SESSION_URL: redis://localhost:6379/2
15+
CACHE_URL: redis://localhost:6379/3
16+
ENKETO_REDIS_MAIN_URL: redis://localhost:6379/0
17+
CELERY_BROKER_URL: redis://localhost:6379/1
18+
KOBOCAT_MEDIA_ROOT: /tmp/test_media
19+
KOBOCAT_URL: http://kobocat
20+
KOBOCAT_INTERNAL_URL: http://kobocat
21+
KOBOFORM_URL: http://kpi
22+
SKIP_TESTS_WITH_CONCURRENCY: 'True'
23+
KPI_SRC_DIR: '.'
24+
strategy:
25+
matrix:
26+
python-version: ['3.10']
27+
services:
28+
postgres:
29+
image: postgis/postgis:14-3.4
30+
env:
31+
POSTGRES_USER: kobo
32+
POSTGRES_PASSWORD: kobo
33+
POSTGRES_DB: kpi_test
34+
ports:
35+
- 5432:5432
36+
options: --health-cmd pg_isready --health-interval 10s --health-timeout 5s --health-retries 5
37+
redis_cache:
38+
image: redis:6.2
39+
ports:
40+
- 6379:6379
41+
steps:
42+
43+
- uses: actions/checkout@v4
44+
45+
- name: Set up Python ${{ matrix.python-version }}
46+
uses: actions/setup-python@v5
47+
with:
48+
python-version: ${{ matrix.python-version }}
49+
50+
- name: Install pip-tools
51+
run: python -m pip install pip-tools==7.\*
52+
53+
- name: Update Debian package lists
54+
run: sudo DEBIAN_FRONTEND=noninteractive apt-get -y update
55+
56+
- name: Install Debian dependencies
57+
# All about YAML line breaks: https://stackoverflow.com/a/21699210
58+
run: >-
59+
sudo DEBIAN_FRONTEND=noninteractive apt-get -y install
60+
gdal-bin gettext libproj-dev postgresql-client ffmpeg
61+
gcc libc-dev build-essential
62+
63+
- name: Install Python dependencies
64+
run: pip-sync dependencies/pip/dev_requirements.txt
65+
66+
- name: Generate OpenAPI
67+
run: ./scripts/generate_openapi_schemas.sh
68+
69+
- name: Fail on uncommitted OpenAPI changes
70+
run: |
71+
git diff
72+
git diff-index --exit-code HEAD

‎.github/workflows/pytest.yml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,6 @@ jobs:
1212
DJANGO_SECRET_KEY: notSecretJustForTestingYep
1313
DJANGO_SETTINGS_MODULE: kobo.settings.testing
1414
REDIS_SESSION_URL: redis://localhost:6379/2
15-
SERVICE_ACCOUNT_BACKEND_URL: redis://localhost:6379/6
1615
CACHE_URL: redis://localhost:6379/3
1716
ENKETO_REDIS_MAIN_URL: redis://localhost:6379/0
1817
CELERY_BROKER_URL: redis://localhost:6379/1

‎.github/workflows/release-2-stabilize.yml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ jobs:
1919
all: ${{ steps.filter.outputs.all }}
2020
darker: ${{ steps.filter.outputs.darker }}
2121
biome: ${{ steps.filter.outputs.biome }}
22+
openapi: ${{ steps.filter.outputs.openapi }}
2223
pytest: ${{ steps.filter.outputs.pytest }}
2324
npm-test: ${{ steps.filter.outputs.npm-test }}
2425

@@ -32,6 +33,11 @@ jobs:
3233
if: needs.changes.outputs.all == 'true' || needs.changes.outputs.biome == 'true'
3334
uses: ./.github/workflows/biome.yml
3435

36+
openapi:
37+
needs: changes
38+
if: needs.changes.outputs.all == 'true' || needs.changes.outputs.openapi == 'true'
39+
uses: ./.github/workflows/openapi.yml
40+
3541
pytest:
3642
needs: changes
3743
if: needs.changes.outputs.all == 'true' || needs.changes.outputs.pytest == 'true'

‎.github/workflows/release-3-tag.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,9 @@ jobs:
1515
biome:
1616
uses: ./.github/workflows/biome.yml
1717

18+
openapi:
19+
uses: ./.github/workflows/openapi.yml
20+
1821
pytest:
1922
uses: ./.github/workflows/pytest.yml
2023

‎biome.jsonc‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,8 @@
55
"node_modules/**",
66
"jsapp/compiled/**",
77
"*.css",
8+
// Ignore generated OpenAPI schema.
9+
"static/openapi/*",
810
// Ignore minified committed JS files.
911
"*.min.js",
1012
"*-min.js",

‎kpi/utils/schema_extensions/url_builder.py‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,12 @@ def build_url_type(viewname: str, **kwargs) -> dict:
1717
This utility helps produce meaningful URLs in Swagger UI, instead of dummy or
1818
unrelated placeholders.
1919
"""
20+
DEV_DOMAIN_NAMES = [
21+
'http://kpi',
22+
'http://kf.kobo.local',
23+
'http://kf.kobo.localhost',
24+
]
25+
2026
example_url = settings.KOBOFORM_URL + '/api/v2/' + viewname
2127
if ':' in viewname:
2228
_, viewname = viewname.split(':')
@@ -85,8 +91,13 @@ def build_url_type(viewname: str, **kwargs) -> dict:
8591
for key, value in kwargs.items():
8692
example_url = example_url.replace(f'{{{key}}}', value)
8793

94+
koboform_url = settings.KOBOFORM_URL
95+
96+
if koboform_url in DEV_DOMAIN_NAMES:
97+
koboform_url = 'https://kf.kobotoolbox.org'
98+
8899
return {
89100
'type': 'string',
90101
'format': 'uri',
91-
'example': settings.KOBOFORM_URL + example_url,
102+
'example': koboform_url + example_url,
92103
}

‎kpi/utils/two_database_configuration_checker.py‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,10 @@ def db_contains_app_migrations(db_connection, app):
142142
return True
143143

144144
def do_checks(self, app_configs, **kwargs):
145+
146+
if settings.ENV == 'testing':
147+
return []
148+
145149
checks = [
146150
self.check_for_two_databases,
147151
self.check_for_distinct_databases,

0 commit comments

Comments
 (0)