Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
3654fe1
working except data migration
valentijnscholten Sep 4, 2025
43260cf
update performance unit test
valentijnscholten Sep 6, 2025
efa3d6e
add management command to import scan
valentijnscholten Sep 6, 2025
899a0c4
remove old test
valentijnscholten Sep 6, 2025
0f85eec
restore __init__.py
valentijnscholten Sep 6, 2025
205b3b3
fix tests
valentijnscholten Sep 10, 2025
e5e4cb7
fix questionaire content type in testdata
valentijnscholten Sep 10, 2025
cc8cdbf
fix delete testcases
valentijnscholten Sep 10, 2025
d1d9999
fix tests
valentijnscholten Sep 10, 2025
d47b8f0
fix query counts
valentijnscholten Sep 11, 2025
7ac5372
default to django-auditlog
valentijnscholten Sep 11, 2025
1cad6df
fix query counts
valentijnscholten Sep 11, 2025
d5d7d91
update query counts and task counts
valentijnscholten Sep 11, 2025
281294d
add indices
valentijnscholten Sep 12, 2025
7d25ee3
default to pghistory
valentijnscholten Sep 12, 2025
c2539e8
ruff
valentijnscholten Sep 12, 2025
e7053b9
migration
valentijnscholten Sep 12, 2025
f5859f3
fix backfill
valentijnscholten Sep 12, 2025
856629b
ruff
valentijnscholten Sep 12, 2025
78f5bd3
optimize diff filter
valentijnscholten Sep 13, 2025
98b04ad
Merge remote-tracking branch 'upstream/dev' into pghistory
valentijnscholten Sep 16, 2025
b26383d
Merge remote-tracking branch 'upstream/dev' into pghistory
valentijnscholten Sep 17, 2025
342137e
Merge remote-tracking branch 'upstream/dev' into pghistory
valentijnscholten Sep 23, 2025
74ff84c
update counts
valentijnscholten Sep 23, 2025
9bdcd19
Merge remote-tracking branch 'upstream/dev' into pghistory
valentijnscholten Sep 24, 2025
45608da
default to django-auditlog
valentijnscholten Sep 26, 2025
49986c7
use current user as actor for delete events
valentijnscholten Sep 27, 2025
d7fdff2
Merge remote-tracking branch 'upstream/dev' into pghistory
valentijnscholten Sep 27, 2025
5fb4889
run unit tests for both auditlog-types
valentijnscholten Sep 27, 2025
f5a0592
output more logs to verify auditlog-type
valentijnscholten Sep 27, 2025
4dc7312
Revert "use current user as actor for delete events"
valentijnscholten Sep 27, 2025
6805a5a
Reapply "use current user as actor for delete events"
valentijnscholten Sep 27, 2025
5ddc508
Revert "output more logs to verify auditlog-type"
valentijnscholten Sep 27, 2025
0c39215
optimize flush audit log
valentijnscholten Sep 27, 2025
738de71
move helper
valentijnscholten Sep 27, 2025
b0277e4
add flushing of pghistory entries
valentijnscholten Sep 27, 2025
6d3207f
Merge remote-tracking branch 'upstream/dev' into pghistory
valentijnscholten Sep 29, 2025
84ebdab
Merge remote-tracking branch 'upstream/dev' into pghistory
valentijnscholten Sep 30, 2025
ee22cb3
update counts
valentijnscholten Sep 30, 2025
93b9e01
Merge remote-tracking branch 'upstream/dev' into pghistory
valentijnscholten Oct 3, 2025
0796810
small reorder
valentijnscholten Oct 3, 2025
1fd90d1
fix tests
valentijnscholten Oct 3, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/integration-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,18 @@ name: Integration tests

on:
workflow_call:
inputs:
auditlog_type:
type: string
default: "django-auditlog"

jobs:
integration_tests:
# run tests with docker compose
name: User Interface Tests
runs-on: ubuntu-latest
env:
AUDITLOG_TYPE: ${{ inputs.auditlog_type }}
strategy:
matrix:
test-case: [
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/rest-framework-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,16 @@ on:
platform:
type: string
default: "linux/amd64"
auditlog_type:
type: string
default: "django-auditlog"

jobs:
unit_tests:
name: Rest Framework Unit Tests
runs-on: ${{ inputs.platform == 'linux/arm64' && 'ubuntu-24.04-arm' || 'ubuntu-latest' }}
env:
AUDITLOG_TYPE: ${{ inputs.auditlog_type }}

strategy:
matrix:
Expand Down
8 changes: 8 additions & 0 deletions .github/workflows/unit-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,18 +25,26 @@ jobs:
strategy:
matrix:
platform: ['linux/amd64', 'linux/arm64']
auditlog_type: ['django-auditlog', 'django-pghistory']
fail-fast: false
needs: build-docker-containers
uses: ./.github/workflows/rest-framework-tests.yml
secrets: inherit
with:
platform: ${{ matrix.platform}}
auditlog_type: ${{ matrix.auditlog_type }}

# only run integration tests for linux/amd64 (default)
test-user-interface:
needs: build-docker-containers
uses: ./.github/workflows/integration-tests.yml
secrets: inherit
strategy:
matrix:
auditlog_type: ['django-auditlog', 'django-pghistory']
fail-fast: false
with:
auditlog_type: ${{ matrix.auditlog_type }}

# only run k8s tests for linux/amd64 (default)
test-k8s:
Expand Down
14 changes: 13 additions & 1 deletion docker/entrypoint-initializer.sh
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,8 @@ python3 manage.py makemigrations --no-input --check --dry-run --verbosity 3 || {
cat <<-EOF

********************************************************************************
WARNING: Missing Database Migrations Detected
********************************************************************************

You made changes to the models without creating a DB migration for them.

Expand All @@ -119,15 +121,25 @@ If you're not familiar with migrations in Django, please read the
great documentation thoroughly:
https://docs.djangoproject.com/en/5.0/topics/migrations/

This is now a WARNING and the container will continue to start.
However, you should create the necessary migrations as soon as possible using:
docker compose exec uwsgi bash -c 'python manage.py makemigrations -v2'

********************************************************************************

EOF
exit 1
echo "WARNING: Continuing startup despite missing migrations..."
}

echo "Migrating"
python3 manage.py migrate

echo "Configuring pghistory triggers based on audit settings"
cat <<EOD | python3 manage.py shell
from dojo.auditlog import configure_pghistory_triggers
configure_pghistory_triggers()
EOD

echo "Admin user: ${DD_ADMIN_USER}"
ADMIN_EXISTS=$(echo "SELECT * from auth_user;" | python manage.py dbshell | grep "${DD_ADMIN_USER}" || true)
# Abort if the admin user already exists, instead of giving a new fake password that won't work
Expand Down
10 changes: 8 additions & 2 deletions dojo/admin.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from auditlog.models import LogEntry
from django.contrib import admin
from django.contrib.admin.sites import NotRegistered
from polymorphic.admin import PolymorphicChildModelAdmin, PolymorphicParentModelAdmin

from dojo.models import (
Expand All @@ -14,7 +14,13 @@
TextQuestion,
)

admin.site.unregister(LogEntry)
# Conditionally unregister LogEntry from auditlog if it's registered
try:
from auditlog.models import LogEntry
admin.site.unregister(LogEntry)
except (ImportError, NotRegistered):
# auditlog not available or LogEntry not registered
pass

# ==============================
# Defect Dojo Engaegment Surveys
Expand Down
8 changes: 8 additions & 0 deletions dojo/apps.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
from django.db import models
from watson import search as watson

from dojo.auditlog import configure_audit_system, register_django_pghistory_models
from dojo.checks import check_configuration_deduplication

logger = logging.getLogger(__name__)
Expand Down Expand Up @@ -87,6 +88,13 @@ def ready(self):
import dojo.test.signals # noqa: PLC0415 raised: AppRegistryNotReady
import dojo.tool_product.signals # noqa: F401,PLC0415 raised: AppRegistryNotReady

# Configure audit system after all models are loaded
# This must be done in ready() to avoid "Models aren't loaded yet" errors
# Note: pghistory models are registered here (no database access), but trigger
# enabling is handled via management command to avoid database access warnings
register_django_pghistory_models()
configure_audit_system()


def get_model_fields_with_extra(model, extra_fields=()):
return get_model_fields(get_model_default_fields(model), extra_fields)
Expand Down
Loading