Test with two database where one is external and readonly

[layoaster]

I have a Django application A that takes the user's data from another Django application B with its own database. So basically I implemented a Django database router so the users are taken from a an external database with read-only permissions because the application A not creating users.

As a result, I have two databases in my settings.:

DATABASE_ROUTERS = ["config.routerdb.AuthRouter"]
DATABASES = {
    "default": {
        "ENGINE": "django.db.backends.postgresql",
        "NAME": environ.get("OPERATIONAL_DB_DATABASE", "moto"),
        "USER": environ.get("OPERATIONAL_DB_USER", "postgres"),
        "PASSWORD": environ.get("OPERATIONAL_DB_PASS", "invalid-pass"),
        "HOST": environ.get("OPERATIONAL_DB_HOST", "localhost"),
        "PORT": environ.get("OPERATIONAL_DB_PORT", "5432"),
    },
    "auth_db": {
        "ENGINE": "django.db.backends.postgresql",
        "NAME": environ.get("AUTH_DB_DATABASE", "cleverea"),
        "USER": environ.get("AUTH_DB_USER", "postgres_ro"),
        "PASSWORD": environ.get("AUTH_DB_PASS", "invalid-pass"),
        "HOST": environ.get("AUTH_DB_HOST", "localhost"),
        "PORT": environ.get("AUTH_DB_PORT", "5432"),
    },
}

Being auth_db the external read-only database. I've been trying to tell pytest-django with the help of a settings.test to simply create the users database on the local database with the following:

from .base import *  # noqa
DATABASE_ROUTERS = []

DATABASES["auth_db"]["NAME"]: environ.get("OPERATIONAL_DB_DATABASE", "moto")
DATABASES["auth_db"]["USER"]: environ.get("OPERATIONAL_DB_USER", "postgres")
DATABASES["auth_db"]["PASSWORD"]: environ.get("OPERATIONAL_DB_PASS", "invalid-pass")
DATABASES["auth_db"]["HOST"]: environ.get("OPERATIONAL_DB_HOST", "localhost")
DATABASES["auth_db"]["PORT"]: environ.get("OPERATIONAL_DB_PORT", "5432")

But for some reason I cannot explain I still get the same error:

Creating test database for alias 'default' ('test_moto')...
Got an error creating the test database: database "test_moto" already exists

Destroying old test database for alias 'default' ('test_moto')...
Creating test database for alias 'auth_db' ('test_cleverea')...
Got an error creating the test database: permission denied to create database

I honestly don't event create any users for tests since I tell factory boy to use a build strategy for the User factory and I use the factory so I can force Django Restf to make authenticated calls.

So I don't know what else I could do. Any Ideas?

[blueyed]

Multiple dbs are not really well supported currently - there are existing issues in that regard already, and maybe even stalled PRs. Please look around there.

I'd be happy to help with getting this solved, of course - it's just a bit tricky and it would be great if you could help there.

[layoaster]

@blueyed I've been looking at related issues PRs but they are more focused to support actual testing over multiple DB's which isn't exactly my case. Also, they are based on manipulating the multidb property that has been deprecated in the latest versions.

Because of the way Django handles initialization one cannot simply override DATABASES so I had to create a dedicated settings file for tests where I simply use one DB and remove any DB router.

Since my application doesn't create users but I have to be able to make authenticated requests with the test client I managed to fake the User model with the help of factory boy so I could inject a fake user to make authenticated requests work nicely.

Not being able to use hierarchical settings for tests is the main drawback of my workaround because I have to keep in sync the settings for tests with the ones in base.py.

However, I couldn't find a way to make pytest-django/django use a single database for tests an remove the routing. Tried playing with the fixture django_db_setup but nothing.

[Cabalist]

I'm running into a similar issue with an external read-only transaction (not 'transactional') database.

I have functions that query this database and load information into the primary db. It is setup in DATABASES in the django settings as a second database. I've loaded it up prior to running tests but I'm getting a psycopg2.errors.UndefinedTable: relation "transactions" does not exist exception whenever I have code that has a cursor in that connection.

@blueyed I've seen your comment here and haven't quite been able to get that to work.

[bluetech]

I'll be opening a new issue on multi-db support, I'll close this one as a duplicate.

[JemPak]

after research, explore and read many documents about how to test a legacy read-only database , If you only want to execute tests using legacy database as a test database, the best and simple solution is to setup the same legacy database as a MIRROR in the TEST configuration of the database. I know, it's not a good practice but this solves my need.

Example:

{
'default': {
    'ENGINE': get_secret("DEFAULT_ENGINE_DB_LOCAL"),
    'NAME': get_secret("DEFAULT_NAME_DB_LOCAL"),
    'HOST': get_secret("DEFAULT_HOST_DB_LOCAL"),
    ....
},
'legacy_db': {
    'ENGINE': get_secret("SIESA_ENGINE_DB_LOCAL"),
    'NAME': get_secret("SIESA_NAME_DB_LOCAL"),
    'HOST': get_secret("SIESA_HOST_DB_LOCAL"),
    ....
    'TEST': {
        'MIRROR': 'legacy_db',
    }
}

also, you can use it with --keepdb command to avoid creating a new test database for your default
python manage.py test --keepdb

I am working with Django 2.2