Add support for RDS DB with read replica

[RealOrangeOne]

What is the context of this PR?

• https://jira.ons.gov.uk/browse/CMS-214
• https://jira.ons.gov.uk/browse/CMS-217

IAM authentication

In production, rather than a username and password, authentication to the database will be done using IAM. This means there are no credentials to leak or rotate.

When a connection is opened, a signed token is generated and used for authentication. This token is only valid for 15 minutes. The complexities of this are handled by an external library.

Connecting the application to IAM itself is done by the cluster, hence no additional configuration.

Multiple database connections

In production, multiple database connections will be used: one for writes and one for reads. The write connection points to a single database server, whilst reads are load balanced between a set of servers ("read replicas").

It's up to the application to decide whether the write or read connections is used, which is what the database router handles. Django internally identifies whether a query is a "read" or a "write", and uses the router to identify the connection. Most of the time, there's no need for a developer to worry about this.

Specifically:

• Writes should always be handled by the write server (read replica connections are read-only)
• Reads should usually be handled by the read replicas
• If there is an open transaction, reads should be handled by the write connection. This ensures that whilst a transaction is open, the application gets a consistent view of state (since data held in the transaction can't be seen by the read connection).

Note: AWS is responsible for balancing traffic across the replicas - this is not the responsibility of the application.

For consistency, local development also uses multiple connections handled by the same router, however these point to the same server, and there's no restriction that the read connection only handles reads.

How to review

This shouldn't make a difference to local development.

Relevant Django documentation: https://docs.djangoproject.com/en/5.1/topics/db/multi-db/

Follow-up Actions

There are discussions happening about being able to handle a completely read-only connection. "read replica" functionality is separate.

[RealOrangeOne]

Putting this PR on hold whilst the pytest issues are resolved in a separate PR

[AdamHawtin]

Looks good to me, and I was also able to get the functional tests working rebased off this branch with a small update 👍.

[MebinAbraham]

Code looks fine, unable to test for 'real' atm.

[sanjeevz3009]

Reviewed =) Have no comments. Some of the code here overlapped with some things I was looking into to come up with the solution for the BDD UI test suite issues, particularly the code in test_db_router.py tests interestingly.

[sanjeevz3009]

LGTM!