Skip to content

Commit e55d8ca

Browse files
dephektdephekt
authored
Merge pull request #5156 from platformsh/5152-mariadb-replication
MariaDB and PostgreSQL read-only replication
2 parents dbea23b + e655de9 commit e55d8ca

File tree

10 files changed

+538
-8
lines changed

10 files changed

+538
-8
lines changed

‎sites/platform/src/add-services/mysql/_index.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,11 @@ Their infrastructure setup is nearly identical, though they differ in some featu
1111
See the [MariaDB documentation](https://mariadb.org/documentation/)
1212
or the Oracle [MySQL Server documentation](https://dev.mysql.com/doc/refman/en/) for more information.
1313

14+
{{% note theme="info" title="MariaDB note" %}}
15+
The [example](#mariadb-example) provided later in this topic is specific to using only a **primary** MariaDB database. For details about using read-only replicas to improve performance of read-heavy applications, see the [MariaDB read-only replication](/add-services/mysql/mysql-readonly-replication.md) topic.
16+
{{% /note %}}
17+
18+
1419
## Use a framework
1520

1621
If you use one of the following frameworks, follow its guide:
@@ -178,6 +183,10 @@ With the above definition, the application container now has [access to the serv
178183

179184
### MariaDB example
180185

186+
{{% note theme="info" %}}
187+
Use the steps and sample code below if your application will connect to a **primary** MariaDB database. For details about using read-only replicas to improve performance of read-heavy applications, see the [MariaDB read-only replication](/add-services/mysql/mysql-readonly-replication.md) topic.
188+
{{% /note %}}
189+
181190
#### [Service definition](/add-services/_index.md)
182191

183192
```yaml {configFile="services"}

‎sites/platform/src/add-services/mysql/mysql-readonly-replication.md‎

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
---
2+
title: "MariaDB read-only replication"
3+
sidebarTitle: "MariaDB read-only replication"
4+
description: Configure and access read-only MariaDB replicas to ease the load on a primary database.
5+
---
6+
7+
You can improve the performance of read-heavy applications by defining read-only replicas of your MariaDB database and then connecting your application to those replicas.
8+
9+
Examples of read-heavy applications include:
10+
- Listing pages or dashboards
11+
- Reporting or analytics jobs
12+
- Background jobs that frequently query data
13+
14+
{{< note theme="info" title="Note" >}}
15+
- **To prevent data loss or interruptions** during replication, you must configure the disk size for each replica. The replica service does not inherit the disk size of the primary database. The replica disk size must at least match the primary service's disk capacity. See the example below.
16+
- **Replication is asynchronous**: Delays of a few milliseconds might occur between writes on the primary database and reads on the replica database.
17+
- **Replicas are read-only**: This restriction ensures data consistency and integrity. Attempts to modify data will result in an SQL error.
18+
{{< /note >}}
19+
20+
### Read-only vs. external replicas
21+
Read-only replicas are used primarily to improve application performance by distributing database read requests from read-heavy applications.
22+
23+
Other common use cases for read-only replicas include:
24+
- Cross-region backup: Replicating data to different geographical regions
25+
- Data warehousing: Extracting data from production to analytics projects
26+
27+
[External replicas](/add-services/mysql/mysql-replication.md) reside on remote servers and have different use cases, including disaster recovery.
28+
29+
### Replica scope and sharing services {#replica-scope-sharing-services}
30+
MariaDB services (which provide access to databases and replicas) defined in a project cannot be accessed by or shared with applications in other projects.
31+
32+
To share the MariaDB service among applications in different projects, you must complete the following high-level steps:
33+
1. Migrate the {{% vendor/name %}} projects to {{% vendor/company_name %}} Flex.
34+
1. Migrate the applications that are in those projects.
35+
1. Move the desired applications into one Flex project, rather than separate Flex projects. In {{% vendor/company_name %}} Flex, multiple applications in one project can share the same services.
36+
37+
For details, see [Converting from {{% vendor/name %}}](https://docs.upsun.com/learn/tutorials/migrating/from-fixed.html) in the {{% vendor/company_name %}} Flex product docs.
38+
39+
## 1. Configure the primary and replica services
40+
41+
The following code fragment defines two MariaDB services: a primary and a replica. You can use this fragment as a template by copying it into your `services.yaml` or `application.yaml` file.
42+
43+
Be sure to:
44+
- Replace `<VERSION>` with the [supported MariaDB version](/add-services/mysql/_index.md#supported-versions) that you need. Use the same version number for the primary and replica services.
45+
- Replace `<SIZE>` with a `disk` size in (MB) that is sufficient for the primary database's disk capacity.
46+
- **Important:** Use `replicator` as the endpoint name when you define the replica service. This is a special endpoint name that the replica service uses to connect to the database.
47+
48+
```yaml {configFile="services"}
49+
services:
50+
db:
51+
type: mariadb:<VERSION>
52+
disk: <SIZE>
53+
configuration:
54+
schemas:
55+
- main
56+
endpoints:
57+
main:
58+
default_schema: main
59+
privileges:
60+
main: admin
61+
replicator:
62+
privileges:
63+
main: replication
64+
65+
db-replica1:
66+
type: mariadb-replica:<VERSION>
67+
disk: <SIZE>
68+
configuration:
69+
schemas:
70+
- main
71+
endpoints:
72+
main:
73+
default_schema: main
74+
privileges:
75+
main: admin
76+
relationships:
77+
primary: db:replicator # Do not change the name `primary`. The service expects to receive this name.
78+
79+
db-replica2:
80+
type: mariadb-replica:<VERSION>
81+
disk: <SIZE>
82+
configuration:
83+
schemas:
84+
- main
85+
endpoints:
86+
main:
87+
default_schema: main
88+
privileges:
89+
main: admin
90+
relationships:
91+
primary: db:replicator # Do not change the name `primary`. The service expects to receive this name.
92+
```
93+
94+
### How it works
95+
96+
Using the sample code fragment above:
97+
98+
1. The primary service (`db`) defines an additional `replicator` endpoint, granting the `replication` privilege. This enables a replica to connect and continuously replicate data from the primary database.
99+
100+
```yaml
101+
replicator:
102+
privileges:
103+
main: replication
104+
```
105+
106+
2. The replica services (`db-replica1` and `db-relica2`) use the `mariadb-replica` image type and connect back to the primary database service through the primary relationship. This establishes a replication link from `db` (the source) to `db-replica` (the target).
107+
108+
```yaml
109+
relationships:
110+
primary: db:replicator
111+
```
112+
113+
The `db-replica1` and `db-replica2` replica services continuously stream data from the primary endpoint, maintaining a read-only copy of the primary database content. Write operations are not permitted on the replicas.
114+
115+
116+
## 2. Define the relationship between the application and the replicas
117+
118+
Even if your application won't access the replication endpoint, you must expose the endpoint to an application as a relationship so that you can connect to it over SSH.
119+
120+
Add a new relationship to your application container, as shown below:
121+
122+
```yaml {configFile="app"}
123+
name: myapp
124+
125+
[...]
126+
127+
# Relationships enable an app container's access to a service.
128+
relationships:
129+
# More information: https://fixed.docs.upsun.com/anchors/fixed/app/reference/relationships/
130+
database:
131+
service: db
132+
endpoint: main
133+
database-readonly:
134+
service: db-replica
135+
```
136+
137+

‎sites/platform/src/add-services/mysql/mysql-replication.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,10 @@
11
---
2-
title: "MariaDB/MySQL External Replication"
3-
sidebarTitle: "MariaDB/MySQL Replication"
2+
title: "MariaDB/MySQL external replication"
3+
sidebarTitle: "External replication"
44
description: In rare cases, it may be useful to maintain a replica instance of your MySQL/MariaDB database outside of {{% vendor/name %}}.
55
---
66

7-
In rare cases, it may be useful to maintain a replica instance of your MySQL/MariaDB database outside of {{% vendor/name %}}.
7+
In rare cases, it may be useful to maintain a replica instance of your MySQL/MariaDB database outside of {{% vendor/name %}} on a remote server.
88

99
Typically, an automated backup is better for short-term usage and a `mysqldump` for longer term storage, but in some cases the data set is large enough that `mysqldump` is prohibitive.
1010
In that case, you can enable external replication using an extra permission.

‎sites/platform/src/add-services/postgresql.md‎ renamed to ‎sites/platform/src/add-services/postgresql/_index.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,10 @@ PostgreSQL is a high-performance, standards-compliant relational SQL database.
88

99
See the [PostgreSQL documentation](https://www.postgresql.org/docs/9.6/index.html) for more information.
1010

11+
{{% note theme="info" title="Note" %}}
12+
The [example](#usage-example) provided later in this topic is specific to using only a **primary** database. For details about using read-only replicas to improve performance of read-heavy applications, see the [PostgreSQL read-only replication](/add-services/postgresql/postgresql-readonly-replication.md) topic.
13+
{{% /note %}}
14+
1115
## Use a framework
1216

1317
If you use one of the following frameworks, follow its guide:
@@ -97,6 +101,10 @@ So your apps should only rely on the `{{% vendor/prefix %}}_RELATIONSHIPS` envir
97101

98102
## Usage example
99103

104+
{{% note theme="info" %}}
105+
Use the steps and sample code below if your application will connect to a **primary** PostgreSQL database. For details about using read-only replicas to improve performance of read-heavy applications, see the [PostgreSQL read-only replication](/add-services/postgresql/postgresql-readonly-replication.md) topic.
106+
{{% /note %}}
107+
100108
### 1. Configure the service
101109

102110
To define the service, use the `postgresql` type:

‎sites/platform/src/add-services/postgresql/postgresql-readonly-replication.md‎

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
---
2+
title: "PostgreSQL read-only replication"
3+
sidebarTitle: "Read-only replication"
4+
description: Configure and access read-only PostgreSQL replicas to ease the load on a primary database.
5+
---
6+
7+
You can improve the performance of read-heavy applications by defining read-only replicas of your PostgreSQL database and then connecting your application to those replicas.
8+
9+
Examples of read-heavy applications include:
10+
- Listing pages or dashboards
11+
- Reporting or analytics jobs
12+
- Background jobs that frequently query data
13+
14+
{{< note theme="info" title="Note" >}}
15+
- **To prevent data loss or interruptions** during replication, you must configure the disk size for each replica. The replica service does not inherit the disk size of the primary database. The replica disk size must at least match the primary service's disk capacity. See the example below.
16+
- **Replication is asynchronous**: Delays of a few milliseconds might occur between writes on the primary database and reads on the replica database.
17+
- **Replicas are read-only**: This restriction ensures data consistency and integrity. Attempts to modify data will result in an SQL error.
18+
{{< /note >}}
19+
20+
### Replica scope and sharing services {#replica-scope-sharing-services}
21+
PostgreSQL services (which provide access to databases and replicas) defined in a project cannot be accessed by or shared with applications in other projects.
22+
23+
To share the PostgreSQL service among applications in different projects, you must complete the following high-level steps:
24+
1. Migrate the projects to {{% vendor/company_name %}} Flex.
25+
1. Migrate the applications that are in those projects.
26+
1. Move the desired applications into one Flex project, rather than separate Flex projects. In {{% vendor/company_name %}} Flex, multiple applications in one project can share the same services.
27+
28+
For details, see [Converting from {{% vendor/name %}}](https://docs.upsun.com/learn/tutorials/migrating/from-fixed.html) in the {{% vendor/company_name %}} Flex product docs.
29+
30+
## 1. Configure the primary and replica services
31+
32+
The following code fragment defines two MariaDB services: a primary and a replica. You can use this fragment as a template by copying it into your `services.yaml` or `application.yaml` file.
33+
34+
Be sure to:
35+
- Replace `<VERSION>` with the [supported PostgreSQL version](/add-services/postgresql/_index.md#supported-versions) that you need. Use the same version number for the primary and replica services.
36+
- Replace `<SIZE>` with a `disk` size in (MB) that is sufficient for the primary database's disk capacity.
37+
- **Important:** Use `replicator` as the endpoint name when you define the replica service. This is a special endpoint name that the replica service uses to connect to the database.
38+
39+
```yaml {configFile="services"}
40+
services:
41+
db:
42+
type: postgresql:<VERSION>
43+
disk: <SIZE>
44+
configuration:
45+
extensions:
46+
- postgis
47+
endpoints:
48+
replicator:
49+
replication: true
50+
51+
db-replica1:
52+
type: postgres-replica:<VERSION>
53+
disk: <SIZE>
54+
configuration:
55+
endpoints:
56+
postgresql:
57+
default_database: main
58+
extensions:
59+
- postgis
60+
relationships:
61+
primary: db:replicator # Do not change the name `primary`. The service expects to receive this name.
62+
63+
db-replica2:
64+
type: postgres-replica:<VERSION>
65+
disk: <SIZE>
66+
configuration:
67+
endpoints:
68+
postgresql:
69+
default_database: main
70+
extensions:
71+
- postgis
72+
relationships:
73+
primary: db:replicator # Do not change the name `primary`. The service expects to receive this name.
74+
```
75+
76+
### How it works
77+
78+
Using the sample code fragment above:
79+
80+
1. After you define the replicator endpoints, the primary service creates a `replicator` user that has permission to replicate the database. You must specify `replicator` as the endpoint name, as described at the start of this topic.
81+
82+
```yaml
83+
endpoints:
84+
replicator:
85+
replication: true
86+
```
87+
88+
2. In the replica services (in this example, db-replica1 and db-replica2), defining the relationship `primary: db:replicator` ensures that the service can connect to the primary database as the `replicator` user.
89+
90+
```yaml
91+
relationships:
92+
primary: db:replicator
93+
```
94+
95+
The `db-replica1` and `db-replica2` replica services continuously stream data from the primary endpoint, maintaining a read-only copy of the primary database content. Write operations are not permitted on the replicas.
96+
97+
98+
## 2. Define the relationship between the application and the replicas
99+
100+
Even if your application won't access the replication endpoint, you must expose the endpoint to an application as a relationship so that you can connect to it over SSH.
101+
102+
Add a new relationship to your application container, as shown below:
103+
104+
```yaml {configFile="app"}
105+
name: myapp
106+
107+
[...]
108+
109+
# Relationships enable an app container's access to a service.
110+
relationships:
111+
# More information: https://fixed.docs.upsun.com/anchors/fixed/app/reference/relationships/
112+
database:
113+
service: db
114+
endpoint: main
115+
database-readonly:
116+
service: db-replica
117+
```
118+
119+

‎sites/upsun/src/add-services/mysql/_index.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,11 @@ Their infrastructure setup is nearly identical, though they differ in some featu
1111
See the [MariaDB documentation](https://mariadb.org/documentation/)
1212
or the Oracle [MySQL Server documentation](https://dev.mysql.com/doc/refman/en/) for more information.
1313

14+
{{% note theme="info" title="MariaDB note" %}}
15+
The [example](#mariadb-example) provided later in this topic is specific to using only a **primary** MariaDB database. For details about using read-only replicas to improve performance of read-heavy applications, see the [MariaDB read-only replication](/add-services/mysql/mysql-readonly-replication.md) topic.
16+
{{% /note %}}
17+
18+
1419
## Supported versions
1520

1621
You can select the major and minor version.
@@ -307,6 +312,10 @@ With the above definition, the application container now has [access to the serv
307312

308313
### MariaDB example
309314

315+
{{% note theme="info" %}}
316+
Use the steps and sample code below if your application will connect to a **primary** MariaDB database. For details about using read-only replicas to improve performance of read-heavy applications, see the [MariaDB read-only replication](/add-services/mysql/mysql-readonly-replication.md) topic.
317+
{{% /note %}}
318+
310319
{{< codetabs >}}
311320

312321
+++

0 commit comments

Comments
 (0)