© Spagbo 26 May, 2023 // 2022-2024

❗️ Introducing

✨1. Credit & Usage // Sumary:

About the project:

• This project will show you 100% on-prem, basically how to Install MongoDB & MariaDB using Helm chart and ArgoCD. Also deploy, backup and restore data on it.
• This project is exactly the second part of the K8s one and includes work on both Kubernetes (K8s) and VPS.

Credit

• This project is written by me and wouldn't be possible without the hard work and contributions of the following individuals: @QuocNVC - Bug fixes and enhancements, @TruongLM - VM script writer.
• For Learning-purposes Only, meant for educational and non-commercial use. Feel free to study, learn from it.
• Has No Unauthorized Copying. Please refrain from directly copying or using it for any commercial or production purposes without proper authorization.
• If you find this project helpful, consider giving credit by linking back to this repository. Mentioning it in your own project's documentation or README is appreciated.

✨2. Todo list, Table of contents:

1. On K8s

• Deploy Mariadb
• Deploy Mongodb
• Dump, Restore data

2. On VPS

• Deploy Mariadb
• Deploy Mongodb
• Dump, Restore data

3. Advanced, Optional

• Covert, install Replicaset, Masterslave

---

❗️ Guides

✨ Setup

• Firstly! as I mentioned, this project is a continuation of k8s project. Therefore, you should consider using the existing VM from it or starting with an entirely new VM. However, I HIGHLY RECOMMEND the first option because we’ll need to deploy the database on Kubernetes after this.
• Secondly, let’s clarify everything once more. I’ll only focus on aspects related to the DB in this project. All setup steps and VM scripts must be within the K8s project. Feel free to review it thoroughly to ensure 100% accuracy before you begin.
• What we need to do in setup steps (requirements)
  • 2 VMs for Master, Worker node.
  • K8s cluster.
  • CI/CD system already in operation. (skip this step if you followed since K8s).

Caution

• Please skip the entire setup steps if you’ve already succeeded with them in a previous Kubernetes project. The setup steps are intended for those who are starting a brand new project.
• This is just a template, you need to modify these script ALL by yourself, even the DB config.
• Highly recommend Nodeport in this case.

✨ Install and work with Database on K8s

1:

• Install and setup ArgocD, create Namespace by follow these steps.
• Change it to LoadBalancer

  $ kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}'

2:

• Skip this step if your CI/CD is still in operation or setup if you don’t have one. For more information, check this.
• Login into ArgoCD, Settings/Add Repository. Add your repo and helm to update the config for it. Dont forget to create an application if you didn't
• Fill the form like the 1st picture and and you will see the exactly the same with the 2nd picture
  
  ^{Pic. 1}
  
  
  
  ^{Pic. 2}
  
  

3:

• In short, this step involves modifying your .yaml files in your repository by adding some configuration for your DB.
• If you’ve started a brand new project, you should check these Helm packages by Bitnami for MariaDB & MongoDB to get more information and read them in detail.
• Or update your Chart.yaml and values.yaml files by adding this and config them as yours
  • Chart.yaml

    dependencies:
      - name: mariadb
        version: 11.0.11
        repository: https://charts.bitnami.com/bitnami
        condition: mariadb.enabled
      - name: mongodb
        version: 12.1.16
        repository: https://charts.bitnami.com/bitnami
        condition: mongodb.enabled

  • values.yaml

    mongodb:
      enabled: true
      nameOverride: "mongodb"
      fullnameOverride: "mongodb"
      architecture: "standalone"
      auth:
        rootUser: "YOUR_ROOT_USER"
        rootPassword: "YOUR_ROOTPASSWORD"
      service:
        type: NodePort or PortForward
        nodePort: YOUR_PORT
      persistence:
        enabled: false
        #existingClaim:
    
    mariadb:
      enabled: true
        image:
        debug: true
      nameOverride: "mariadb"
      fullnameOverride: "mariadb"
      architecture: "standalone"
      auth:
        rootPassword: "YOUR_ROOT_PASSWORD"
        password: "YOUR_PASSWORD"
      primary:
        resources:
          requests:
          cpu: 
          memory: 
          limits:
          cpu:
          memory:
      service:
        type: NodePort or PortForward
        nodePort: YOUR_PORT
      persistence:
        enabled: false
        #existingClaim:
      volumePermissions:
        enabled: true

• Commit and refresh.

4:

• THIS STEP IS OPTIONAL FOR ADVANCED USERS // Once you have done all these steps upthere you'll see that you have Dbs in your k8s cluster. And to make sure about that we need tools to test the connection before you move the next step.

Note

Installation guide

MariaDB

• You need to download mySQL to test your connection.
• For installation check this

MongoDB

• With MongoDB we have so many different options, you can decide it.
• For installation check this

5: (Optional)

• You can import your real data, or even fake data. I'm gonna use Mockaroo create some basic data.
• In case you using Mockaroo, fill it to generate your data then Save as .CSV or .JSON.

6:

• After the databases have been created, we need to create a database within them. YES! You’re not wrong if you’ve followed along until this step. In the previous step, we just applied to get a database on the cluster, but it’s empty.
• Explain a few small, simple steps I'll do next:
  • Exec into the Database via Rancher terminal.
  • Create Database, Table and insert Data
  • For more syntax, please visit dev.mysql.com and mongodb.com/docs.

7:

MariaDB:

• Use Mysql Workbench to test the connection and work with it. If you're following my method fill it like in the form below. or nevgative to MariaDB using Rancher. Select mariadb-node and execute shell to start it

  
  ^{Using Workbench}
  
  
  
  ^{Using Rancher}
  
  

• Run this command with the following password in your .yaml file.

  $ mysql -u root -p

• Create database by run the command below or use Mysql Workbench

  # command
  $ create database example_name
  
  # result
  $ Query OK, 1 row affected (0.02 sec)

• Use these command below: (read the # each)

  # Show all the database
  $ show databases;
  
  # Use the database you want
  $ use database_name;
  
  # Create a table (in this case, I've created a table with 5 columns and inserted values through each line).
  $ create table employee_list_2 (employee_id INT, firstname VARCHAR(16), lastname VARCHAR(16), jobtitle VARCHAR(16), salary VARCHAR(16));
  
  # Insert data into the table
  $ INSERT INTO employee_list_2 (employee_id, firstname, lastname, jobtitle, salary) VALUES ('1', 'Sigrid', 'Bowkett', 'Librarian', '51907');

• Dump / backup and restore with --all (for more dump and restore options, check this)

  # Dump
  $ mysqldump -u admin -p test > backup.sql
  
  # Restore
  $ mysql -u admin -p test < backup.sql

  
  
  

• Result:

  
  
  

MongoDB:

Caution

(Coming soon) I'm currently working to fix this errors asap. Because of some errors, so I'm still trying. Thankss for your patience.

---

✨ Install and work with Database on VPS

1:

MariaDB

Warning

• During the installation, you will be prompted to set a root password for MariaDB. Enter a secure password and remember it, as you will need it later.
• Based on your requirements, different versions could be chosen.

• Update the package list for upgrades and new package installations:

  $ sudo apt update

• Install MariaDB by running the following command:

  $ sudo apt install mariadb-server

• After the installation is complete, MariaDB should be start automatically. However, you can verify its status:

  $ sudo systemctl status mariadb

• Secure your MariaDB installation by running the following command:

  $ sudo mysql_secure_installation

• Access:

  $ sudo mysql -u root -p

2:

MongoDB

• Update the package list for upgrades and new package installations:

  $ sudo apt update

• Install MongoDB:

  $ sudo apt install mongodb

• Optional | MongoDB will start automatically. However, you can verify its status:

  $ sudo systemctl status mongodb

• With default configuration, MongoDB listens on the localhost interface. To access it from external machines, you may need to modify the MongoDB configuration.

  $ sudo nano /etc/mongodb.conf

  Inside the configuration file, look for the bind_ip directive and change its value to the IP address you want MongoDB to listen on. If you want MongoDB to listen on all available IP addresses, set it to 0.0.0.0

• Save the changes and restart MongoDB to apply the configuration changes:

  $ sudo systemctl restart mongodb

3:

MariaDB

• Create a user with FULL permission although maria request to create a password during the installations. Check this and following exactly same steps.

• Start with this command. In this case I created an account and grant privileges for an account name admin

  $ mysql -u admin -p

• Once you're logged in. Start with those command below to show databases and create one if you dont have 1.

  $ show databases;
  $ create database example_name;

• To create table with the title of the columns and the quantity of them (how many, what kind of data,...etc). In this case my template just for and list with ID, first, last name, job and salary. For more about datatype for exmaple varchar(16) check this

  $ create table employee_list_2 (employee_id INT, firstname VARCHAR(16), lastname VARCHAR(16), jobtitle VARCHAR(16), salary VARCHAR(16));

• Imagine it might be a table with 5 columns like this and now you have to insert data into it thru each row. This command to insert data and values:

  
  

  ID	First Name	Last Name	Job title	Salary

  
  

  $ INSERT INTO employee_list_2 (employee_id, firstname, lastname, jobtitle, salary) VALUES ('1', 'Sigrid', 'Bowkett', 'Librarian', '51907');

• Once you're done. Roll back by type exit. With me im gonna create a backup folder name backup cd into it and run the backup commnad and cat these file to make sure that my data was succesfully dumped.

  $ mysqldump -u admin -p test > backup.sql

  
  
  

• And the you can get into your db and run the same command up there to drop your table. remove your data and start to practice the backup

  
  
  

• Ok. Now you're done. Backup your data into your db again

  $ mysql -u admin -p test < backup.sql

  
  
  

MongoDB:

• After the installations, starting by run the command:

  $ mongo

• Show the databases you're having // use or create a db if you havent created one

  $ show dbs
  $ use db_name

  
  ^{in this case im using "test_mongo"}
  
  

• Insert data into it

  $ db.items.insertOne({document})
  $ db.items.insertMany([{document 1}, {document 2}])

• In this case im gonna insert a name list have 3 people and ID, age with 2 methods insertOne & insertMany. Result will return as below

  
  
  

• Dump and restore data --all

  # Dump, backup
  $ mongodump -d test_mongo -o /backup
  
  # Restore
  $ mongorestore --db test_mongo /backup/test_mongo

  
  
  

✨Explain:

• I created a directory, folder name backup by running mkdir backup
• Then ran the dump command up there to backup data and it will automatically create a folder name test_mongo included 2 files: items.bson and items.metadata.json
• Type mongo again and drop the table in the test_mongo to remove the data
  
  
  
• Then use the backup command to restore data into the test_mongo again. here is the results:
  
  
  

---

✨ Install Master-slave for MariaDB, Replicaset for MongoDB

Warning

• My configuration is stored at HA-Setup
• This is optional in this project!
• Configuring can be considered intermediate to advanced skills
• You can create a consistent backup of the master database using your preferred method (e.g., mysqldump, XtraBackup) and transfer the backup file to the slave server.
• Remember to allow incoming traffic to your VMs using UFW.

  ufw allow from x.x.x.x to any port 27017
  ufw allow from x.x.x.x to any port 27017

MariaDB:

• run this command:

  $ sudo nano /etc/mysql/mariadb.conf.d/50-server.cnf

• Copy the config below or in mariadb.cnf in the master folder // Find and modify the following lines:

  [mysqld]
  server-id = 1          # Unique ID for the master
  log-bin = /var/log/mysql/mysql-bin.log
  binlog_do_db = your_database_name

• Restart to apply the config

  $ sudo systemctl restart mariadb

• Setup the master node

  # Login
  $ sudo mysql -u root -p
  
  # Create replication user
  $ CREATE USER 'replication_user'@'%' IDENTIFIED BY 'password';
  $ GRANT REPLICATION SLAVE ON *.* TO 'replication_user'@'%';
  $ FLUSH PRIVILEGES;
  $ EXIT;

• Copy the config below or in mariadb.cnf in the slave folder // Find and modify the following lines:

  [mysqld]
  server-id = 2          # Unique ID for the slave

• Setup the slave node

  # Login
  $ sudo mysql -u root -p
  
  # Setup command
  CHANGE MASTER TO MASTER_HOST='master_ip_address',
  MASTER_USER='replication_user',
  MASTER_PASSWORD='password',
  MASTER_LOG_FILE='binlog_file_from_master',
  MASTER_LOG_POS=binlog_position_from_master;

• Run the slave, and check the status

  START SLAVE;
  SHOW SLAVE STATUS \G;

• Look for and your're all done

  Slave_IO_Running: Yes
  Slave_SQL_Running: Yes

  
  
  

MongoDB:

• Access the config file

  $ nano /etc/mongod.conf

• Copy the config below or in mongod.cnf copy the config.

  replication:
  replSetName: "rs0"

  
  
  

• Restart to apply the config

  $ systemctl restart mongod

• Log in to the primary node

  $ mongo
  $ rs.initiate()

• Add replica-set

  $ rs.add(“mongo-db2:27017”)
  $ rs.add(“mongo-db3:27017”)

Tip

• Replace "server1", "server2", and "server3" with your actual server hostnames or IP addresses.
• In this case, mine was db1, db2,...

  rs.initiate({
    _id: "rs0",
    members: [
      { _id: 0, host: "server1:27017" },
      { _id: 1, host: "server2:27017" },
      { _id: 2, host: "server3:27017" }
    ]
  })

• OPTIONAL To make sure a specific member-node becomes the primary:

  cfg = rs.conf()
  cfg.members[0].priority = 2
  rs.reconfig(cfg)

• Once you add the nodes, you will see the output as {‘ok’:1}, which indicates a successful addition of nodes in the replica set. Check it by running

  $ rs.status()

• If your output look ike this, congrats!

  { 
  "set" : "test",
  "date" : ISODate("2023-07-28T04:31:50.570Z"),
  "myState" : 1, 
  "members" : [ 
     { 
        "_id" : 0, 
        "name" : "192.168.56.200:27017", 
        "health" : 1, 
        "state" : 1, 
        "stateStr" : "PRIMARY", 
        "uptime" : 303165, 
        "optime" : Timestamp(1644516902, 1), 
        "optimeDate" : ISODate("2022-02-10T06:15:02Z"), 
        "self" : true 
     }, 
     { 
        "_id" : 1, 
        "name" : "192.168.56.201:27017", 
        "health" : 1, 
        "state" : 2, 
        "stateStr" : "SECONDARY", 
        "uptime" : 302985, 
        "optime" : Timestamp(1644516902, 1), 
        "optimeDate" : ISODate("2022-02-10T06:15:02Z"), 
        "lastHeartbeat" : ISODate("2022-02-10T06:15:02Z"), 
        "lastHeartbeatRecv" : ISODate("2014-08-12T06:15:02Z"), 
        "pingMs" : 0, 
        "syncingTo" : "10.20.30.40:27017" 
     },
     { 
        "_id" : 2, 
        "name" : "192.168.56.202:27017", 
        "health" : 1, "state" : 2, 
        "stateStr" : "SECONDARY", 
        "uptime" : 302985, 
        "optime" : Timestamp(1644516902, 1), 
        "optimeDate" : ISODate("2022-02-10T06:15:02Z"), 
        "lastHeartbeat" : ISODate("2022-02-10T06:15:02Z"), 
        "lastHeartbeatRecv" : ISODate("2022-02-10T06:15:02Z"), 
        "pingMs" : 0, 
        "syncingTo" : "192.168.56.200:27017" 
     } 
  ], 
  "ok" : 1 
  } 

---

Congrats! you're doing great. Thanks for reading until this and good luck with your journey.
✨ Best wishes, 𝓃𝓃𝒷𝒸,