Skip to content

Maintenance

Radu Nichita edited this page Aug 28, 2020 · 30 revisions

Upgrades

At some point after installing, you'll want to upgrade to a newer version of the Liquid bundle.

Watch out for breaking changes in the configuration by looking at the changelogs (tag messages) in:

1. Update the cluster to version X.Y.Z:

cd /opt/cluster
git fetch
git checkout vX.Y.Z
# check for new settings in `examples/cluster.ini` and add them to `cluster.ini`
bin/docker.sh --rm --pull --image liquidinvestigations/cluster:X.Y.Z
# wait for services to be ready
docker exec cluster ./cluster.py wait

2. Update liquid to version X.Y.Z:

cd /opt/node
git fetch
git checkout vX.Y.Z
# check for new settings in `examples/liquid.ini` and add them to `liquid.ini`
./liquid deploy

Shutting down

To shut down the bundle, stop the cluster container, with a generous timeout:

docker stop -t 120 cluster

If Nomad isn't configured to drain jobs (or the drain operation fails) you may also need to kill all containers it left behind:

docker stop $(docker ps -qa)

Clean reset

Some Nomad versions require clearing the server data for upgrades. In those cases it's best to clear the docker system images too, using the following command sequence.

docker stop -t 120 cluster
docker stop $(docker ps -qa)
docker system prune --all --force --volumes
sudo rm -rf /opt/cluster/var/nomad

... and continue with deploying normally as described in the beginning of this article:

cd /opt/cluster
bin/docker.sh --rm --pull --image liquidinvestigations/cluster:X.Y.Z
docker exec cluster ./cluster.py wait

cd /opt/node
./liquid deploy

Note: if the docker system prune --all --force --volumes command takes too long to finish, sudo systemctl restart docker and re-run it.

Backup

run ./liquid backup [OPTIONS] $DIR to back up the system.

Optional parameters to specify which parts of the system will be backed up:

--no-apps : excludes app data from backup
--no-collections : excludes collections from backup
--no-pg : excludes collection databases from backup
--no-es : excludes collection es snapshots from backup
--no-blobs : excludes collection blobs (Binary Large OBjects) from backup
--collection $NAME : backup specific collection

When no optional parameter is given, the whole system will be backed up.

Restore

Make sure a clean system is installed before starting with the restore process.

Restoring apps: ./liquid restore-apps $DIR

Restoring collections: ./liquid restore-all-collections $DIR

Note: The collections have to be disabled in the liquid.ini to be restored. After the restore process is finished the collections have to be enabled in the liquid.ini again to be accessible.

To finish the restore process run ./liquid deploy.

Other problems

Use the Admin FAQ for miscellaneous operating details and the Security page on how to keep a production system secure.

Clone this wiki locally