restore data in background or via cron job on a different server to create another copy of the data

README.md

@@ -24,6 +24,47 @@ This script automates encrypted, deduplicated backups of local directories to a
 
 -----
 
+## Quick Start
+
+For those familiar with setting up backup scripts, here is a fast track to get you up and running.
+
+1. **Download Files:**
+
+    ```sh
+    mkdir -p /root/scripts/backup && cd /root/scripts/backup
+    curl -LO https://raw.githubusercontent.com/buildplan/restic-backup-script/refs/heads/main/restic-backup.sh
+    curl -LO https://raw.githubusercontent.com/buildplan/restic-backup-script/refs/heads/main/restic-backup.conf
+    curl -LO https://raw.githubusercontent.com/buildplan/restic-backup-script/refs/heads/main/restic-excludes.txt
+    chmod +x restic-backup.sh
+    ```
+
+2. **Edit Configuration:**
+    - Modify `restic-backup.conf` with your repository details, source paths, and password file location.
+    - Set secure permissions: `chmod 600 restic-backup.conf`.
+
+3. **Create Password & Initialize:**
+
+    ```sh
+    # Create the password file (use a strong password)
+    echo 'your-very-secure-password' | sudo tee /root/.restic-password
+    sudo chmod 400 /root/.restic-password
+
+    # Initialize the remote repository
+    sudo ./restic-backup.sh --init
+    ```
+
+4. **Run First Backup & Schedule:**
+
+    ```sh
+    # Run your first backup with verbose output
+    sudo ./restic-backup.sh --verbose
+
+    # Set up a recurring schedule with the interactive wizard
+    sudo ./restic-backup.sh --install-scheduler
+    ```
+
+-----
+
 ## Usage
 
 ### Run Modes
@@ -38,6 +79,8 @@ This script automates encrypted, deduplicated backups of local directories to a
 - `sudo ./restic-backup.sh --install-scheduler` - Run the interactive wizard to set up an automated backup schedule (systemd/cron).
 - `sudo ./restic-backup.sh --uninstall-scheduler` - Remove a schedule created by the wizard.
 - `sudo ./restic-backup.sh --restore` - Start the interactive restore wizard.
+- `sudo ./restic-backup.sh --background-restore <snapshot> <dest>` - Restore in the background (non-blocking).
+- `sudo ./restic-backup.sh --sync-restore <snapshot> <dest>` - Restore in a cronjob (helpful for 3-2-1 backup strategy).
 - `sudo ./restic-backup.sh --forget` - Manually apply the retention policy and prune old data.
 - `sudo ./restic-backup.sh --diff` - Show a summary of changes between the last two snapshots.
 - `sudo ./restic-backup.sh --stats` - Display repository size, file counts, and stats.
@@ -49,6 +92,75 @@ This script automates encrypted, deduplicated backups of local directories to a
 
 > *Default log location: `/var/log/restic-backup.log`*
 
+-----
+
+### Restoring Data
+
+Script provides three distinct modes for restoring data, each designed for a different scenario.
+
+#### 1. Interactive Restore (`--restore`)
+
+This is an interactive wizard for guided restores. It is the best option when you are at the terminal and need to find and recover specific files or directories.
+
+- **Best for**: Visually finding and restoring specific files or small directories.
+- **Process**:
+      - Lists available snapshots for you to choose from.
+      - Asks for a destination path.
+      - Performs a "dry run" to show you what will be restored before making any changes.
+      - Requires your confirmation before proceeding with the actual restore.
+
+**Usage:**
+
+```sh
+sudo ./restic-backup.sh --restore
+```
+
+#### 2. Background Restore (`--background-restore`)
+
+This mode is designed for restoring large amounts of data (e.g., a full server recovery) without needing to keep your terminal session active.
+
+- **Best for**: Large, time-consuming restores or recovering data over a slow network connection.
+- **How it works**:
+  - This command is **non-interactive**. You must provide the snapshot ID and destination path as arguments directly on the command line.
+  - The restore job is launched in the background, immediately freeing up terminal.
+  - All output is saved to a log file in `/tmp/`.
+  - A success or failure notification (via ntfy, Discord, etc.) upon completion.
+
+**Usage:**
+
+```sh
+# Restore the latest snapshot to a specific directory in the background
+sudo ./restic-backup.sh --background-restore latest /mnt/disaster-recovery
+
+# Restore a specific snapshot by its ID
+sudo ./restic-backup.sh --background-restore a1b2c3d4 /mnt/disaster-recovery
+```
+
+#### 3. Synchronous Restore (`--sync-restore`)
+
+This mode runs the restore in the foreground and waits for it to complete before exiting. It's a reliable, non-interactive way to create a complete, consistent copy of backup data.
+
+- **Best for**: Creating a secondary copy of backup (for example, via a cron job) on another server (for a 3-2-1 strategy) or for use in any automation where subsequent steps depend on the restore being finished.
+- **How it works**:
+  - This command is **non-interactive** and requires the snapshot ID and destination path as command-line arguments.
+  - It runs as a synchronous (blocking) process. When a cron job executes the command, the job itself will not finish until the restore is 100% complete.
+  - This guarantees the data copy is finished before any other commands are run or the cron job is marked as complete.
+
+**Usage:**
+
+```sh
+# On a second server, pull a full copy of the latest backup
+sudo ./restic-backup.sh --sync-restore latest /mnt/local-backup-copy
+
+# On your secondary server, run a sync-restore every day at 5:00 AM.
+0 5 * * * /path/to/your/script/restic-backup.sh --sync-restore latest /path/to/local/restore/copy >> /var/log/restic-restore.log 2>&1
+
+# Can also be used in a script to ensure a process runs only after a restore
+sudo ./restic-backup.sh --sync-restore latest /srv/app/data && systemctl restart my-app
+```
+
+-----
+
 #### Diagnostics & Error Codes
 
 The script uses specific exit codes for different failures to help with debugging automated runs.
@@ -115,8 +227,11 @@ uname -m
 
 ```sh
 # Download the latest binary for your architecture from the Restic GitHub page
-# Example 0.18.0 is latest as of Aug,2025 for amd64:
-curl -LO https://github.com/restic/restic/releases/download/v0.18.0/restic_0.18.0_linux_amd64.bz2
+# Go to the Restic GitHub releases page to find the URL for the latest version:
+# https://github.com/restic/restic/releases
+
+# Download the latest binary for your architecture (replace URL with the one you found)
+curl -LO <URL_of_latest_restic_linux_amd64.bz2>
 ```
 
 ```sh
@@ -126,6 +241,8 @@ chmod +x restic_*
 sudo mv restic_* /usr/local/bin/restic
 ```
 
+-----
+
 #### Package Breakdown
 
 | Package       | Required For                                                                                                         |
@@ -183,6 +300,8 @@ The most reliable way for the script to connect to a remote server is via an SSH
     sudo ssh storagebox pwd
     ```
 
+-----
+
 ### 3. Place and Configure Files
 
 1. Create your script directory:
@@ -287,6 +406,8 @@ Before the first backup, you need to create the repository password file and ini
     sudo ./restic-backup.sh --init
     ```
 
+-----
+
 ### 5. Set up an Automated Schedule (Recommended)
 
 The easiest and most reliable way to schedule your backups is to use the script's built-in interactive wizard. It will guide you through creating and enabling either a modern `systemd timer` (recommended) or a traditional `cron job`.
@@ -334,6 +455,6 @@ To run the backup automatically, edit the root crontab.
 
     ```
 
-    *For pune job in your `restic-backup.conf`, set `PRUNE_AFTER_FORGET=true`.*  
+    *For prune job in your `restic-backup.conf`, set `PRUNE_AFTER_FORGET=true`.*  
     *For more details on how forget flag work, see the [official Restic documentation on removing snapshots](https://restic.readthedocs.io/en/stable/060_forget.html).*  
     *Redirecting output to `/dev/null` is recommended, as the script handles its own logging and notifications.*

restic-backup.sh

@@ -1,14 +1,14 @@
 #!/usr/bin/env bash
 
 # =================================================================
-#         Restic Backup Script v0.37.2 - 2025.10.02
+#         Restic Backup Script v0.38 - 2025.10.04
 # =================================================================
 
 set -euo pipefail
 umask 077
 
 # --- Script Constants ---
-SCRIPT_VERSION="0.37.2"
+SCRIPT_VERSION="0.38"
 SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)
 CONFIG_FILE="${SCRIPT_DIR}/restic-backup.conf"
 LOCK_FILE="/tmp/restic-backup.lock"
@@ -302,6 +302,8 @@ display_help() {
     printf "  ${C_GREEN}%-20s${C_RESET} %s\n" "--forget" "Apply retention policy; optionally prune."
     printf "  ${C_GREEN}%-20s${C_RESET} %s\n" "--unlock" "Remove stale repository locks."
     printf "  ${C_GREEN}%-20s${C_RESET} %s\n" "--restore" "Interactive restore wizard."
+    printf "  ${C_GREEN}%-20s${C_RESET} %s\n" "--background-restore" "Run a non-interactive restore in the background."
+    printf "  ${C_GREEN}%-20s${C_RESET} %s\n" "--sync-restore" "Run a non-interactive restore in the foreground (for cron)."
     printf "  ${C_GREEN}%-20s${C_RESET} %s\n" "--dry-run" "Preview backup changes (no snapshot)."
     printf "  ${C_GREEN}%-20s${C_RESET} %s\n" "--test" "Validate config, permissions, connectivity."
     printf "  ${C_GREEN}%-20s${C_RESET} %s\n" "--install-scheduler" "Install an automated schedule (systemd/cron)."
@@ -311,6 +313,7 @@ display_help() {
     echo -e "  Run a backup now:            ${C_GREEN}sudo $prog${C_RESET}"
     echo -e "  Verbose diff summary:        ${C_GREEN}sudo $prog --verbose --diff${C_RESET}"
     echo -e "  Fix perms (interactive):     ${C_GREEN}sudo $prog --fix-permissions --test${C_RESET}"
+    echo -e "  Background restore:          ${C_GREEN}sudo $prog --background-restore latest /mnt/restore${C_RESET}"
     echo
     echo -e "${C_BOLD}${C_YELLOW}DEPENDENCIES:${C_RESET}"
     echo -e "  This script requires: ${C_GREEN}restic, curl, gpg, bzip2, less, jq, flock${C_RESET}"
@@ -1327,26 +1330,117 @@ run_restore() {
         echo -e "${C_GREEN}✅ Restore completed${C_RESET}"
 
         # Set file ownership logic
-        if [[ "$restore_dest" == /home/* ]]; then
-            local dest_user
-            dest_user=$(stat -c %U "$(dirname "$restore_dest")" 2>/dev/null || echo "${restore_dest#/home/}" | cut -d/ -f1)
-            if [[ -n "$dest_user" ]] && id -u "$dest_user" &>/dev/null; then
-                echo -e "${C_CYAN}ℹ️  Home directory detected. Setting ownership of restored files to '$dest_user'...${C_RESET}"
-                if chown -R "${dest_user}:${dest_user}" "$restore_dest"; then
-                    log_message "Successfully changed ownership of $restore_dest to $dest_user"
-                    echo -e "${C_GREEN}✅ Ownership set to '$dest_user'${C_RESET}"
-                else
-                    log_message "WARNING: Failed to change ownership of $restore_dest to $dest_user"
-                    echo -e "${C_YELLOW}⚠️  Could not set file ownership. Please check permissions manually.${C_RESET}"
-                fi
-            fi
-        fi
+        _handle_restore_ownership "$restore_dest"
+
         send_notification "Restore SUCCESS: $HOSTNAME" "white_check_mark" \
             "${NTFY_PRIORITY_SUCCESS}" "success" "Restored $snapshot_id to $restore_dest"
     fi
     rm -f "$restore_log"
 }
 
+_handle_restore_ownership() {
+    local restore_dest="$1"
+
+    if [[ "$restore_dest" == /home/* ]]; then
+        local dest_user
+        dest_user=$(stat -c %U "$(dirname "$restore_dest")" 2>/dev/null || echo "${restore_dest#/home/}" | cut -d/ -f1)
+
+        if [[ -n "$dest_user" ]] && id -u "$dest_user" &>/dev/null; then
+            log_message "Home directory detected. Setting ownership of restored files to '$dest_user'."
+            if chown -R "${dest_user}:${dest_user}" "$restore_dest"; then
+                log_message "Successfully changed ownership of $restore_dest to $dest_user"
+            else
+                log_message "WARNING: Failed to change ownership of $restore_dest to $dest_user. Please check permissions manually."
+            fi
+        fi
+    fi
+}
+
+_run_restore_command() {
+    local snapshot_id="$1"
+    local restore_dest="$2"
+    shift 2
+    mkdir -p "$restore_dest"
+
+    # Build the command
+    local restic_cmd=(restic)
+    restic_cmd+=("$(get_verbosity_flags)")
+    restic_cmd+=(restore "$snapshot_id" --target "$restore_dest")
+
+    # Add optional file paths to include
+    if [ $# -gt 0 ]; then
+        for path in "$@"; do
+            restic_cmd+=(--include "$path")
+        done
+    fi
+
+    # Execute and return success or failure
+    if run_with_priority "${restic_cmd[@]}"; then
+        return 0 # Success
+    else
+        return 1 # Failure
+    fi
+}
+
+run_background_restore() {
+    echo -e "${C_BOLD}--- Background Restore Mode ---${C_RESET}"
+
+    local snapshot_id="${1:?--background-restore requires a snapshot ID}"
+    local restore_dest="${2:?--background-restore requires a destination path}"
+
+    if [[ "$snapshot_id" == "latest" ]]; then
+        snapshot_id=$(restic snapshots --latest 1 --json | jq -r '.[0].id')
+    fi
+    if [[ -z "$restore_dest" || "$restore_dest" != /* ]]; then
+        echo -e "${C_RED}Error: Destination must be a non-empty, absolute path. Aborting.${C_RESET}" >&2
+        exit 1
+    fi
+
+    local restore_log="/tmp/restic-restore-${snapshot_id:0:8}-$(date +%s).log"
+    echo "Restore job started. Details will be logged to: ${restore_log}"
+    log_message "Starting background restore of snapshot ${snapshot_id} to ${restore_dest}. See ${restore_log} for details."
+
+    (
+        local start_time=$(date +%s)
+        if _run_restore_command "$@"; then
+            local end_time=$(date +%s)
+            local duration=$((end_time - start_time))
+            _handle_restore_ownership "$restore_dest"
+            log_message "Background restore SUCCESS: ${snapshot_id} to ${restore_dest} in ${duration}s."
+            local notification_message
+            printf -v notification_message "Successfully restored snapshot %s to %s in %dm %ds." \
+                "${snapshot_id:0:8}" "${restore_dest}" "$((duration / 60))" "$((duration % 60))"
+            send_notification "Restore SUCCESS: $HOSTNAME" "white_check_mark" \
+                "${NTFY_PRIORITY_SUCCESS}" "success" "$notification_message"
+        else
+            log_message "Background restore FAILED: ${snapshot_id} to ${restore_dest}."
+            send_notification "Restore FAILED: $HOSTNAME" "x" \
+                "${NTFY_PRIORITY_FAILURE}" "failure" "Failed to restore snapshot ${snapshot_id:0:8} to ${restore_dest}. Check log: ${restore_log}"
+        fi
+    ) > "$restore_log" 2>&1 &
+
+    echo -e "${C_GREEN}✅ Restore job launched in the background. You will receive a notification upon completion.${C_RESET}"
+}
+
+run_sync_restore() {
+    log_message "Starting synchronous restore."
+    local restore_dest="$2"
+
+    if _run_restore_command "$@"; then
+        _handle_restore_ownership "$restore_dest"
+
+        log_message "Sync-restore SUCCESS."
+        send_notification "Sync Restore SUCCESS: $HOSTNAME" "white_check_mark" \
+            "${NTFY_PRIORITY_SUCCESS}" "success" "Successfully completed synchronous restore."
+        return 0
+    else
+        log_message "Sync-restore FAILED."
+        send_notification "Sync Restore FAILED: $HOSTNAME" "x" \
+            "${NTFY_PRIORITY_FAILURE}" "failure" "Synchronous restore failed. Check the logs for details."
+        return 1
+    fi
+}
+
 run_snapshots_delete() {
     echo -e "${C_BOLD}--- Interactively Delete Snapshots ---${C_RESET}"
     echo -e "${C_BOLD}${C_RED}WARNING: This operation is permanent and cannot be undone.${C_RESET}"
@@ -1494,6 +1588,28 @@ case "${1:-}" in
         run_preflight_checks "restore" "quiet"
         run_restore
         ;;
+    --background-restore)
+        shift
+        run_preflight_checks "restore" "quiet"
+        run_background_restore "$@"
+        ;;
+    --sync-restore)
+        shift
+        run_preflight_checks "restore" "quiet"
+        log_message "=== Starting sync-restore run ==="
+        restore_exit_code=0
+        if ! run_sync_restore "$@"; then
+            restore_exit_code=1
+        fi
+        log_message "=== Sync-restore run completed ==="
+        # --- Ping Healthchecks.io (Success or Failure) ---
+        if [ "$restore_exit_code" -eq 0 ] && [[ -n "${HEALTHCHECKS_URL:-}" ]]; then
+            curl -fsS -m 15 --retry 3 "${HEALTHCHECKS_URL}" >/dev/null 2>>"$LOG_FILE"
+        elif [ "$restore_exit_code" -ne 0 ] && [[ -n "${HEALTHCHECKS_URL:-}" ]]; then
+            curl -fsS -m 15 --retry 3 "${HEALTHCHECKS_URL}/fail" >/dev/null 2>>"$LOG_FILE"
+        fi
+        exit "$restore_exit_code"
+        ;;
     --check)
         run_preflight_checks "backup" "quiet"
         run_check