Skip to content

PD-2218 / 25.10 / Pd 2218 update replication task content (by micjohnson777) #4204

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Oct 9, 2025
Merged

PD-2218 / 25.10 / Pd 2218 update replication task content (by micjohnson777) #4204

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
15b7dc7
Empty commit to create PR on github.
bugclerk Oct 3, 2025
bb98e11
PD-2218 / 26.04 / Pd 2218 update replication task content (#4191)
micjohnson777 Oct 3, 2025
68e3c3c
Merge branch 'PD-2218-25.10' of https://github.com/truenas/documentat…
micjohnson777 Oct 7, 2025
69ab70c
PD-2218 Fix broken links
micjohnson777 Oct 7, 2025
df5b2a4
Merge branch '25.10' into PD-2218-25.10
micjohnson777 Oct 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 12 additions & 86 deletions ...ataProtection/Replication/AdvancedReplication/ReplicationWithEncryptionSCALE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,112 +17,38 @@ You can use encrypted datasets in a local replication.
You can set up a replication task for a dataset encrypted with a passphrase or a hex encryption key, but you must unlock the dataset before the task runs or the task fails.
{{< /hint>}}

With the implementation of the **Local Administrator** user and role-based permissions, when setting up remote replication tasks when logged in as an admin user requires selecting **Use Sudo For ZFS Commands**.
With the implementation of the **Local Administrator** user and role-based permissions, when setting up remote replication tasks when logged in as an admin user, requires selecting **Use Sudo For ZFS Commands**.

{{< include file="/static/includes/ReplicationIntroSCALE.md" >}}

Remote replication with datasets also require an SSH connection in TrueNAS. You can use an existing SSH connection if it has the same user credentials you want to use for the new replication task.

## Setting Up a Simple Replication Task Overview

This section provides a simple overview of setting up a remote replication task for an encrypted dataset.
It also covers the related steps you should take prior to configuring the replication task.

{{< expand "Replication Task General Overview" "v" >}}

1. Set up the data storage for where you want to save replicated snapshots.

2. Make sure the admin user has a home directory assigned.

3. Create an SSH connection between the local TrueNAS system and the remote system.
You can do this by either going to **Credentials > Backup Credentials > SSH Connection** and clicking **Add** or from the **Replication Task Wizard** using the **Generate New** option for the remote system.

4. Unlock the encrypted dataset(s) and export the encryption key to a text editor like Notepad.

5. Go to **Data Protection > Replication Tasks** and click **Add** to open the **Replication Task Wizard**.
Specify the from and to sources, task name, and set the schedule.

Setting options change based on the source selections. Replicating to or from a local source does not requires an SSH connection.

This completes the general process for all replication tasks.
{{< /expand >}}
{{< include file="/static/contents/BasicReplicationProcess.md" >}}

## Creating a Remote Replication Task for an Encrypted Dataset

To streamline creating simple replication tasks use the **Replication Task Wizard** to create and copy ZFS snapshots to another system.
To streamline creating simple replication tasks, use the **Replication Task Wizard** to create and copy ZFS snapshots to another system.
The wizard assists with creating a new SSH connection and automatically creates a periodic snapshot task for sources that have no existing snapshots.

If you have an existing replication task, you can select it on the **Load Previous Replication Task** dropdown list to load the configuration settings for that task into the wizard, and then make change such as assigning it a different destination, select encryption options, schedule, or retention lifetime, etc.
If you have an existing replication task, you can select it on the **Load Previous Replication Task** dropdown list to load the configuration settings for that task into the wizard, and then make changes such as assigning it a different destination, selecting other options like encryption, schedule, or retention lifetime, etc.
Saving changes to the configuration creates a new replication task without altering the task you loaded into the wizard.
This saves some time when creating multiple replication tasks between the same two systems.

{{< include file="/static/includes/ReplicationCreateDatasetAndAdminHomeDirSteps.md" >}}

3. Unlock the source dataset and export the encryption key to a text editor such as Notepad.
Go to **Datasets** select the source dataset, locate the **ZFS Encryption** widget and unlock the dataset if locked.
Export the key and paste it in any text editor such as Notepad. If you set up encryption to use a passphrase, you do not need to export a key.

4. Go to **Data Protection** and click **Add** on the **Replication Tasks** widget to open the **Replication Task Wizard**. Configure the following settings:

{{< trueimage src="/images/SCALE/DataProtection/CreateRemoteReplicationTask.png" alt="New Remote Replication Task" id="New Remote Replication Task" >}}

a. Select **On this System** on the **Source Location** dropdown list.
If your source is the local TrueNAS system, you must select **On a Different System** from the **Destination Location** dropdown list to do remote replication.

If your source is a remote system, create the replication task as the root user and select **On a Different System**. The **Destination Location** automatically changes to **On this System**.

TrueNAS shows the number of snapshots available for replication.
Unlock the source dataset and export the encryption key to a text editor such as Notepad.
Go to **Datasets** select the source dataset, locate the **ZFS Encryption** widget and unlock the dataset if locked.
Export the key and paste it in any text editor such as Notepad. If you set up encryption to use a passphrase, you do not need to export a key.

b. Select an existing SSH connection to the remote system or create a new connection.
Select **Create New** to open the **[New SSH Connection](#configure-a-new-ssh-connection)** configuration screen.

c. Browse to the source pool/dataset(s), then click on the dataset(s) to populate the **Source** with the path.
You can select multiple sources or manually type the names into the **Source** field. Separate multiple entries with commas.
Selecting **Recursive** replicates all snapshots contained within the selected source dataset snapshots.

d. Repeat to populate the **Destination** field.
You cannot use zvols as a remote replication destination.
Add a **/*datasetname*** to the end of the destination path to create a new dataset in that location.

e. (Optional) Select **Encryption** to add a [second layer of encryption](#adding-additional-encryption) over the already encrypted dataset.

f. Select **Use Sudo for ZFS Commands**. Only displays when logged in as the admin user (or the name of the admin user).
This removes the need to issue the cli `zfs allow` command in Shell on the remote system.
When the dialog displays, click **Use Sudo for ZFS Comands**. If you close this dialog, select the option on the **Add Replication Task** wizard screen.

{{< trueimage src="/images/SCALE/DataProtection/UseSudoForZFSCommandsDialog.png" alt="Select Use Sudo for ZFS Commands" id="Select Use Sudo for ZFS Commands" >}}

This option only displays when logged in as the admin user.
If not selected you need to issue the cli `zfs allow` command in Shell on the remote system.

g. Select **Replicate Custom Snapshots**, then accept the default value in **Naming Schema**.
Remote sources require entering a snapshot naming schema to identify the snapshots to replicate.
A naming schema is a pattern of naming custom snapshots you want to replicate.
If you want to change the default schema, enter the name and [strftime(3)](https://man7.org/linux/man-pages/man3/strftime.3.html) %Y, %m, %d, %H, and %M strings that match the snapshots to include in the replication.
Separate entries by pressing <kbd>Enter</kbd>. The number of snapshots matching the patterns display.

h. (Optional) Enter a name for the snapshot in **Task Name**.
TrueNAS populates this field with the default name using the source and destination paths separated by a hyphen, but this default can make locating the snapshot in destination dataset a challenge.
To make it easier to find the snapshot, give it a name that is easy for you to identify. For example, a replicated task named *dailyfull* for a full file system snapshot taken daily.

{{< include file="/static/includes/ReplicationScheduleAndRetentionSteps.md" >}}
Configure the remote replication as described in the [Creating a Remote Replication Task]({{< ref "RemoteReplicationSCALE.md" >}}).

{{<include file="/static/includes/addcolumnorganizer.md">}}

### Configure a New SSH Connection

{{< include file="/static/includes/ReplicationConfigNewSSHConnection.md" >}}

### Using SSH Transfer Security

{{< include file="/static/includes/ReplicationSSHTransferSecurity.md" >}}

### Unlocking the Destination Dataset

After the replication task runs and creates the snapshot on the destination, you must unlock it to access the data.
After the replication task runs and creates the snapshot on the destination, you must unlock the dataset to access the data.
Click the <span class="iconify" data-icon="mdi:download"></span> from the replication task options to download a key file that unlocks the destination dataset.

### Replicating to an Unencrypted Destination Dataset

{{< hint type=important >}}
TrueNAS does not support preserving encrypted dataset properties when trying to re-encrypt an already encrypted source dataset.
{{< /hint >}}
Expand All @@ -141,10 +67,10 @@ This replicates the unlocked encrypted source dataset to an unencrypted destinat

### Using Additional Encryption Options

When you replicate an encrypted pool or dataset you have one level of encryption applied at the data storage level.
When you replicate an encrypted pool or dataset, you have one level of encryption applied at the data storage level.
Use the passphrase or key created or exported from the dataset or pool to unlock the dataset on the destination server.

To add a second layer of encryption at the replication task level, select **Encryption** on the **Replication Task Wizard**, then select the type of encryption you want to apply.
To add a second layer of encryption at the replication task level, select **Encryption** on the **Replication Task Wizard** or on the **Add Replication Task** screen, then select the type of encryption you want to apply.

{{< trueimage src="/images/SCALE/DataProtection/ReplicationTaskEncryptionOptions.png" alt="Replication Task Encryption Options" id="Replication Task Encryption Options" >}}

Expand Down
45 changes: 22 additions & 23 deletions content/SCALETutorials/DataProtection/Replication/AdvancedReplication/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ keywords:
- data backup and recovery
---

TrueNAS advanced replication allows users to create one-time or regularly scheduled snapshots of data stored in pools, datasets or zvols on their TrueNAS system as a way to back up stored data.
TrueNAS advanced replication allows users to create one-time or regularly scheduled snapshots of data stored in pools, datasets, or zvols on their TrueNAS system as a way to back up stored data.
When properly configured and scheduled, local or remote replication using the **Advanced Replication Creation** option takes regular snapshots of storage pools or datasets and saves them in the destination location on the same or another system.

{{< include file="/static/includes/ReplicationIndexContentSCALE.md" >}}

The **Advanced Replication Creation** option opens the **Add Replication Task** screen.
This screen provides access to the same settings found in the replication wizard but has more options to specify:
This screen provides access to the same settings found in the replication wizard, but has more options to specify:

* Full file system replication
* Stream compression
Expand Down Expand Up @@ -56,30 +56,29 @@ Turn on SSH service. Go to **System > Services** screen, verify the **SSH** serv

## Creating a Simplified Advanced Replication Task

To access advanced replication settings, click **Advanced Replication Creation** at the bottom of the first screen of the **Replication Task Wizard**.
To access advanced replication settings, click **Advanced Replication Creation** at the bottom of the first replication wizard screen.
The **Add Replication Task** configuration screen opens.

{{< include file="/static/includes/ReplicationCreateDatasetAndAdminHomeDirSteps.md" >}}

3. Give the task a name and set the direction of the task.
1. Give the task a name.
Unlike the wizard, the **Name** does not automatically populate with the *source*/*destination* task name after you set the source and destination for the task.
Each task name must be unique, and we recommend you name it in a way that makes it easy to remember what the task is doing.
Each task name must be unique, and we recommend naming it in a way that makes it easy to remember what the task is doing.

4. Select the direction of the task. **Pull** replicates data from a remote system to the local system. **Push** sends data from the local system to the remote.
2. Select the direction of the task. **Pull** pulls data from a remote system to the local system. **Push** sends data from the local system to the remote.

5. Select the method of tranfer for this replication from the **Transport** dropdown list.
Select **LOCAL** to replicate data to another location on the same system.
Select **SSH** is the standard option for sending or receiving data from a remote system. Select the existing **SSH Connection** from the dropdown list.
Select **SSH+Netcat** is available as a faster option for replications that take place within completely secure networks.
**SSH+Netcat** requires defining netcat ports and addresses to use for the Netcat connection.
3. Select the transfer method for this replication from the **Transport** dropdown list.
* Select **LOCAL** to replicate data to another location on the same system.
* Select **SSH** is the standard option for sending or receiving data from a remote system.
When selected, also select an SSH connection from the **SSH Connection** dropdown list.
* Select **SSH+Netcat** is a faster option for replication when it occurs within a completely secure network.
**SSH+Netcat** requires defining NETCAT ports and addresses to use for the NETCAT connection.

With SSH-based replications, select the **SSH Connection** to the remote system that sends or receives snapshots.
With SSH-based replications, select the SSH Connection with the remote system from which you want to receive snapshots or send snapshots to.
To create a new connection to use for replication from a destination to this local system, select **newpullssh**.

Select **Use Sudo for Zfs Commands** to controls whether the user used for SSH/SSH+NETCAT replication has passwordless sudo enabled to execute zfs commands on the remote host.
Select **Use Sudo for Zfs Commands** to control whether the user for SSH/SSH+NETCAT replication has passwordless sudo enabled to execute zfs commands on the remote host.
If not selected, you must enter `zfs allow` on the remote system to to grant non-user permissions to perform ZFS tasks.

6. Specify the source and destination paths. Adding /*name* to the end of the path creates a new dataset in that location.
4. Specify the source and destination paths. Destination paths allow adding /*name* to the end of the path to create a new dataset in that location.
Click the arrow to the left of each folder or dataset name to expand the options and browse to the dataset, then click on the dataset to populate the **Source**.
Choose a preconfigured periodic snapshot task as the source of snapshots to replicate.
Pulling snapshots from a remote source requires a valid **SSH Connection** before the file browser can show any directories.
Expand All @@ -95,11 +94,11 @@ The **Add Replication Task** configuration screen opens.
*DO NOT* use zvols as remote destinations.
{{< /hint >}}

7. Select a previously configured periodic snapshot task for this replication task in **Periodic Snapshot Tasks**.
5. Select a previously configured periodic snapshot task for this replication task in **Periodic Snapshot Tasks**.
The replication task selected must have the same values in **Recursive** and **Exclude Child Datasets** as the chosen periodic snapshot task.
Selecting a periodic snapshot schedule removes the **Schedule** field.

If a periodic snapshot task does not exist, exist the advanced replication task configuration, go configure a periodic snapshot task, then return to the **Advanced Replication** screen to configure the replication Task.
If a periodic snapshot task does not exist, before creating the advanced replication task, configure a periodic snapshot task, then return to the **Add Replication Task** screen to configure the replication Task.
Select **Replicate Specific Snapshots** to define specific snapshots from the periodic task to use for the replication.
This displays the schedule options for the snapshot task. Enter the schedule.
The only periodically generated snapshots included in the replication task are those that match your defined schedule.
Expand All @@ -109,9 +108,9 @@ The **Add Replication Task** configuration screen opens.
For example, entering the naming schema `custom-%Y-%m-%d_%H-%M` finds and replicates snapshots like `custom-2020-03-25_09-15`.
Enter multiple schemas by pressing <kbd>Enter</kbd> to separate each schema.

8. Set the replication schedule to use and define when the replication task runs.
6. Set the replication schedule to use and define when the replication task runs.
Leave **Run Automatically** selected to use the snapshot task specified and start the replication immediately after the related periodic snapshot task completes.
Select **Schedule** to display scheduling options for this replication task and To automate the task according to its own schedule.
Select **Schedule** to display scheduling options for this replication task, and to run the task according to its own schedule.

Selecting **Schedule** allows scheduling the replication to run at a separate time.
Choose a time frame that gives the replication task enough time to finish and is during a time of day when network traffic for both source and destination systems is minimal.
Expand All @@ -121,7 +120,7 @@ The **Add Replication Task** configuration screen opens.
{{< include file="/static/includes/SCALEAdvancedScheduler.md" >}}
{{< /expand >}}

9. Click **Save**.
7. Click **Save**.

### Setting a Replication Compression Level

Expand All @@ -140,7 +139,7 @@ Replication tasks using large block replication only continue to work as long as

By default, the replication task uses snapshots to quickly transfer data to the receiving system.
Selecting **Full Filesystem Replication** means the task completely replicates the chosen **Source**, including all dataset properties, snapshots, child datasets, and clones.
When using this option, we recommended allocating additional time for the replication task to run.
When using this option, we recommend allocating additional time for the replication task to run.

### Replicating Dataset Properties

Expand Down Expand Up @@ -177,7 +176,7 @@ You can store the encryption key either in the TrueNAS system database or in a c

{{< hint type=important >}}
**Synchronizing Destination Snapshots With Source** destroys any snapshots in the destination that do not match the source snapshots.
TrueNAS also does a full replication of the source snapshots as if the replication task never run, which can lead to excessive bandwidth consumption.
TrueNAS also does a full replication of the source snapshots as if the replication task had not run, which can lead to excessive bandwidth consumption.

This can be a very destructive option.
Make sure that any snapshots deleted from the destination are obsolete or otherwise backed up in a different location.
Expand Down
Loading