btrfs-progs: docs: add more chapters (part 2)

The feature pages share the contents with the manual page section 5 so
put the contents to separate files. Progress: 2/3.

Signed-off-by: David Sterba <[email protected]>

Author: kdave

Documentation/Auto-repair.rst

@@ -1,4 +1,8 @@
 Auto-repair on read
 ===================
 
-...
+Data or metadata that are found to be damaged (eg. because the checksum does
+not match) at the time they're read from the device can be salvaged in case the
+filesystem has another valid copy when using block group profile with redundancy
+(DUP, RAID1, RAID5/6). The correct data are returned to the user application
+and the damaged copy is replaced by it.

Documentation/Convert.rst

@@ -1,4 +1,4 @@
 Convert
 =======
 
-...
+.. include:: ch-convert-intro.rst

Documentation/Deduplication.rst

@@ -1,4 +1,44 @@
 Deduplication
 =============
 
-...
+Going by the definition in the context of filesystems, it's a process of
+looking up identical data blocks tracked separately and creating a shared
+logical link while removing one of the copies of the data blocks. This leads to
+data space savings while it increases metadata consumption.
+
+There are two main deduplication types:
+
+* **in-band** *(sometimes also called on-line)* -- all newly written data are
+  considered for deduplication before writing
+* **out-of-band** *(sometimes alco called offline)* -- data for deduplication
+  have to be actively looked for and deduplicated by the user application
+
+Both have their pros and cons. BTRFS implements **only out-of-band** type.
+
+BTRFS provides the basic building blocks for deduplication allowing other tools
+to choose the strategy and scope of the deduplication.  There are multiple
+tools that take different approaches to deduplication, offer additional
+features or make trade-offs. The following table lists tools that are known to
+be up-to-date, maintained and widely used.
+
+.. list-table::
+   :header-rows: 1
+
+   * - Name
+     - File based
+     - Block based
+     - Incremental
+   * - `BEES <https://github.com/Zygo/bees>`_
+     - No
+     - Yes
+     - Yes
+   * - `duperemove <https://github.com/markfasheh/duperemove>`_
+     - Yes
+     - No
+     - Yes
+
+Legend:
+
+- *File based*: the tool takes a list of files and deduplicates blocks only from that set
+- *Block based*: the tool enumerates blocks and looks for duplicates
+- *Incremental*: repeated runs of the tool utilizes information gathered from previous runs

Documentation/Defragmentation.rst

@@ -1,4 +1,22 @@
 Defragmentation
 ===============
 
-...
+Defragmentation of files is supposed to make the layout of the file extents to
+be more linear or at least coalesce the file extents into larger ones that can
+be stored on the device more efficiently. The reason there's a need for
+defragmentation stems from the COW design that BTRFS is built on and is
+inherent. The fragmentation is caused by rewrites of the same file data
+in-place, that has to be handled by creating a new copy that may lie on a
+distant location on the physical device. Fragmentation is the worst problem on
+rotational hard disks due to the delay caused by moving the drive heads to the
+distant location. With the modern seek-less devices it's not a problem though
+it may still make sense because of reduced size of the metadata that's needed
+to track the scattered extents.
+
+File data that are in use can be safely defragmented because the whole process
+happens inside the page cache, that is the central point caching the file data
+and takes care of synchronization. Once a filesystem sync or flush is started
+(either manually or automatically) all the dirty data get written to the
+devices. This however reduces the chances to find optimal layout as the writes
+happen together with other data and the result depens on the remaining free
+space layout and fragmentation.

Documentation/Flexibility.rst

@@ -1,6 +1,18 @@
 Flexibility
 ===========
 
-* dynamic inode creation (no preallocated space)
+The underlying design of BTRFS data structures allows a lot of flexibility and
+making changes after filesystem creation, like resizing, adding/removing space
+or enabling some features on-the-fly.
 
-* block group profile change on-the-fly
+* **dynamic inode creation** -- there's no fixed space or tables for tracking
+  inodes so the number of inodes that can be created is bounded by the metadata
+  space and it's utilization
+
+* **block group profile change on-the-fly** -- the block group profiles can be
+  changed on a mounted filesystem by running the balance operation and
+  specifying the conversion filters
+
+* **resize** -- the space occupied by the filesystem on each device can be
+  resized up (grow) or down (shrink) as long as the amount of data can be still
+  contained on the device

Documentation/Qgroups.rst

@@ -1,4 +1,4 @@
 Quota groups
 ============
 
-...
+.. include:: ch-quota-intro.rst

Documentation/Reflink.rst

@@ -1,4 +1,29 @@
 Reflink
 =======
 
-...
+Reflink is a type of shallow copy of file data that shares the blocks but
+otherwise the files are independent and any change to the file will not affect
+the other. This builds on the underlying COW mechanism. A reflink will
+effectively create only a separate metadata pointing to the shared blocks which
+is typically much faster than a deep copy of all blocks.
+
+The reflink is typically meant for whole files but a partial file range can be
+also copied, though there are no ready-made tools for that.
+
+.. code-block:: shell
+
+   cp --reflink=always source target
+
+There are some constaints:
+
+- cross-filesystem reflink is not possible, there's nothing in common between
+  so the block sharing can't work
+- reflink crossing two mount points of the same filesystem does not work due
+  to an artificial limitation in VFS (this may change in the future)
+- reflink requires source and target file that have the same status regarding
+  NOCOW and checksums, for example if the source file is NOCOW (once created
+  with the chattr +C attribute) then the above command won't work unless the
+  target file is pre-created with the +C attribute as well, or the NOCOW
+  attribute is inherited from the parent directory (chattr +C on the directory)
+  or if the whole filesystem is mounted with *-o nodatacow* that would create
+  the NOCOW files by default

Documentation/Resize.rst

@@ -1,4 +1,12 @@
 Resize
 ======
 
-...
+A BTRFS mounted filesystem can be resized after creation, grown or shrunk. On a
+multi device filesystem the space occupied on each device can be resized
+independently. Data tha reside in the are that would be out of the new size are
+relocated to the remaining space below the limit, so this constrains the
+minimum size to which a filesystem can be shrunk.
+
+Growing a filesystem is quick as it only needs to take note of the available
+space, while shrinking a filesystem needs to relocate potentially lots of data
+and this is IO intense. It is possible to shrink a filesystem in smaller steps.

Documentation/Scrub.rst

@@ -1,4 +1,4 @@
 Scrub
 =====
 
-...
+.. include:: ch-scrub-intro.rst

Documentation/Send-receive.rst

@@ -1,4 +1,23 @@
-Balance
-=======
+Send/receive
+============
 
-...
+Send and receive are complementary features that allow to transfer data from
+one filesystem to another in a streamable format. The send part traverses a
+given read-only subvolume and either creates a full stream representation of
+its data and metadata (*full mode*), or given a set of subvolumes for reference
+it generates a difference relative to that set (*incremental mode*).
+
+Receive on the other hand takes the stream and reconstructs a subvolume with
+files and directories equivalent to the filesystem that was used to produce the
+stream. The result is not exactly 1:1, eg. inode numbers can be different and
+other unique identifiers can be different (like the subvolume UUIDs). The full
+mode starts with an empty subvolume, creates all the files and then turns the
+subvolume to read-only. At this point it could be used as a starting point for a
+future incremental send stream, provided it would be generated from the same
+source subvolume on the other filesystem.
+
+The stream is a sequence of encoded commands that change eg. file metadata
+(owner, permissions, extended attributes), data extents (create, clone,
+truncate), whole file operations (rename, delete). The stream can be sent over
+network, piped directly to the receive command or saved to a file. Each command
+in the stream is protected by a CRC32C checksum.