Skip to content

HDFS-17794 Documentation for dfs.datanode.synconclose #8065

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: trunk
Choose a base branch
from
+15 −0
Open

HDFS-17794 Documentation for dfs.datanode.synconclose #8065

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
d2f4b7e
Added documentation for dfs.datanode.synconclose to hdfs-default.xml
Jimvin May 27, 2025
5617e8d
Merge pull request #1 from Jimvin/synconclose-docs-patch
Jimvin May 27, 2025
019a3da
Merge branch 'apache:trunk' into trunk
Jimvin Nov 7, 2025
56b344e
HDFS-17854 Fix for Namenode Web UI file deletion bug
Nov 7, 2025
62767df
Revert "HDFS-17854 Fix for Namenode Web UI file deletion bug"
Nov 7, 2025
62bb759
Update hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-defaul…
Jimvin Nov 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1779,6 +1779,21 @@
</description>
</property>

<property>
<name>dfs.datanode.synconclose</name>
<value>false</value>
<description>
If this configuration is enabled, the datanode will instruct the
Copy link

@dlmarion dlmarion Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If this configuration is enabled, the datanode will instruct the
If this property is enabled, the datanode will instruct the

operating system to sync data block files contents to disk when the
blockfile is closed.
Copy link

@dlmarion dlmarion Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
blockfile is closed.
blockfile is closed. This can help data loss in the event of a malfunction such as a power
failure.


This has the benefit of ensuring that data is written to disk
immediately when a block is closed, avoiding the data being held
in the operating system cache. This can help data loss in the event
of a malfunction such as a power failure.
Comment on lines +1790 to +1793
Copy link

@dlmarion dlmarion Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

..., avoiding the data being held in the operating system cache.

Does the sync of the dirty block in the page cache force the removal of the block from the page cache? I'm not sure that it does. The user can advise the OS, where supported, that it doesn't need the block in the page cache through the dfs.datanode.drop.cache.behind.writes property or related flags when opening the file.

</description>
</property>

<property>
<name>dfs.client.failover.max.attempts</name>
<value>15</value>
Expand Down