Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix a rare CME in LatencyUtils #1984

Open
wants to merge 1 commit into
base: 2.0
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 36 additions & 6 deletions src/main/java/ac/grim/grimac/utils/latency/LatencyUtils.java
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,18 @@
import ac.grim.grimac.utils.data.Pair;
import com.github.retrooper.packetevents.netty.channel.ChannelHelper;

import java.util.ArrayList;
import java.util.LinkedList;
import java.util.ListIterator;

public class LatencyUtils {
private final LinkedList<Pair<Integer, Runnable>> transactionMap = new LinkedList<>();
private final GrimPlayer player;

// Built from transactionMap and cleared at start of every handleNettySyncTransaction() call
// The actual usage scope of this variable's use is limited to within the synchronized block of handleNettySyncTransaction
private final ArrayList<Runnable> tasksToRun = new ArrayList<>();

public LatencyUtils(GrimPlayer player) {
this.player = player;
}
Expand Down Expand Up @@ -41,30 +46,55 @@ public void addRealTimeTask(int transaction, boolean async, Runnable runnable) {
}

public void handleNettySyncTransaction(int transaction) {
/*
* This code uses a two-pass approach within the synchronized block to prevent CMEs.
* First we collect and remove tasks using the iterator, then execute all collected tasks.
*
* The issue:
* We cannot execute tasks during iteration because if a runnable modifies transactionMap
* or calls addRealTimeTask, it will cause a ConcurrentModificationException.
* While only seen on Folia servers, this is theoretically possible everywhere.
*
* Why this solution:
* Rather than documenting "don't modify transactionMap in runnables" and risking subtle
* bugs from future contributions or Check API usage, we prevent the issue entirely
* at a small performance cost.
*
* Future considerations:
* If this becomes a performance bottleneck, we may revisit using a single-pass approach
* on non-Folia servers. We could also explore concurrent data structures or parallel
* execution, but this would lose the guarantee that transactions are processed in order.
Comment on lines +63 to +66

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An alternative solution if this is a performance issue, is to do the queuing on the add method instead.

If any of the add task methods are called while we're iterating, add them to a list, and once the iteration is done add everything from that list to the transaction map.
If everything is working ideally the list will always be empty and be a non-issue, if something breaks the list may see some use

*/
synchronized (this) {
for (ListIterator<Pair<Integer, Runnable>> iterator = transactionMap.listIterator(); iterator.hasNext(); ) {
tasksToRun.clear();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why does this happen first, and not last? This is preventing it being a local variable. Am I missing something?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could totally be a local variable, but by having it non-local you save on the memory allocation overhead and just use already allocated memory
at least that's my assumption of why its done this way

Copy link
Contributor Author

@Axionize Axionize Feb 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its a very minor optimization that saves you the cost of having to call new on every iteration (that is allocate and deallocate the ArrayList), in addition since its an ArrayList the underlying list will never be shrunk, meaning fewer O(n) resizes than if we used a local variable. If you log the size of the list you'll see that most of the time its 1 and occasionally it'll spike to 50 and every so often a few hundred.

In the case where a few hundred entries are added several resizes are required under the hood which costs time and allocations we save by clearing instead.

This was partially explained in my original PR post

Added a class-level ArrayList to store tasks between passes, which is cleared and reused each call to avoid unnecessary allocations.

  • Using ArrayList for tasksToRun as it only needs sequential adds and iteration
  • Keeping it as a class field to reuse the backing array


// First pass: collect tasks and mark them for removal
ListIterator<Pair<Integer, Runnable>> iterator = transactionMap.listIterator();
while (iterator.hasNext()) {
Pair<Integer, Runnable> pair = iterator.next();

// We are at most a tick ahead when running tasks based on transactions, meaning this is too far
if (transaction + 1 < pair.first())
return;
break;

// This is at most tick ahead of what we want
if (transaction == pair.first() - 1)
continue;

tasksToRun.add(pair.second());
iterator.remove();
}

for (Runnable runnable : tasksToRun) {
try {
// Run the task
pair.second().run();
runnable.run();
} catch (Exception e) {
LogUtil.error("An error has occurred when running transactions for player: " + player.user.getName(), e);
// Kick the player SO PEOPLE ACTUALLY REPORT PROBLEMS AND KNOW WHEN THEY HAPPEN
if (!Boolean.getBoolean("grim.disable-transaction-kick")) {
player.disconnect(MessageUtil.miniMessage(MessageUtil.replacePlaceholders(player, GrimAPI.INSTANCE.getConfigManager().getDisconnectPacketError())));
}
}
// We ran a task, remove it from the list
iterator.remove();
}
}
}
Expand Down