fix: order mesh public/channel timeline by source packet timestamp (#525)

[gunjanjaswal]

Incoming public and channel messages were added to the timeline in receive order. When a peer reconnects and store-forwards or gossip-syncs an older backlog, those hour-old messages ended up at the bottom of the list, interleaved with current ones.

This is parity with iOS, which orders these timelines the same way: ConversationStore.insert(_:) does an upper-bound binary search with a <= comparison so equal timestamps keep arrival order. insertByTimestamp is that algorithm. Android was the one appending in receive order.

Each BitchatMessage already carries the source packet timestamp, so the fix inserts each message at its timestamp position instead of appending. It's applied in the AppStateStore add paths (which the UI timeline reads from) and the matching MessageManager paths, for the public and channel timelines.

insertByTimestamp keeps a fast path, also matching iOS: a message at or after the tail is appended in O(1), and only an out-of-order message pays for the binary search. In-order arrival is the common case, so the hot path is unchanged.

On private messages

An earlier version of this PR also ordered DMs by timestamp. While it sat in review, main gained PrivateMessageArrivalOrder (with ContactDirectory.canonicalizePrivateChats), which deliberately orders DMs by arrival sequence rather than sender timestamp — its own comment explains why: "Peer-provided timestamps cannot safely order a conversation because clocks can differ."

That's a deliberate decision, and it's in direct tension with timestamp-ordering DMs, so I've dropped that part: this PR now covers public and channel only, and leaves DM ordering to that arrival-order path (canonicalizePrivateChats would re-sort a timestamp insert straight back anyway). Happy to revisit if the maintainers decide DMs should be timestamp-ordered after all.

Tests

MessageOrderingTest covers out-of-order backlog insertion (public and channel), dedup, stable equal-timestamp ordering, and a mixed sequence that exercises both the fast-path append and the binary-search branch.

Fixes #525. Same underlying problem as #425, #420, and #302.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9aedd06dfb

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Normalize remote timestamps before ordering timelines

When a connected peer's clock is slow—MessageHandler.handleAnnounce explicitly accepts up to ten minutes of clock skew—a live message is inserted minutes above the end of a busy public or channel timeline. MessagesList reacts to the size change by scrolling to index 0 (the timestamp-latest item), so the newly received message can remain off-screen despite the haptic/notification; a fast peer can similarly pin future-dated messages at the bottom. Clamp or normalize remote timestamps, or apply source-timestamp insertion only to identified replay/backlog messages.

Useful? React with 👍 / 👎.

[habibgurdov77-star]

Fixes #739

[habibgurdov77-star]

Run

[gunjanjaswal]

Good flag on clock skew. It is a real consequence of ordering by the source packet timestamp: a peer whose clock runs slow (we accept up to ~10 min skew on announces) can land a live message a few minutes back in the list instead of at the bottom.

I left it as-is because ordering by packet timestamp is what #525 asks for, and it matches the sortBy { it.timestamp } that used to live in MessageRetentionService before it was commented out, so skew handling felt like a separate change. If you would rather not trust remote clocks that far, an easy guard is to clamp a future-dated timestamp to now (or fall back to receive time past a threshold). Happy to fold that in here if you want it.

[callebtc]

Android UI verification

Code-level review identifies a visible message-timeline ordering behavior change.

Before	After
Screenshot capture unavailable in this environment.	Screenshot capture unavailable in this environment.

Expected comparison: the same synthetic messages displayed in prior insertion order versus source-packet timestamp order.

Limitation: the required Java runtime, Android SDK, emulator, and adb are unavailable, so no before/after pixels could be captured or verified.

[gunjanjaswal]

Thanks for running this. Since there's no emulator in the loop here, a quick note on what to look for by hand, because this one won't show up as a pixel diff.

Nothing about the styling or layout changes. The only thing that moves is the order messages land in. To see it: let a peer drop off a channel for a while as messages keep flowing, then bring it back so it store-forwards its backlog. Before this change that hour-old backlog appended to the bottom, interleaved with current messages. After it, each message slots in at its source-packet timestamp, so the backlog drops into its correct chronological place and the tail stays "now". Messages that share a timestamp keep their arrival order (the insertion is a stable binary search), so nothing reshuffles under normal live use.

So there isn't really a before/after screenshot to capture, it's purely timeline ordering, which the insertion logic handles deterministically.

[Chessing234]

this already matches ios, which is worth putting in the body — it makes the change parity rather than a proposed behaviour change.

ConversationStore.insert(_:) does the same thing: upper-bound binary search, <= comparison so equal timestamps keep arrival order. your insertByTimestamp is the same algorithm line for line. i'd cite it; "ios has ordered timelines this way all along and android appends in receive order" is a much easier thing to merge than "here is a new ordering policy".

two differences worth acting on.

ios applies it to direct conversations too. ConversationStore.append(_:to:) routes every ConversationID — including .direct — through that same insert. this PR deliberately leaves private chats on append order, so the bug you're fixing stays for DMs on android, and the two platforms would still disagree there. a store-forwarded DM backlog is arguably the more likely case, since that's what the courier path exists for.

ios keeps a fast path. it only binary-searches when the incoming message is older than the tail:

if let last = messages.last, message.timestamp < last.timestamp { ...binary search... }
else { messages.append(message) }

in-order arrival is the overwhelmingly common case and it stays O(1) there. cheap to add and it keeps the hot path identical to what it is today, which also shrinks the surface of this change.

the dedup-unchanged and stable-equal-timestamp tests are the right things to have pinned.

[gunjanjaswal]

All three landed — thanks, these were good calls.

• Fast path: insertByTimestamp now appends in O(1) when the message is at or after the tail, and only binary-searches when it predates the tail. Hot path is unchanged.
• DMs: routed private chats through the same insert, in both MessageManager (addPrivateMessage/…NoUnread) and AppStateStore.addPrivateMessage. You're right that the store-forwarded DM backlog is the case that most needed it.
• Framing: rewrote the description around iOS parity with ConversationStore.insert rather than a new policy.

Added tests for the DM backlog ordering and a mixed sequence that hits both the append and binary-search branches. I don't have an Android SDK on this machine so I couldn't run the suite locally, but the JUnit tests will run in CI.

[gunjanjaswal]

Following up — rebasing this on main turned up a wrinkle worth surfacing, on the DM point specifically.

Since your review, main landed PrivateMessageArrivalOrder (plus ContactDirectory.canonicalizePrivateChats), which deliberately orders DMs by arrival sequence instead of sender timestamp. Its own comment makes the case: "Peer-provided timestamps cannot safely order a conversation because clocks can differ." So the project has since made the opposite call for DMs than the iOS-parity one — and canonicalizePrivateChats runs on every DM write, so a timestamp insert would just get re-sorted straight back to arrival order.

Rather than fight that, I've pulled the DM ordering back out. The PR now covers public and channel only (where timestamp ordering is uncontested and matches iOS), and leaves DMs to the arrival-order path. If the maintainers would rather DMs were timestamp-ordered after all, that's really a conversation about PrivateMessageArrivalOrder, not something to slip in here.

The fast path you suggested stayed in — that one's a clean win regardless. Thanks again for the review; it made the public/channel side better and flagged the DM question that turned out to already have an answer upstream.