Skip to content

feat(followers): implement CRDT-based follower count to eliminate los… - #761

Open
Alaka-ibr wants to merge 1 commit into
accesslayerorg:mainfrom
Alaka-ibr:feat/crdt-follower-count-757
Open

feat(followers): implement CRDT-based follower count to eliminate los…#761
Alaka-ibr wants to merge 1 commit into
accesslayerorg:mainfrom
Alaka-ibr:feat/crdt-follower-count-757

Conversation

@Alaka-ibr

Copy link
Copy Markdown
Contributor

feat(followers): implement CRDT-based follower count to eliminate lost updates (#757)

Summary

Closes #757

Under concurrent follow and unfollow bursts for the same creator, direct integer increment/decrement updates suffer from lost updates and race conditions. This PR replaces the simple counter with a G-Counter CRDT structure per node, computing the true count as the sum of all increments minus decrements across all nodes at read time.

Changes

1. Database Schema (prisma/schema/follower.prisma)

  • FollowerCounterShard: creatorWallet, nodeId, increments (BigInt), decrements (BigInt), updatedAt. Unique key: (creatorWallet, nodeId).
  • FollowEvent: followerWallet, creatorWallet, direction (FOLLOW | UNFOLLOW), createdAt. Unique key: (followerWallet, creatorWallet).

2. Node Identification (src/config.schema.ts)

  • Added NODE_ID configuration parameter (defaults to node-local).

3. Atomic Follow/Unfollow & Count Resolution (src/modules/followers/follower.service.ts)

  • Follow & Unfollow: Idempotent event check. Performs database-level atomic shard increment (UPDATE ... SET increments = increments + 1) for local NODE_ID.
  • Count Resolution: getFollowerCount sums increments and decrements across all shards for creatorWallet, returns $\max(0, \sum \text{inc} - \sum \text{dec})$. Cached in Redis with a 10-second TTL (invalidated on write).
  • Atomic Compaction: compactShardsForCreator merges all shards for a creator into a single canonical shard in a single transaction (skips creators updated in the last 5 minutes).

4. Endpoints & Nightly Job (src/modules/followers/follower.controllers.ts, src/jobs/follower-shard-compaction.job.ts)

  • POST /api/v1/followers/:creatorWallet/follow
  • POST /api/v1/followers/:creatorWallet/unfollow
  • GET /api/v1/followers/:creatorWallet/count
  • Nightly compaction job runNightlyFollowerShardCompaction.

Test Coverage

Unit tests in src/modules/followers/follower.service.test.ts (5 passed):

  • 100 concurrent follows from different wallets produces count of exactly 100 with zero lost updates.
  • 50 concurrent follows and 30 concurrent unfollows produces net count of 20.
  • Double-follow from the same wallet is idempotent and increments count only once.
  • Resolves count correctly across multiple distinct node shards.
  • Nightly compaction merges shards atomically while preserving identical resolved count.

@Alaka-ibr
Alaka-ibr force-pushed the feat/crdt-follower-count-757 branch from 12a152a to 3eb6a5b Compare August 24, 2026 17:05
…t updates (accesslayerorg#757)

Replace monolithic follower count with a per-node G-Counter CRDT structure
to prevent lost updates under high concurrency.

- FollowerCounterShard and FollowEvent Prisma schema definitions
- NODE_ID configuration option in config schema
- Follow and unfollow operations using atomic shard increments and idempotent events
- getFollowerCount summing across all node shards with 10s Redis caching
- Nightly shard compaction merging per-node shards atomically
- Unit tests verifying 100 concurrent follows, 50 follows + 30 unfollows,
  double-follow idempotency, multi-node resolution, and compaction
@Alaka-ibr
Alaka-ibr force-pushed the feat/crdt-follower-count-757 branch from 3eb6a5b to f110e78 Compare August 24, 2026 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement a conflict-free replicated data type (CRDT) based follower count to eliminate race conditions under concurrent follow/unfollow bursts

1 participant