Skip to content

Latest commit

 

History

History
661 lines (488 loc) · 19.7 KB

REPUTATION.md

File metadata and controls

661 lines (488 loc) · 19.7 KB

Node Reputation System

Complete guide to the Node Reputation System in XDAGJ-P2P.

Table of Contents


Overview

The Node Reputation System is a credit scoring mechanism for P2P nodes, similar to credit scores in financial systems. It tracks the reliability and performance of each node in the network, helping to:

  • Automatically identify reliable nodes - Prioritize connections to high-reputation nodes
  • Filter out unreliable nodes - Avoid wasting resources on unresponsive peers
  • Maintain network quality - Improve overall network health and performance
  • Persist across restarts - Remember node behavior history

Key Features

✅ Persistent Storage - Reputation survives node restarts ✅ Automatic Decay - Gives nodes a chance to improve over time ✅ Thread-Safe - Safe for concurrent access ✅ Atomic Operations - Backup and recovery mechanisms ✅ Zero Dependencies - Self-contained implementation


How It Works

Reputation Score Range

Score Range: 0 - 200
Neutral:     100 (initial value for new nodes)
Good Nodes:  > 100
Bad Nodes:   < 100
Dead Nodes:  < 20 (automatically marked as DEAD)

Score Updates

The reputation system integrates with Kademlia DHT node discovery:

Event Score Change Location
PONG received successfully +5 points NodeHandler.handlePong()
PING timeout -5 points NodeHandler.checkPingTimeout()
Node marked as DEAD Score < 20 NodeHandler.updateReputation()

Lifecycle

1. New Node Discovered
   └─> Initialize with score 100 (neutral)

2. Node Responds to PING
   └─> Score increases (+5 per successful PONG)

3. Node Times Out
   └─> Score decreases (-5 per timeout)

4. Node Becomes Unreliable (score < 20)
   └─> Marked as DEAD, excluded from routing table

5. Node Goes Offline (no updates)
   └─> Score decays towards neutral (100) over time

Architecture

Components

┌─────────────────────────────────────────────────────────────┐
│                      KadService                              │
│  ┌────────────────────────────────────────────────────────┐ │
│  │           ReputationManager                            │ │
│  │  ┌──────────────────────────────────────────────────┐ │ │
│  │  │  ConcurrentHashMap<String, ReputationData>       │ │ │
│  │  │  - nodeId → (score, timestamp)                   │ │ │
│  │  └──────────────────────────────────────────────────┘ │ │
│  │                                                        │ │
│  │  ┌──────────────────────────────────────────────────┐ │ │
│  │  │  Persistence Layer                               │ │ │
│  │  │  - reputation.dat (main file)                    │ │ │
│  │  │  - reputation.dat.bak (backup)                   │ │ │
│  │  │  - Atomic file operations                        │ │ │
│  │  └──────────────────────────────────────────────────┘ │ │
│  └────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
         ↑                                      ↑
         │ setReputation()                      │ getReputation()
         │ (on PONG/timeout)                    │ (on startup)
         │                                      │
┌────────┴──────────┐                 ┌────────┴──────────┐
│   NodeHandler     │                 │   NodeHandler     │
│  (Update scores)  │                 │  (Load history)   │
└───────────────────┘                 └───────────────────┘

Data Structure

private static class ReputationData implements Serializable {
    private final int score;        // Reputation score (0-200)
    private final long timestamp;   // Last update time (milliseconds)

    public int getDecayedScore() {
        // Calculate decay based on time elapsed
        long daysSinceUpdate = (now - timestamp) / (24 * 60 * 60 * 1000);
        int decay = daysSinceUpdate * 5;  // 5 points per day

        if (score > 100) {
            return max(100, score - decay);  // Good nodes decay down
        } else if (score < 100) {
            return min(100, score + decay);  // Bad nodes recover up
        }
        return 100;
    }
}

File Storage

Location: data/reputation/reputation.dat

Format: Java serialized ConcurrentHashMap<String, ReputationData>

Backup: data/reputation/reputation.dat.bak (created on each save)

Save Strategy:

  1. Write to temporary file (reputation.tmp)
  2. Backup existing file (reputation.dat → reputation.dat.bak)
  3. Atomically replace (reputation.tmp → reputation.dat)

This ensures data integrity even if the process crashes during save.


Configuration

Default Settings

// In ReputationManager.java
private static final long DEFAULT_SAVE_INTERVAL_MS = 60_000;  // 1 minute
private static final int DEFAULT_INITIAL_REPUTATION = 100;    // Neutral score

// In ReputationData
private static final long DECAY_INTERVAL_MS = 86_400_000;     // 1 day
private static final int DECAY_AMOUNT = 5;                    // Points per day
private static final int NEUTRAL_SCORE = 100;                 // Target score

Custom Configuration

// In KadService initialization
String reputationDir = p2pConfig.getDataDir() != null
    ? p2pConfig.getDataDir() + "/reputation"
    : "data/reputation";

// Create with custom save interval (e.g., 30 seconds)
ReputationManager reputationManager = new ReputationManager(reputationDir, 30_000);

P2pConfig Settings

P2pConfig config = new P2pConfig();
config.setDataDir("custom/data");  // Reputation stored in custom/data/reputation/

Usage Examples

Example 1: Node Discovery Workflow

// Day 1: Node joins network
NodeHandler handler = new NodeHandler(kadService, node);
// Initial reputation: 100 (loaded from ReputationManager)

// Node responds well to PINGs
handler.handlePong(pongMessage);  // +5 points
handler.handlePong(pongMessage);  // +5 points
handler.handlePong(pongMessage);  // +5 points
// Current reputation: 115

// Later, node times out
handler.checkPingTimeout();       // -5 points
// Current reputation: 110

// Reputation is automatically saved to disk every 60 seconds

Example 2: Restart Recovery

// Node restarts - reputation is automatically persisted and restored
P2pConfig config = new P2pConfig();
config.setDataDir("data");  // Reputation stored in data/reputation/
config.addP2pEventHandle(new MyEventHandler());

P2pService p2pService = new P2pService(config);
p2pService.start();

// ReputationManager automatically loads previous scores during initialization
// All nodes' reputation scores are restored from disk automatically
// Node with ID "abc123" had reputation 115 before restart, 
// will be restored (possibly with slight decay)

Note: Reputation persistence is handled automatically by the internal ReputationManager. The reputation scores are loaded when P2pService.start() is called and the discovery service initializes.

Example 3: Understanding Reputation Behavior

The reputation system works automatically - you don't need to manually access ReputationManager. However, understanding how it works helps:

// Reputation is tracked internally per node ID
// When a node connects and behaves well, reputation increases
// When a node misbehaves or disconnects abnormally, reputation decreases

// The system automatically:
// 1. Loads reputation from disk on startup
// 2. Updates reputation based on node behavior
// 3. Saves reputation to disk periodically (every 60 seconds)
// 4. Applies decay over time (reputation decreases by 1 per hour)

// Reputation affects connection decisions:
// - Low reputation nodes are less likely to be connected
// - High reputation nodes are prioritized
// - Banned nodes cannot connect until ban expires

Reputation Decay

Why Decay?

Reputation decay gives nodes a "second chance" - prevents permanent blacklisting due to temporary issues (network problems, maintenance, etc.).

Decay Mechanism

Decay Rate: 5 points per day Decay Direction: Always towards neutral (100)

Example Timeline:

Day Good Node (150) Bad Node (50) Neutral Node (100)
0 150 50 100
1 145 55 100
5 125 75 100
10 100 ✓ 100 ✓ 100
30 100 100 100

Key Points:

  • Good nodes (score > 100) decay down to 100
  • Bad nodes (score < 100) decay up to 100
  • Neutral nodes (score = 100) don't decay
  • Decay stops at neutral score (100)

Implementation

public int getDecayedScore() {
    long ageMs = System.currentTimeMillis() - timestamp;
    long daysSinceUpdate = ageMs / 86_400_000;  // milliseconds per day

    if (daysSinceUpdate == 0) {
        return score;  // No decay within first day
    }

    int totalDecay = (int) (daysSinceUpdate * 5);

    if (score > 100) {
        return Math.max(100, score - totalDecay);  // Decay down
    } else if (score < 100) {
        return Math.min(100, score + totalDecay);  // Recover up
    }

    return 100;
}

Best Practices

1. Let the System Work Automatically

✅ Recommended:

// Let NodeHandler manage reputation automatically
// No manual intervention needed in production

❌ Not Recommended:

// Don't manually adjust scores unless for testing/debugging
reputationManager.setReputation(nodeId, 200);

2. Monitor Reputation Distribution

// Periodically check reputation statistics
Map<String, Integer> reputationStats = new HashMap<>();
reputationStats.put("excellent", 0);  // > 150
reputationStats.put("good", 0);       // 120-150
reputationStats.put("neutral", 0);    // 80-120
reputationStats.put("poor", 0);       // 50-80
reputationStats.put("bad", 0);        // < 50

// Classify nodes by reputation
for (String nodeId : allNodeIds) {
    int rep = reputationManager.getReputation(nodeId);
    if (rep > 150) reputationStats.put("excellent", reputationStats.get("excellent") + 1);
    else if (rep > 120) reputationStats.put("good", reputationStats.get("good") + 1);
    // ... etc
}

3. Backup Reputation Data

The system creates automatic backups, but for production:

# Periodic backup of reputation data
cp data/reputation/reputation.dat backups/reputation-$(date +%Y%m%d).dat

# Restore from backup if needed
cp backups/reputation-20251021.dat data/reputation/reputation.dat

4. Handle Node ID Changes

If node IDs change (e.g., key rotation), reputation data will be lost for those nodes. Consider:

// Migration helper (if needed)
public void migrateNodeId(String oldId, String newId) {
    int oldReputation = reputationManager.getReputation(oldId);
    reputationManager.setReputation(newId, oldReputation);
    // Note: Old ID will naturally decay to neutral over time
}

5. Configure Appropriate Data Directory

// Production: Use absolute path
config.setDataDir("/var/lib/xdagj-p2p/data");

// Development: Use relative path
config.setDataDir("data");

// Ensure directory has write permissions

Monitoring

Log Messages

Startup:

[INFO] ReputationManager started: file=data/reputation/reputation.dat, saveInterval=60000ms
[INFO] Loaded 42 node reputations from data/reputation/reputation.dat

During Operation:

[DEBUG] Saved 42 node reputations to data/reputation/reputation.dat
[INFO] Loaded reputation 115 for node 192.168.1.100:16783

Shutdown:

[INFO] Stopping ReputationManager
[DEBUG] Saved 42 node reputations to data/reputation/reputation.dat

Metrics to Track

  1. Total Nodes Tracked: reputationManager.size()
  2. Average Reputation: Calculate from all tracked nodes
  3. Low Reputation Nodes: Count nodes with score < 50
  4. High Reputation Nodes: Count nodes with score > 150
  5. File Size: Monitor reputation.dat file size growth

Health Checks

// Check if reputation system is working
public boolean isReputationSystemHealthy() {
    // 1. Check if manager is initialized
    if (reputationManager == null) return false;

    // 2. Check if file exists and is readable
    Path repFile = Paths.get("data/reputation/reputation.dat");
    if (Files.exists(repFile) && Files.isReadable(repFile)) {
        return true;
    }

    // 3. If no file yet, that's ok (first run)
    return reputationManager.size() >= 0;
}

Troubleshooting

Issue 1: Reputation Data Not Persisting

Symptoms:

  • Reputation resets to 100 after restart
  • No reputation.dat file created

Diagnosis:

# Check if directory exists and is writable
ls -la data/reputation/
# Should show: drwxr-xr-x ... reputation

# Check logs for errors
grep "ReputationManager" logs/xdagj-p2p.log

Solutions:

# Create directory with correct permissions
mkdir -p data/reputation
chmod 755 data/reputation

# Check disk space
df -h data/

# Verify no permission issues
touch data/reputation/test.txt && rm data/reputation/test.txt

Issue 2: Reputation File Corrupted

Symptoms:

[ERROR] Failed to load reputation data from data/reputation/reputation.dat
java.io.StreamCorruptedException: invalid stream header

Solutions:

# Try loading from backup
cp data/reputation/reputation.dat.bak data/reputation/reputation.dat

# If backup also corrupted, start fresh
mv data/reputation/reputation.dat data/reputation/reputation.dat.corrupted
# System will create new file automatically

Issue 3: All Nodes Have Same Reputation

Symptoms:

  • All nodes showing reputation 100
  • No score changes over time

Diagnosis:

// Check if NodeHandler is calling setReputation
// Add debug logging in NodeHandler:
log.debug("Updating reputation for {} from {} to {}",
    node.getId(), oldRep, newRep);

Common Causes:

  • Node IDs are null (check key generation)
  • ReputationManager not initialized in KadService
  • PING/PONG messages not being processed

Issue 4: High Memory Usage

Symptoms:

  • Memory grows over time
  • Many nodes tracked (thousands)

Solutions:

// Implement periodic cleanup of very old entries
public void cleanupOldEntries() {
    long cutoffTime = System.currentTimeMillis() - (30 * 86_400_000L); // 30 days

    reputations.entrySet().removeIf(entry -> {
        ReputationData data = entry.getValue();
        return data.getTimestamp() < cutoffTime &&
               data.getDecayedScore() == 100;  // Fully decayed
    });
}

Technical Details

Thread Safety

  • ConcurrentHashMap for node reputation storage
  • synchronized methods for load/save operations
  • Atomic file operations for persistence
  • Single-threaded executor for periodic saves

Performance

  • Memory: ~100 bytes per node (nodeId + ReputationData)
  • CPU: Minimal (<0.1% for 1000 nodes)
  • I/O: One write every 60 seconds + shutdown save
  • Lookup: O(1) - hash map access

Scalability

Tested:

  • 1,000 nodes: No issues
  • 10,000 nodes: ~1 MB memory, <1ms lookup
  • 100,000 nodes: ~10 MB memory, <1ms lookup

Limits:

  • Practical limit: ~1M nodes (~100 MB memory)
  • File size grows linearly with node count
  • Save time increases with node count

Integration with Kademlia DHT

Automatic Integration

The reputation system is automatically integrated with the Kademlia DHT discovery system. You don't need to write any code to enable this - it works automatically when you use P2pService.

How it works internally (implementation details, not part of public API):

  • When a node is discovered or connected, its reputation is automatically loaded from disk
  • Reputation is updated automatically based on node behavior (ping/pong responses, connection stability)
  • When reputation changes, it's automatically saved to disk by the ReputationManager
  • Low-reputation nodes are less likely to be selected for connections
  • High-reputation nodes are prioritized during peer selection

As a user, you simply:

  1. Configure dataDir in P2pConfig (where reputation files will be stored)
  2. Start P2pService - reputation tracking starts automatically
  3. The system handles everything else - loading, updating, saving

Decision Making

// Mark node as DEAD if reputation too low (NodeHandler.java)
if (getReputationScore() < 20) {
    node.setStateType(NodeStateType.DEAD);
    log.warn("Node {} marked as DEAD due to low reputation {}",
        node.getPreferInetSocketAddress(), getReputationScore());
}

API Reference

ReputationManager Class

Constructor

public ReputationManager(String dataDir)
public ReputationManager(String dataDir, long saveIntervalMs)

Public Methods

// Get reputation score for a node (with decay applied)
public int getReputation(String nodeId)

// Set reputation score for a node
public void setReputation(String nodeId, int score)

// Get number of tracked nodes
public int size()

// Clear all reputation data
public void clear()

// Stop manager and perform final save
public void stop()

Package-Private Methods

// Load reputation data from disk
synchronized void load()

// Save reputation data to disk
synchronized void save()

Testing

The reputation system has 20 comprehensive unit tests covering:

  • Basic CRUD operations
  • Persistence and backup/recovery
  • Time-based decay
  • Concurrent access (10 threads × 100 operations)
  • Edge cases (null IDs, empty strings, extreme values)
  • Integration scenarios

Run tests:

mvn test -Dtest=ReputationManagerTest

Test Results:

Tests run: 20, Failures: 0, Errors: 0, Skipped: 0
Time elapsed: ~3 seconds

See ReputationManagerTest.java for details.


References


Status: Production-ready | Version: v0.1.6 | Tests: 20/20 passing