[epic]: Payment and Router separation

[yyforyongyu]

paymentLifecyle is the main struct to handle sending payments and can be moved out of routing package, which has the benefits,

• indepedent package is easier to analyze/maintain, e.g., we can easily increase logging verbosity for this subservice.
• fixed an interface violation which can cause a race condition when sending HTLCs.

Current State

As of today, multiple components are involved when sending payments, as shown below,

flowchart LR
		pay_req-->paymentLifecycle
		
		subgraph paymentLifecycle
	    p[resumePayment]
    end
    
    subgraph PaymentSession
	    RequestRoute-->p
	    p-->UpdateAdditionalEdge
	    p-->GetAdditionalEdgePolicy
    end
    
    subgraph MissionControl
      p-->ReportPaymentSuccess
      p-->ReportPaymentFailure
    end
    
    subgraph PaymentAttemptDispatcher
    	p-->SendHTLC
    	GetAttemptResult-->p
    end
    
    subgraph ControlTower
    	crud[DB operations]-->p
    	FetchPayment
    	FailPayment
    	more[...]
    end
    
    subgraph ChannelRouter
        p--->ChannelRouter.applyChannelUpdate
    end

Loading

The components are,

• ControlTower: handles interaction with DB(CRUD)
• MissionControl: records success probablity
• PaymentSession(paymentSession): requests routes and updates edges/policies from private channels
• PaymentAttemptDispatcher: interacts with Switch to send HTLC and get HTLC attempt results, which is an interface violation.
• ChannelRouter.applyChannelUpdate: updates edges using failed HTLCs

Proposed Design

The proposed architecture,

flowchart LR
    pay_req-->paymentLifecycle
    
    subgraph paymentLifecycle
      p[resumePayment]
      ct
    end

    subgraph ct[ControlTower]
      crud[DB operations]
      FetchPayment
      FailPayment
      more[...]
    end
    
    subgraph cg[ChannelRouter]
      p-->sa[SendAttempt]
      ps
      mc
      pad
    end
    
    subgraph ps[PaymentSession]
      RequestRoute
      UpdateAdditionalEdge
      GetAdditionalEdgePolicy
    end
    
    subgraph mc[MissionControl]
      ReportPaymentSuccess
      ReportPaymentFailure
    end
    
    subgraph pad[PaymentAttemptDispatcher]
      SendHTLC
      GetHTLCResult
    end

Loading

Some highlights,

• Payment lifecycle is in its own package (payment) and interacts with the third layer ChannelRouter.
• The only interface method used by paymentLifecycle is ChannelRouter.SendAttempt, which takes an attempt request specifying params like routes, onions, amt, etc.
• "HTLC" is a concept used in htlcswitch (layer-two), and "HTLC attempt" or "Attempt" (or other better names?) is a concept used in routing (layer-three).

By adding SendAttempt, htlcswitch is no longer exposed to paymentLifecycle. Instead, an HTLC attempt always goes through the graph managed by ChannelRouter first, and it's up to ChannelRouter to decide whether a route can be used or not, which solves the race condition when two HTLC attempts are using the same route without being aware of each other, as manifested in itest flakes.

Plan

• Create a new package and move paymentLifecycle out of routing.
• Add interface method SendAttempt, implement it in routing and use it in paymentLifecycle.

[ziggie1984]

classDiagram
    class payment_control {
        -payment_db
    }
    
    class control_tower {
        -payment_control
        +subscriber_notification_logic()
    }
    
    class ChanRouter {
        <<routing package>>
        +config
        -config.control_tower
        +sendPayment()
        +sendToRoute()
    }
    
    class PaymentLifeCycle {
        <<routing package>>
    }
    
    class switch {
        <<package>>
    }
    
    class routerrpc {
        <<package>>
        +exposes_control_tower()
        +external_payment_subscriptions()
    }
    
    class rpcserver {
        <<package>>
        +QueryPayments()
        +DeletePayments()
    }
    
    class payment_db {
        +store_payment()
        +retrieve_payment()
        +update_payment()
    }
    
    class key_value_store {
        <<interface>>
        +get(key)
        +put(key, value)
        +delete(key)
    }
    
    control_tower *-- payment_control : wraps
    ChanRouter *-- control_tower : uses
    PaymentLifeCycle --> ChanRouter : uses for control_tower access
    PaymentLifeCycle --> switch : uses
    routerrpc ..> control_tower : exposes for external callers
    payment_control --> payment_db : accesses
    rpcserver --> payment_db : accesses
    payment_db ..|> key_value_store : implements

Loading

[ziggie1984]

This epic is partly affected by the Payment SQLization, where we refactor everything below (including) the Control_Tower and moving it into a dedicated package