Channel does not reconnect after recent ACINQ downtime

[Filouman]

After recent downtime/maintenance of the ACINQ node, my channel does not come back online.

My node:
03641a88d80a2a85bbecd770577aca9b5495616e9fef63d66ef2631b7cca1d395d

Running LND 0.19.3

I have tried to manually disconnect/reconnect the channel a number of times, but I am seeing unable to read message from peer: read next header: EOF errors in the logs each time LND attempts to bring the channel back online.

I'm wondering if it's in any way related to #3149 ?

This is the only channel currently offline on my node, and it was stable and routing regularly before the recent ACINQ downtime.

Logs from my side during reconnection attempt below:

Sep 09 18:46:10 nodelou lnd[1365]: 2025-09-09 18:46:10.341 [INF] HSWC: ChannelLink(3dc32fba5da71617fa30cb6736e92a9088da6b2aa53fc0bd9feb57a9f936e97e:4): Attempting to re-synchronize channel: SCID=870962:1423:4, status=ChanStatusDefault, initiator=true, pending=false, local commitment has height=122290, local_htlc_index=39277, local_log_index=63257, remote_htlc_index=21406, remote_log_index=60683, remote commitment has height=122257, local_htlc_index=39277, local_log_index=63257, remote_htlc_index=21406, remote_log_index=60683
Sep 09 18:46:10 nodelou lnd[1365]: 2025-09-09 18:46:10.341 [INF] PEER: Peer(03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f): unable to read message from peer: read next header: EOF
Sep 09 18:46:10 nodelou lnd[1365]: 2025-09-09 18:46:10.341 [DBG] PEER: Peer(03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f): Peer hasn't finished starting up yet, waiting on startReady signal before closing connection
Sep 09 18:46:10 nodelou lnd[1365]: 2025-09-09 18:46:10.341 [DBG] PEER: Peer(03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f): Sending NodeAnnouncement(node=03641a88d80a2a85bbecd770577aca9b5495616e9fef63d66ef2631b7cca1d395d, update_time=2025-09-09 18:46:10 +0000 UTC) to 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f@3.33.236.230:9735
Sep 09 18:46:10 nodelou lnd[1365]: 2025-09-09 18:46:10.341 [INF] PEER: Peer(03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f): disconnecting 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f@3.33.236.230:9735, reason: read handler closed
Sep 09 18:46:10 nodelou lnd[1365]: 2025-09-09 18:46:10.341 [DBG] PEER: Peer(03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f): Sending ChannelReestablish(chan_id=7ee936f9a957eb9fbdc03fa52a6bda88902ae93667cb30fa1716a75dba2fc339, next_local_height=122291, remote_tail_height=122257) to 03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f@3.33.236.230:9735
Sep 09 18:46:10 nodelou lnd[1365]: 2025-09-09 18:46:10.341 [WRN] HSWC: ChannelLink(3dc32fba5da71617fa30cb6736e92a9088da6b2aa53fc0bd9feb57a9f936e97e:4): error when syncing channel states: unable to send chan sync message for ChannelPoint(3dc32fba5da71617fa30cb6736e92a9088da6b2aa53fc0bd9feb57a9f936e97e:4): write tcp 10.8.0.2:55202->3.33.236.230:9735: use of closed network connection
Sep 09 18:46:10 nodelou lnd[1365]: 2025-09-09 18:46:10.341 [ERR] HSWC: ChannelLink(3dc32fba5da71617fa30cb6736e92a9088da6b2aa53fc0bd9feb57a9f936e97e:4): failing link: unable to synchronize channel states: unable to send chan sync message for ChannelPoint(3dc32fba5da71617fa30cb6736e92a9088da6b2aa53fc0bd9feb57a9f936e97e:): write tcp 10.8.0.2:55202->3.33.236.230:9735: use of closed network connection with error: unable to resume channel, recovery required
Sep 09 18:46:10 nodelou lnd[1365]: 2025-09-09 18:46:10.341 [INF] HSWC: ChannelLink(3dc32fba5da71617fa30cb6736e92a9088da6b2aa53fc0bd9feb57a9f936e97e:4): exited
Sep 09 18:46:10 nodelou lnd[1365]: 2025-09-09 18:46:10.341 [INF] HSWC: ChannelLink(3dc32fba5da71617fa30cb6736e92a9088da6b2aa53fc0bd9feb57a9f936e97e:4): stopping
Sep 09 18:46:10 nodelou lnd[1365]: 2025-09-09 18:46:10.341 [ERR] PEER: Peer(03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f): unable to send msg to remote peer: peer exiting

lncli listchannels:

{
    "channels": [
        {
            "active": false,
            "remote_pubkey": "03864ef025fde8fb587d989186ce6a4a186895ee44a926bfc370e2c366597a3f8f",
            "channel_point": "3dc32fba5da71617fa30cb6736e92a9088da6b2aa53fc0bd9feb57a9f936e97e:4",
            "chan_id": "7ee936f9a957eb9fbdc03fa52a6bda88902ae93667cb30fa1716a75dba2fc339",
            "scid": "957632846444298244",
            "scid_str": "870962x1423x4",
            "capacity": "25000000",
            "local_balance": "4575319",
            "remote_balance": "20423704",
            "commit_fee": "317",
            "commit_weight": "1116",
            "fee_per_kw": "282",
            "unsettled_balance": "0",
            "total_satoshis_sent": "311551532",
            "total_satoshis_received": "291127828",
            "num_updates": "122290",
            "pending_htlcs": [],
            "csv_delay": 720,
            "private": false,
            "initiator": true,
            "chan_status_flags": "ChanStatusDefault",
            "local_chan_reserve_sat": "250000",
            "remote_chan_reserve_sat": "250000",
            "static_remote_key": false,
            "commitment_type": "ANCHORS",
            "lifetime": "165776",
            "uptime": "105688",
            "close_address": "",
            "push_amount_sat": "0",
            "thaw_height": 0,
            "local_constraints": {
                "csv_delay": 720,
                "chan_reserve_sat": "250000",
                "dust_limit_sat": "354",
                "max_pending_amt_msat": "11250000000",
                "min_htlc_msat": "1",
                "max_accepted_htlcs": 30
            },
            "remote_constraints": {
                "csv_delay": 2016,
                "chan_reserve_sat": "250000",
                "dust_limit_sat": "546",
                "max_pending_amt_msat": "24750000000",
                "min_htlc_msat": "1",
                "max_accepted_htlcs": 483
            },
            "alias_scids": [],
            "zero_conf": false,
            "zero_conf_confirmed_scid": "0",
            "peer_alias": "ACINQ",
            "peer_scid_alias": "0",
            "memo": "",
            "custom_channel_data": ""
        }
    ]
}

Please let me know if I could provide any additional details to help solve the issue.

[t-bast]

The issue is that lnd has already activated the staging taproot feature bit (lightning/bolts#995) but doesn't activate the option_simple_close feature (which is needed because otherwise there is no way to properly cooperatively close a taproot channel), which should be required when using taproot. The specification for taproot channels is still a draft, and I'm surprised that this is activated by default on mainnet, it's quite expected that this can cause compatibility issues...

We will work around this by deploying #3161, which relaxes the requirements until the official taproot feature ships. Meanwhile, you can simply disable option_simple_taproot_staging on your node to be able to reconnect with our node.

[t-bast]

We've re-deployed our node with the work-around, this should be fixed.

[Filouman]

Thank you for the fix! Channel is back online.

I do see that I had taproot enabled but simple close was commented out, so I fixed that to avoid running into this in the future.

# Set to enable support for the experimental taproot channel type
protocol.simple-taproot-chans=true
# Set to enable  support for RBF based coop close.
protocol.rbf-coop-close=true

[Roasbeef]

The specification for taproot channels is still a draft, and I'm surprised that this is activated by default on mainnet, it's quite expected that this can cause compatibility issues...

We've been using the staging bit since the spec draft was released. Only recently did we define the RBF coop close extensions into the spec. Prior to that, the specified behavior was to just accept the initiator's fee as a stop gap, but flagging the coop close transaction.

IMO we should still gate behavior on the explicit presence of both feature bits (so taproot signal and the rbf signal). Our PR to add rbf close support is here: lightningnetwork/lnd#10063.

[Roasbeef]

but doesn't activate the option_simple_close feature (which is needed because otherwise there is no way to properly cooperatively close a taproot channel)

Also in the spec, the non-rbf coop close extensions are still defined (note the two sections: one for coop close, and the other for rbf coop close): https://github.com/Roasbeef/lightning-rfc/blob/4c1314a1eff12d814127aba2219d173340c11459/bolt-simple-taproot.md#cooperative-closure. IIRC y'all had this version working in the past as well?

[t-bast]

Also in the spec, the non-rbf coop close extensions are still defined (note the two sections: one for coop close, and the other for rbf coop close): https://github.com/Roasbeef/lightning-rfc/blob/4c1314a1eff12d814127aba2219d173340c11459/bolt-simple-taproot.md#cooperative-closure. IIRC y'all had this version working in the past as well?

No, we never implemented that, the goal was always to only use option_simple_close for taproot (since it was explicitly designed for this, and specified and implemented before taproot!).

I don't think the BOLTs should include an option to mutual close without option_simple_close: that is unnecessary and only increases technical debt? Note that option_simple_close can be used without doing any RBF if you wish, but it does ensure that each node has a closing transaction with the fee they wish to pay and clean nonce management for musig2.

[Roasbeef]

I don't think the BOLTs should include an option to mutual close without option_simple_close: that is unnecessary and only increases technical debt?

I agree, just wanted to point out that as spec'd one can use taproot channels without RBF close, as taproot channels was drafted before we had finished RBF close. Once we get final interop, we can remove that section. lnd will continue to use the existing specified version without the new RBF close messages for backwards compat reasons.