Async BumpTransactionEventHandler #3752

joostjager · 2025-04-28T19:09:23Z

Converts BumpTransactionEventHandler to async.

Changes the CoinSelectionSource and WalletSource traits to be async and provides WalletSourceSyncWrapper as a helper for users that want to implement a sync wallet source.

TestWalletSource is kept sync, to prevent a cascade of async conversions in tests.

Fixes #3540

ldk-reviews-bot · 2025-04-28T19:09:26Z

👋 Thanks for assigning @tnull as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

codecov · 2025-05-01T11:16:59Z

Codecov Report

Attention: Patch coverage is 90.08621% with 23 lines in your changes missing coverage. Please review.

Project coverage is 90.78%. Comparing base (35748f6) to head (e7d43db).
Report is 16 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/events/bump_transaction.rs	89.92%	5 Missing and 9 partials ⚠️
lightning/src/events/bump_transaction_sync.rs	90.00%	8 Missing ⚠️
lightning/src/util/test_utils.rs	85.71%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3752      +/-   ##
==========================================
+ Coverage   89.92%   90.78%   +0.86%     
==========================================
  Files         160      161       +1     
  Lines      129322   137021    +7699     
  Branches   129322   137021    +7699     
==========================================
+ Hits       116290   124393    +8103     
+ Misses      10345     9956     -389     
+ Partials     2687     2672      -15

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

tnull

Did a first / quickish pass. Generally looks good so far.

tnull · 2025-05-15T13:17:48Z

lightning/Cargo.toml

@@ -48,12 +48,14 @@ backtrace = { version = "0.3", optional = true }

 libm = { version = "0.2", default-features = false }
 inventory = { version = "0.3", optional = true  }
+tokio = { version = "1.35", features = [ "macros", "rt" ], optional = true }


Please set default-features = false here and below.

Added. Why is this? I looked at the tokio crate and it seems there are no default features?

It's 'best practice' and will also make sure we'd not add arbitrary dependencies if tokio decided to add default features in some release.

tnull · 2025-05-15T13:21:04Z

lightning/src/events/bump_transaction.rs

-					*commitment_tx_fee_satoshis,
-					anchor_descriptor,
-				) {
+				if let Err(_) = self


IMO these would be a bit more readable if we did the the whole call including .await on a separate line and then simply did if res.is_err() { .. }

Tried it, but I don't think it is an improvement with that temp var.

let res = self .handle_channel_close( *claim_id, *package_target_feerate_sat_per_1000_weight, commitment_tx, *commitment_tx_fee_satoshis, anchor_descriptor, ) .await; if res.is_err() { log_error!( self.logger, "Failed bumping commitment transaction fee for {}", commitment_tx.compute_txid() ); }

Also tried with map_err, but that doesn't work all that well with the parent fn without a return result.

Also tried with map_err, but that doesn't work all that well with the parent fn without a return result.

How about:

self.handle_channel_close( *claim_id, *package_target_feerate_sat_per_1000_weight, commitment_tx, *commitment_tx_fee_satoshis, anchor_descriptor, ) .await .unwrap_or_else(|_| { log_error!( self.logger, "Failed bumping commitment transaction fee for {}", commitment_tx.compute_txid() ); });

Ah this is nicer. Wasn't aware of the unwrap_or_else on result. Updated.

lightning/src/events/bump_transaction.rs

tnull · 2025-05-15T13:27:08Z

lightning/src/ln/functional_tests.rs

+	#[allow(clippy::expect_used, clippy::diverging_sub_expression)]
+	{
+		return tokio::runtime::Builder::new_current_thread()
+			.enable_all()


FWIW, I don't think we need to enable_all here, could just actually enable features we need. Same goes for all the tests that use [tokio::test], although not sure if it makes a big difference in practice.

I just expanded the macro. Can make customizations, but maybe it is better to keep to the original expansion? Also if it doesn't matter in practice...

Yeah, no strong opinion. If we increase the number of tests using tokio, we might want to some benchmarks though, as the runtime of cargo test accumulates over time, so every second we can squeeze out of it would be appreciated.

Removed then. If we get more of these xtests, we might need to duplicate the macro or something like that.

lightning/src/events/bump_transaction.rs

tnull · 2025-05-15T13:52:19Z

lightning/src/events/bump_transaction.rs

+			let configs = [(false, false), (false, true), (true, false), (true, true)];
+			let mut last_err = None;
+			for (force_conflicting_utxo_spend, tolerate_high_network_feerates) in configs {
+				log_debug!(
 				self.logger,
 				"Attempting coin selection targeting {} sat/kW (force_conflicting_utxo_spend = {}, tolerate_high_network_feerates = {})",
 				target_feerate_sat_per_1000_weight,
 				force_conflicting_utxo_spend,
 				tolerate_high_network_feerates
 			);


nit: Indent off here.

I don't see the indent off. The closing parenthesis?

Actually the parenthesis and all arguments of log_debug too (they need to be indented by one tab).

I don't see the problem anymore in the diff now, but github still shows it on the old code fragment? In my IDE it all looks good with tabs, so I think it is solved.

Nope, still there on HEAD:

Ah, happened in the next commit. Fixed now.

ldk-reviews-bot · 2025-05-15T13:52:43Z

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

TheBlueMatt · 2025-06-06T12:48:26Z

lightning/src/events/bump_transaction.rs

+	}
+}
+
+impl<W: Deref + MaybeSync + MaybeSend, L: Deref + MaybeSync + MaybeSend> CoinSelectionSource


I believe we should instead implement CoinSelectionSourceSync so that we can use BumpTransactionEventHandlerSync, no?

Oh man, you are totally right. Now I can revert all the asyncification of the tests 😅

I will be back

It seems that the compiler doesn't recognize the drop and complains that the mutex crosses an await (introduced in later commit), even though it doesn't.

joostjager · 2025-06-06T15:11:39Z

lightning/src/util/test_utils.rs

@@ -1874,36 +1873,36 @@ impl Drop for TestScorer {

 pub struct TestWalletSource {
 	secret_key: SecretKey,
-	utxos: RefCell<Vec<Utxo>>,
+	utxos: Mutex<Vec<Utxo>>,


Still need this in the sync code it seems, unfortunately.

I wonder if this is a show stopper for the whole sync wrapping concept. I hope not.

joostjager · 2025-06-09T07:02:24Z

As already mentioned in #3752 (comment), a lot of the asyncification could be reverted now that the right wrappers are available.

I also isolated the wrappers in their own file.

tnull

Seems CI is failing now.

joostjager · 2025-06-09T13:52:33Z

CI passes

joostjager · 2025-06-09T14:18:51Z

Reviewers, didn't do fix up commits as the commit structure changed substantially. Thanks for your patience with this PR. Should have realized that the sync wrappers could avoid all the test changes that have now been reverted.

tnull

Basically LGTM, some nits/comments.

Also double-checked this still works with LDK Node.

lightning/src/events/bump_transaction.rs

tnull · 2025-06-10T10:39:21Z

lightning/src/events/bump_transaction_sync.rs

Not sure having this module is parallel is the way to go. If you really prefer to move the wrappers to their own file, should they be in events/bump_transaction/blocking.rs (or sync.rs, but the latter is usually associated with synchronization primitives in Rust), for example?

Also, this module is missing a licensing header.

I put them in a separate file to avoid large files. Quite a few devs indicated that they did not like large files.

What is wrong with bump_transaction_sync.rs? It aligns with the types inside. Something like blocking.rs might grow big again.

What is wrong with bump_transaction_sync.rs?

In terms of crate organization, it's just a file in-parallel even though it hold utilities for the bump_transaction logic. So would just make sense to have it part of the same sub-module rather an in-parallel module, IMO, but no blocker.

Something like blocking.rs might grow big again.

Not sure I'm following how the name relates to what/how much we put in a file?

Ah, I missed that you suggested a bump_tx subdir. Made the change in a fix up commit. Generated docs look better now, with sync residing below bump_transaction. Let me know if this is what you had in mind.

lightning/src/events/bump_transaction_sync.rs

This preparatory commit converts do_coin_selection from a closure into a standard function.

tnull

Feel free to squash fixups from my side.

joostjager · 2025-06-10T13:25:56Z

Squashed

lightning/src/events/bump_transaction/sync.rs

joostjager force-pushed the async-wallet-bump branch 3 times, most recently from d9b1c75 to 873516b Compare May 1, 2025 11:08

joostjager force-pushed the async-wallet-bump branch 6 times, most recently from 4b4b898 to 3816c66 Compare May 5, 2025 11:37

joostjager force-pushed the async-wallet-bump branch 13 times, most recently from 1a86521 to e1509f1 Compare May 14, 2025 09:55

joostjager changed the title ~~Async wallet bump~~ Async BumpTransactionEventHandler May 14, 2025

joostjager marked this pull request as ready for review May 14, 2025 11:06

joostjager requested a review from tnull May 14, 2025 11:06

joostjager added the weekly goal Someone wants to land this this week label May 14, 2025

tnull reviewed May 15, 2025

View reviewed changes

joostjager force-pushed the async-wallet-bump branch 2 times, most recently from d8f4c67 to 462afa0 Compare June 6, 2025 08:29

joostjager requested review from TheBlueMatt and tnull June 6, 2025 12:15

TheBlueMatt reviewed Jun 6, 2025

View reviewed changes

Add scope instead of drop

decddcb

It seems that the compiler doesn't recognize the drop and complains that the mutex crosses an await (introduced in later commit), even though it doesn't.

joostjager force-pushed the async-wallet-bump branch from 462afa0 to c34c042 Compare June 6, 2025 15:10

joostjager commented Jun 6, 2025

View reviewed changes

joostjager removed the request for review from tnull June 6, 2025 15:33

joostjager force-pushed the async-wallet-bump branch from c34c042 to d01009c Compare June 9, 2025 06:58

tnull reviewed Jun 9, 2025

View reviewed changes

joostjager force-pushed the async-wallet-bump branch 2 times, most recently from 1892700 to e7d43db Compare June 9, 2025 11:08

joostjager requested review from tnull and TheBlueMatt June 9, 2025 14:17

tnull reviewed Jun 10, 2025

View reviewed changes

Rewrite closure do_coin_selection in Wallet impl

048230a

This preparatory commit converts do_coin_selection from a closure into a standard function.

joostjager force-pushed the async-wallet-bump branch 2 times, most recently from 93e5d7e to 2fdcf05 Compare June 10, 2025 12:54

tnull reviewed Jun 10, 2025

View reviewed changes

joostjager force-pushed the async-wallet-bump branch from 2fdcf05 to 94bbb5c Compare June 10, 2025 13:25

tnull reviewed Jun 10, 2025

View reviewed changes

lightning/src/events/bump_transaction/sync.rs Outdated Show resolved Hide resolved

Convert BumpTransactionEventHandler to async

0ca7739

joostjager force-pushed the async-wallet-bump branch from 94bbb5c to 0ca7739 Compare June 10, 2025 13:35

joostjager requested a review from tnull June 10, 2025 13:37

tnull approved these changes Jun 10, 2025

View reviewed changes

Async BumpTransactionEventHandler #3752

Are you sure you want to change the base?

Async BumpTransactionEventHandler #3752

Conversation

joostjager commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ldk-reviews-bot commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented May 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

tnull left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joostjager May 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joostjager May 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joostjager May 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ldk-reviews-bot commented May 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joostjager commented Jun 9, 2025

Uh oh!

tnull left a comment

Choose a reason for hiding this comment

Uh oh!

joostjager commented Jun 9, 2025

Uh oh!

joostjager commented Apr 28, 2025 •

edited

Loading

ldk-reviews-bot commented Apr 28, 2025 •

edited

Loading

codecov bot commented May 1, 2025 •

edited

Loading

tnull left a comment •

edited

Loading

joostjager May 15, 2025 •

edited

Loading

joostjager May 15, 2025 •

edited

Loading

joostjager May 16, 2025 •

edited

Loading