Support group keys on `SendPayment` & `AddInvoice` #1423

GeorgeTsagk · 2025-03-05T19:02:16Z

Description

This PR takes care of the rest of the work required to create and pay invoices by using group keys.

Using a group_key on SendPayment means that only channels that contain assets which belong to that group are going to be considered for sending this payment.
Using a group_key on AddInvoice means that only channels that contain assets which belong to that group may be considered for generating an RFQ quote and encoding it in the invoice.

LiT PR for itest

coveralls · 2025-03-05T19:20:13Z

Pull Request Test Coverage Report for Build 14266166576

Details

23 of 305 (7.54%) changed or added relevant lines in 9 files are covered.
47 unchanged lines in 10 files lost coverage.
Overall coverage decreased (-0.02%) to 28.373%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
tapchannel/aux_invoice_manager.go	17	18	94.44%
rfqmsg/records.go	6	9	66.67%
asset/asset.go	0	8	0.0%
taprpc/tapchannelrpc/tapchannel.pb.go	0	10	0.0%
rfq/manager.go	0	13	0.0%
tapchannel/aux_traffic_shaper.go	0	13	0.0%
rfq/order.go	0	29	0.0%
rpcserver.go	0	82	0.0%
rfq/marshal.go	0	123	0.0%

Files with Coverage Reduction	New Missed Lines	%
rfqmsg/records.go	1	64.11%
asset/asset.go	2	50.6%
asset/group_key.go	2	57.89%
asset/mock.go	2	64.72%
tapchannel/aux_invoice_manager.go	2	86.18%
tappsbt/create.go	2	26.74%
tapchannel/aux_leaf_signer.go	5	43.08%
tapgarden/caretaker.go	7	68.15%
address/address.go	8	67.47%
address/mock.go	16	86.93%

Totals
Change from base Build 14199175190:	-0.02%
Covered Lines:	25876
Relevant Lines:	91198

💛 - Coveralls

GeorgeTsagk · 2025-03-05T19:21:56Z

The current LiT itests are going to pass, as they will keep using a single asset ID for payments and invoices.

This LiT PR uses the new group key arguments for creating/paying invoices.

GeorgeTsagk · 2025-03-11T19:40:32Z

Rebased on main after merging base branch #1382

rfqmsg/records.go

rfq/order.go

tapchannel/aux_invoice_manager.go

rfq/manager.go

tapchannel/aux_traffic_shaper.go

rpcserver.go

guggero

Did a first pass.

Concerning the discussion around group key in the asset sum entry.
I failed to consider the group key send case when designing the p2p messages.

Once an HTLC is locked in, the allocation code makes sure each HTLC has a specific, clearly defined set of asset pieces, all with IDs defined.
But as long as the HTLC is just transmitted (e.g. on the SendPayment RPC, in ProduceHtlcExtraData function of the aux traffic shaper, or even in the UpdateAddHtlc wire message), we don't actually know the ID of the piece(s) that are going to be used for the HTLC.
So in those scenarios we only really check two things in the policies:

The sum of the assets being transferred in the HTLC message
Whether the assets match the policy

For the second check, I think it makes sense to indirectly put the group key in there. Whether we hash it first (as we do for the universe key) or whether we just use the x-coordinate of the group key doesn't matter too much IMO. But I think we need to be very clear wherever we do that and definitely add a comment on why we're doing that (feel free to copy my explanation above).

rfqmsg/records.go

rfq/order.go

rfq/manager.go

rpcserver.go

GeorgeTsagk · 2025-03-26T17:26:49Z

Changed base to lnd-19-rc1 in order to be able to cross test with LitD itests

guggero

Very nice!
Super super close, just a couple of nits and improvement requests.

guggero · 2025-03-27T21:58:58Z

rfq/manager.go

@@ -949,6 +951,10 @@ func (m *Manager) getAssetGroupKey(ctx context.Context,
 	// Perform the DB query.
 	group, err := m.cfg.GroupLookup.QueryAssetGroup(ctx, id)
 	if err != nil {
+		if errors.Is(err, address.ErrAssetGroupUnknown) {


Do we still need this commit, given that we now check whether a given asset ID is a group key before querying the DB?
It just feels weird to return fn.None[btcec.PublicKey](). To me that means: the given asset ID is _not_ part of a group.

Yes this is still valid for cases where AssetMatchesSpecifier is called with an asset ID and a specifier which includes a group key

An actual group lookup would have to be performed for that set of inputs. If that asset doesn't belong to a group we would run into the above line.

An actual group lookup would have to be performed for that set of inputs. If that asset doesn't belong to a group we would run into the above line.

Seems related to the first comment I left in this round of review. Looks like we would do a DB look up to make sure that we didn't accidentally interpret an asset ID as a group key.

guggero · 2025-03-27T21:59:47Z

rfq/manager.go

+			return true, nil
+		}
+
+		// Now let's make an actual query to find this assetID's group,


Yeah, given that we don't call getAssetGroupKey with a "fake" asset ID anymore, I think we can potentially remove the previous commit.

previous reply

tapchannel/aux_traffic_shaper.go

taprpc/tapchannelrpc/tapchannel.proto

guggero · 2025-03-27T22:04:25Z

rpcserver.go

+		return err
+	}
+
+	switch {


Should we add this switch and the error check above as a new function asset.NewExclusiveSpecifier()?
To make sure we use the same semantics everywhere.

The above error comes from the rpc marshaller, we could make such a func for the switch itself (i.e make a specifier that only contains one of assetID/groupKey) but I'm not sure if there's a clear way of saying which one to keep? Would we default to keeping the groupKey over the assetID or vice versa?

taprpc/tapchannelrpc/tapchannel.proto

guggero · 2025-03-27T22:05:11Z

rpcserver.go

@@ -7684,18 +7684,31 @@ func checkOverpayment(quote *rfqrpc.PeerAcceptedSellQuote,
 func (r *rpcServer) AddInvoice(ctx context.Context,
 	req *tchrpc.AddInvoiceRequest) (*tchrpc.AddInvoiceResponse, error) {

+	if len(req.AssetId) > 0 && len(req.GroupKey) > 0 {


Re-use potential, see my previous comment for NewExclusiveSpecifier.

ah, since there's a new occurrence here it makes sense to add smth for this

I guess can add a simple NewExclusiveSpecifier that accepts both assetID/groupKey and prioritizes groupKey over assetID, given that it's more generic

This check can be moved as well.

Roasbeef

LGTM ❄️

I dig the change to move to using the x-coord instead of the hash.

Changes look good pending the final comments from the other reviewer.

Will check out the corresponding itest changes now.

Roasbeef · 2025-04-01T00:51:35Z

rfq/order.go

+		groupKey := c.AssetSpecifier.UnwrapGroupKeyToPtr()
+		groupKeyX := schnorr.SerializePubKey(groupKey)
+
+		assetID = asset.ID(groupKeyX)


This is nice compared to the hash version, as this way we aren't actually losing any information.

However, it's possible that a normal asset ID, can actually be interpreted as a valid x coordinate. To avoid this confusion, when we decode on the other side, and conclude it might be a pubkey, we should check our local db to make sure it's actually a group key that we've validated.

I think this is the receiver's part that you're describing

taproot-assets/rfq/manager.go

Lines 984 to 990 in e2e8e47

// Let's directly check if the ID is equal to the hash of the

// group key. This is used by the sender to indicate that any

// asset that belongs to this group may be used.

groupKeyX := schnorr.SerializePubKey(specifierGK)

if asset.ID(groupKeyX) == id {

return true, nil

}

a receiver is always checking this "assetID" against a specifier. If that specifier contains a groupkey we'll immediately check if the X coordinates match before doing any DB lookups

we should check our local db to make sure it's actually a group key that we've validated.

That's an interesting point. Do we really care if the group key is validated?

If that group key made it into a channel or RFQ quote, then it can't be a fake or un-validated group key. If you have DoS vectors in mind, I haven't digged a lot into that.

Roasbeef · 2025-04-01T00:55:13Z

rfq/manager.go

@@ -949,6 +951,10 @@ func (m *Manager) getAssetGroupKey(ctx context.Context,
 	// Perform the DB query.
 	group, err := m.cfg.GroupLookup.QueryAssetGroup(ctx, id)
 	if err != nil {
+		if errors.Is(err, address.ErrAssetGroupUnknown) {


An actual group lookup would have to be performed for that set of inputs. If that asset doesn't belong to a group we would run into the above line.

Seems related to the first comment I left in this round of review. Looks like we would do a DB look up to make sure that we didn't accidentally interpret an asset ID as a group key.

rfq/manager.go

guggero

One last simplification request, otherwise LGTM 🎉

Pending @Roasbeef's other comments.

rpcserver.go

guggero · 2025-04-03T15:10:40Z

rpcserver.go

@@ -7684,18 +7684,31 @@ func checkOverpayment(quote *rfqrpc.PeerAcceptedSellQuote,
 func (r *rpcServer) AddInvoice(ctx context.Context,
 	req *tchrpc.AddInvoiceRequest) (*tchrpc.AddInvoiceResponse, error) {

+	if len(req.AssetId) > 0 && len(req.GroupKey) > 0 {


This check can be moved as well.

We add a new interface to the HTLC SumAssetBalance method, which helps check the identifier of the asset against a specifier. This allows for checking asset inclusion in a group, which is a bit involved and not the responsibility of the HTLC model.

We extend the interface of the rfq Policy in order to allow the specifier checker to be involved. This extends certain checks, and allows us to use asset specifiers that only have a group key.

When intercepting an HTLC which is incoming from a btc channel and outgoing towards an asset channel, we need to create an HTLC record which reflects the asset related balance change. What really matter in these records is the total asset amount, but we also set the assetID field to be either equal to the specific asset we're sending, or the group of assets. This field does not dictate what the actual assets being sent are going to be, that will be decided later in the asset allocation process.

This allows us to get rid of a circular dependency issue that would occur in a follow up commit, where we import the adress package in the rfq package. Co-authored-by: Oliver Gugger <[email protected]>

We add the specifier checker interface to the AuxInvoiceManager too, as it is needed to validate incoming HTLCs which may use asset IDs that belong to a group, while the RFQ is based on a group key.

Adds some coverage to the invoice manager unit tests, which involve an RFQ quote over a group key, plus an HTLC with multiple asset balances, which may belong or not to the group.

We may be performing a group lookup on an asset that doesn't belong to a group. Instead of returning the error that originates from the ErrNoRows of sql we instead return a nil result, signalling that no group was found and no error occurred.

In some cases the sender of an asset HTLC may just encode the x part of the group key as the assetID of the balance in the custom records. This is done as a way to signal that any asset ID that belongs to this group may be picked to carry out the payment. This commit makes the specifier matcher aware of that case.

As mentioned in the previous commit, we occasionally want to just encode the hash of the group key as the asset ID of the balance that is encoded in the custom record. We make the ProduceHtlcExtraData hook aware of that case, which is triggered when the quote is made upon a group key and not a specific asset ID.

We add this new specifier creator which may accept both an asset id and group key. This is needed for callers who may have both fields available and simply want to derive an asset specifier from them. By default a specifier over the group key is prioritized as that's considered more generic over the specific asset id. This is used in a later commit.

We have taken care of the groupkey RFQ negotiation in previous commits. All we need now to support sending a payment over a group of assets, is to propagate the user specifier groupkey to the corresponding fields.

In this commit we take the user defined group key and allow the asset specifier to be created over it. All the calls that accept it as an argument are already groupkey aware.

GeorgeTsagk self-assigned this Mar 5, 2025

GeorgeTsagk mentioned this pull request Mar 5, 2025

Support group keys on HTLC Interceptor & Invoice HTLC Modifier #1416

Closed

levmi added this to Taproot-Assets Project Board Mar 6, 2025

github-project-automation bot moved this to 🆕 New in Taproot-Assets Project Board Mar 6, 2025

levmi moved this from 🆕 New to 🏗 In progress in Taproot-Assets Project Board Mar 6, 2025

GeorgeTsagk force-pushed the rfq-negotiation-groupkey branch from 7c660bd to 6cf7190 Compare March 10, 2025 15:12

Base automatically changed from rfq-negotiation-groupkey to main March 11, 2025 17:31

GeorgeTsagk force-pushed the taprpc-groupkey-support branch from 9c9de09 to b53b3a2 Compare March 11, 2025 19:40

GeorgeTsagk force-pushed the taprpc-groupkey-support branch from b53b3a2 to f13f4a0 Compare March 19, 2025 11:22

GeorgeTsagk requested review from guggero and Roasbeef March 19, 2025 11:22

Roasbeef requested changes Mar 21, 2025

View reviewed changes

github-project-automation bot moved this from 🏗 In progress to 👀 In review in Taproot-Assets Project Board Mar 21, 2025

GeorgeTsagk force-pushed the taprpc-groupkey-support branch from f13f4a0 to 27faee9 Compare March 24, 2025 11:00

guggero reviewed Mar 24, 2025

View reviewed changes

GeorgeTsagk force-pushed the taprpc-groupkey-support branch from 27faee9 to 99e85a7 Compare March 26, 2025 17:17

GeorgeTsagk mentioned this pull request Mar 26, 2025

Custom channels itest: Use group keys on some payments & invoices lightninglabs/lightning-terminal#995

Open

GeorgeTsagk requested review from Roasbeef and guggero March 26, 2025 17:32

GeorgeTsagk mentioned this pull request Mar 27, 2025

Multi rfq receive (AddInvoice multiple hop hints) #1457

Open

guggero reviewed Mar 27, 2025

View reviewed changes

GeorgeTsagk force-pushed the taprpc-groupkey-support branch from 99e85a7 to e2e8e47 Compare March 31, 2025 19:19

Roasbeef approved these changes Apr 1, 2025

View reviewed changes

GeorgeTsagk requested a review from guggero April 1, 2025 14:33

guggero approved these changes Apr 3, 2025

View reviewed changes

GeorgeTsagk and others added 14 commits April 4, 2025 13:03

rfq: policies use specifier checker

cdf63b5

We extend the interface of the rfq Policy in order to allow the specifier checker to be involved. This extends certain checks, and allows us to use asset specifiers that only have a group key.

multi: move rfq marshal code into rfq package

67f16f0

This allows us to get rid of a circular dependency issue that would occur in a follow up commit, where we import the adress package in the rfq package. Co-authored-by: Oliver Gugger <[email protected]>

tapchannel: aux invoice manager uses specifier checker

48b32a2

We add the specifier checker interface to the AuxInvoiceManager too, as it is needed to validate incoming HTLCs which may use asset IDs that belong to a group, while the RFQ is based on a group key.

tapchannel: add group key tests to AuxInvoiceManager

581665c

Adds some coverage to the invoice manager unit tests, which involve an RFQ quote over a group key, plus an HTLC with multiple asset balances, which may belong or not to the group.

taprpc: add groupkey to SendPaymentRequest

cd6026c

rpcserver: add asset specifier marshaller

b5fc58a

rpcserver: SendPayment uses groupkey

75a67e6

We have taken care of the groupkey RFQ negotiation in previous commits. All we need now to support sending a payment over a group of assets, is to propagate the user specifier groupkey to the corresponding fields.

taprpc: add groupkey to AddInvoice

154b001

rpcserver: AddInvoice uses groupkey

e9e1de8

In this commit we take the user defined group key and allow the asset specifier to be created over it. All the calls that accept it as an argument are already groupkey aware.

GeorgeTsagk force-pushed the taprpc-groupkey-support branch from e2e8e47 to e9e1de8 Compare April 4, 2025 13:13

GeorgeTsagk enabled auto-merge April 4, 2025 13:39

GeorgeTsagk added this pull request to the merge queue Apr 4, 2025

Merged via the queue into main with commit c63d418 Apr 4, 2025
18 checks passed

github-project-automation bot moved this from 👀 In review to ✅ Done in Taproot-Assets Project Board Apr 4, 2025

guggero deleted the taprpc-groupkey-support branch April 4, 2025 14:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support group keys on `SendPayment` & `AddInvoice` #1423

Support group keys on `SendPayment` & `AddInvoice` #1423

GeorgeTsagk commented Mar 5, 2025 •

edited

Loading

coveralls commented Mar 5, 2025 •

edited

Loading

GeorgeTsagk commented Mar 5, 2025

GeorgeTsagk commented Mar 11, 2025

guggero left a comment

GeorgeTsagk commented Mar 26, 2025

guggero left a comment

guggero Mar 27, 2025

GeorgeTsagk Mar 31, 2025

Roasbeef Apr 1, 2025

guggero Mar 27, 2025

GeorgeTsagk Mar 31, 2025

guggero Mar 27, 2025

GeorgeTsagk Mar 31, 2025

guggero Mar 27, 2025

GeorgeTsagk Mar 31, 2025

guggero Apr 3, 2025

Roasbeef left a comment

Roasbeef Apr 1, 2025

GeorgeTsagk Apr 4, 2025

Roasbeef Apr 1, 2025

guggero left a comment

guggero Apr 3, 2025

	// Let's directly check if the ID is equal to the hash of the
	// group key. This is used by the sender to indicate that any
	// asset that belongs to this group may be used.
	groupKeyX := schnorr.SerializePubKey(specifierGK)
	if asset.ID(groupKeyX) == id {
	return true, nil
	}

Support group keys on SendPayment & AddInvoice #1423

Support group keys on SendPayment & AddInvoice #1423

Conversation

GeorgeTsagk commented Mar 5, 2025 • edited Loading

Description

coveralls commented Mar 5, 2025 • edited Loading

Pull Request Test Coverage Report for Build 14266166576

Details

💛 - Coveralls

GeorgeTsagk commented Mar 5, 2025

GeorgeTsagk commented Mar 11, 2025

guggero left a comment

Choose a reason for hiding this comment

GeorgeTsagk commented Mar 26, 2025

guggero left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Roasbeef left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

guggero left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Support group keys on `SendPayment` & `AddInvoice` #1423

Support group keys on `SendPayment` & `AddInvoice` #1423

GeorgeTsagk commented Mar 5, 2025 •

edited

Loading

coveralls commented Mar 5, 2025 •

edited

Loading