-
Notifications
You must be signed in to change notification settings - Fork 5.8k
Add OP_TEMPLATEHASH and TH+CSFS+IK bundle BIP #1974
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Some previous related discussions:
|
I'm afraid I'm going to write a lot and say very little, but I have a few thoughts maybe worth sharing. The improvements on CTV are obvious, pushing the hash onto the stack is better, leveraging only cached intermediate hashes is better. scriptSig CommitmentDropping the scriptSig commitment seems debatable, but I accept the given rationale for omitting it. Furthermore, CTV's scriptSig commitment introduces a new variable length chunk of data to hash. The Annex CommitmentThe annex commitment is of course the most controversial addition. On the one hand, it is incredibly useful to eltoo style protocols, and I can imagine, for example, that CISA might leverage the annex to set the order of public keys in an aggregate key. In this scenario, committing to the entire annex ahead of time would prevent using CISA, though once again I confess I don't know what such a scheme would actually look like. My instinct here is to reduce the scope of On the other hand, introducing a future This is why I'm ok with the proposal as-is, if the annex commitment proves to be a mistake, it can still serve a useful purpose now and into the future, and a second opcode could be introduced to address the annex-less case. Crazy TalkI'd like to very tentatively float another way to dodge the annex issue: a hash selector bitfield. To this, instagibbs said something like "why not go full TXHASH then?" Unlike TXHASH which has a huge amount of flexibility, a bitfield that permits choosing to commit to all or none of the sequences, all or none of the outputs, and all or none of the annex should not change the stopping point. One could also leverage the other five bits of a 1 byte stack item to potentially commit to the first N<32 outpoints, to support the sibling commitments. This last one in particular changes the performance profile of I can say that for me, committing to all of the outputs but none of the input sequences (and therefore not the input count) would be very useful for one of my projects which currently uses a deleted key ALL|ANYONECANPAY signature. Hashing PerformanceToday one can use the script With At exactly n = 80, the SHA256 script will hash 41600 bytes with a 763 byte script. Less naively, one can use the script Similarly, the The less naive SHA256 script will hash 57720 bytes in 782 script bytes, and the less naive I suppose all of that is to say "This is strictly less hashing than is necessary for other existing operations." seems correct, assuming there aren't more clever pathological scripts. Furthermore, the result of |
Wouldn't the hash naturally be cached in implementation? |
Thanks for the feedback @Ademan.
Some amount of bikeshedding is justified for a proposal as consequential as a consensus change. :)
If i understand correctly, your criticism is that while committing to the annex is very useful for the flagship usecase of this proposal, this may clash with potential future usages of the annex. That on its own i don't think is a valid reason not to have the annex commitment, if only because you can make the same argument in the other direction: not committing to the annex would be a design mistake as it allows to be forward-compatible with future usages of the annex, and is already provably useful for some of the main applications enabled by this proposal. To substantiate your point you bring up that the annex would be the only witness element committed to by As more of a general point, i think starting from |
In the Taproot signature hash (which this proposal re-uses), various parts of the transaction are pre-hashed such that the hash can be cached across signatures. For each signature, those pre-computed hashes are hashed together along with some fixed-size transaction fields, such that the maximum amount of data hashed per signature is capped: Line 128 in 87f3fe1
This final hash is pointless to cache, because other opcodes let you hash strictly more arbitrary (i.e. uncachable) data (and even have less restrictions on them). |
Clearly, I think, this is the most debatable point precisely because there isn't widespread use (or demand yet) for such space.
Any scheme that wants to shift from sign-time OP_RETURN to witness data may want to use it in the same manner. It's my fault for making ln-symmetry the go-to during discussion. Granted, this is a fairly minimal improvement, and can be done by many other means with CSFS and friends, or just eating the OP_RETURN cost. In the end, "Any annex scheme that is compatible with OP_CHECKSIG(ADD) is compatible with OP_TEMPLATEHASH" seems the most straight-forward and least foot-gunny to me.
I feel like without a specific proposal, we're going to miss the real costs in terms of complexity. I'm also worried that if a proposal becomes "general purpose" without actually being general, it won't capture the behavior we actually want, with respect to "programmable money".
No BitVM bridge has launched, and the space is moving very very fast. It's on my docket to try to learn more about the fundamental requirements of these systems to transition from permissioned systems to permissionless for both safety of funds and unilateral exit (these are different concepts IIUC), but I think this is out of scope, and the answer may be "we need GSR / btclisp / Simplicity" and stop trying to guess what specific systems need? I have plenty of controversial thoughts on this but seems way out of scope here, unless we find the next-tx capability and rebindable signatures insufficient motivation for a softfork. |
My take on the annex is as long as there is no clear potential use for the annex proposed, treating it consistently seems to make the most sense. We treat it exactly like the taproot sighashes are, so that's consistent. Take note that we're only committing the current input's annex, if the annex would be used for something that doesn't work with TEMPLATEHASH, it just means they can't be used together, which seems fine. It would also not be able to be used together with any other sighash-(all/default)-using scheme. I feel that since new use-cases for the annex will almost certainly come with their own soft fork, they can be resolved when the time comes. You speak about CISA, which I think is a great example, but in my opinion by the time we have a solid CISA soft fork proposal, the TXHASH and TXSIGHASH ideas will hopefully also be more mature and they can solve the annex aspect more elegantly. This proposal is trying to move the needle for simple next-tx commitment and basic rebindable signatures. Two arguably very simple features that are undeniably useful for almost all second-layer protocols currently being developed and used. |
If nodes start relaying transactions with non-empty annex the result will predictably a new exogenous asset protocols and a new simpler more efficient ways to inscribe arbitrary data without the need for pre-commitment. Unleashing the annex is a predictable footgun socially speaking. |
To be clear, this PR / implementation isn't about relaxing these rules for relay. |
Yes, this is only relevant further down the line. I think what @Ademan was also picking at is that this proposal feels predictive direction wise. |
Sure! I hope it's productive...
From my very limited perspective, that seems considerably less likely than the opposite, but that is kind of the problem, this is ~guessing. Maybe it would be useful to at least attempt to identify concrete or partially specified proposals that use the annex. I suppose the onus is on me since I'm the one suggesting they may conflict, but you, instagibbs, steven and moonsettler are all likely closer to the people who might be working on them.
Absolutely! I just want to ensure that "it's useful" isn't overriding compatibility concerns. There are other ways to do data publication, it doesn't necessarily need to be a part of
To be clear, I'm distinguishing between ahead of time commitment (via I think I may have been talked out of my position, but I still think it deserves some litigation (perhaps with a better advocate than I). To give it one last go, I suspect (admittedly without evidence!) that varying annex data independently of the rest of the TX will be the norm and desirable if/when the annex finds uses besides data publication.
Fair enough, I definitely agree with the second part. |
transaction. `OP_CHECKTEMPLATEVERIFY` gives txid stability when the committed spending transaction has a single input, | ||
and when the scriptSig of this single input has been committed by the hash. | ||
Taproot scriptSigs must be empty and therefore under the single input case `OP_TEMPLATEHASH` has no requirement | ||
to commit to scriptSigs to achieve txid stability. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
everytime I read through this always my biggest question is "why not commit to all script sigs being empty" or "what are the potential downsides to having multi input malleability"
would be nice to have an explanation in this to explain why this trade off is okay
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- the input is segwit, so it already explicitly commits to its own scriptSig being blank
- the purported benefit to committing to other scriptSigs in addition was to ensure txid stability, but without fixing the other inputs' prevout ahead of time, this cannot be achieved in general. (The "BitVM Trick" was to ensure this was actually possible, but I consider this not an intentional design decision; I'd rather the "sibling input commitment" be done properly if that's what we want!)
Given that it doesn't ensure the "next transaction" commitment we are aiming for which includes additional hashing on top of taproot hashes already existing, it was not added to the proposal.
@instagibbs how do you feel about adding OP_SHA256TAGGED to this mix? Even with the "French CTV" it would be a pretty viable LNhance alternative I could support without any misgivings. |
- *hash_type*: this is the sighash type identifier. Only a single hash type is supported by `OP_TEMPLATEHASH`, so there | ||
is no need to commit to such an identifier. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If that’s the case, it should be mentioned in the specification.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The specification only specifies a single hash type, I'm not sure what you mean for me to do?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don’t see any references to sighash types in the specification section, let alone that OP_TEMPLATEHASH only supports a single hash type. What is the sighash type that you need to use with OP_TEMPLATEHASH, and shouldn’t that be mentioned in the spec?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is no signature, so there is no sighash type byte. It is absent, so it is not part of the specification. In this paragraph we simply underline the differences with the BIP341 signature message. Because the sighash type is committed there, the difference is underlined here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I see. The phrase “only a single hash type is supported by OP_TEMPLATEHASH
” confused me in the context of hash_type
being omitted, and I think the context could be made clearer. Perhaps something like:
- *hash_type*: this is the sighash type identifier. Only a single hash type is supported by `OP_TEMPLATEHASH`, so there | |
is no need to commit to such an identifier. | |
- *hash_type*: refers to the sighash type identifier in the context of BIP341 signatures. The input for `OP_TEMPLATEHASH` is fixed, so there | |
is no need for a mechanism to modify the hash composition. |
committed in the output itself. Committing to all other prevouts or scriptpubkeys would introduce hashing a quantity | ||
of data quadratic in the number of inputs. It would also prevent spending two coins encumbered by a template hash | ||
check in the same transaction. Finally, the flexibility of not committing to the specific coins spent is also | ||
desirable to recover from mistakes[^no-commit-other-coins]. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wouldn’t the commitment to all of the inputs’ sequence fields, sha_sequences
, indirectly commit to the count of inputs, and therefore prevent adding another input?
(Or maybe I’m misconstruing what sha_sequences
exactly is, feel free to correct me in that case.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
commit to the count of inputs, and therefore prevent adding another input?
Yes, it commits to the total number of inputs. Typically this would be 1, but you can certainly have more for batching scenarios.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right, but when your OP_TEMPLATEHASH commits to the spending transaction having a single input, you would not have the flexibility to add a second output to provide more funds, i.e., doesn’t that contradict your flexibility point?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there's an included footnote for a (weak) example with respect to value, though it doesn't apply to prevouts per se. If the other coin gets swept, you can "contribute" a new utxo to make it whole and rescue the locked funds.
It's not the primary motivation for the design decision compared to the others.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On second reading, I do see what you mean. It seems to me that this could be discussed more explicitly than such a subtle footnote.
under a large number of different conditions. It reuses the [Bitcoin Core Taproot test framework][feature_taproot.py] | ||
introduced with the implementation of BIP341. Format details and usage demonstration are available | ||
[here](bip-templatehash/test_vectors/README.md). | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there already a reference implementation for this proposal? If so, please include a section linking to it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
bip-templatehash-csfs-ik.md
Outdated
``` | ||
BIP: ? | ||
Layer: Consensus (soft fork) | ||
Title: OP_TEMPLATEHASH + OP_CHECKSIGFROMSTACK + OP_INTERNALKEY |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Title exceeds 44 characters.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
renamed for now
bip-templatehash-csfs-ik.md
Outdated
This document proposes bundling three new operations for [Tapscript][tapscript-bip]: | ||
[`OP_TEMPLATEHASH`][templatehash-bip], [`OP_CHECKSIGFROMSTACK`][csfs-bip], and [`OP_INTERNALKEY`][internalkey-bip]. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please include the BIP numbers for the BIPs that already exist in the text here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
bip-templatehash-csfs-ik.md
Outdated
The three proposed operations are simple, well-understood, and enable powerful new capabilities while minimizing the | ||
risk of surprising behavior or unintended applications. They improve existing, well-studied protocols and make promising | ||
new ones possible, while minimizing the risk of surprising behavior or unintended or undesirable applications. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This duplicates “while minimizing the risk of surprising behavior or unintended or undesirable applications.” This phrase also appears a third time below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
removed the second and third ones
bip-templatehash-csfs-ik.md
Outdated
risk of surprising behavior or unintended applications. They improve existing, well-studied protocols and make promising | ||
new ones possible, while minimizing the risk of surprising behavior or unintended or undesirable applications. | ||
|
||
`OP_TEMPLATEHASH` enables commitment to the transaction spending an output. `OP_CHECKSIGFROMSTACK` enables |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
“enables commitment” sounds funny to me:
`OP_TEMPLATEHASH` enables commitment to the transaction spending an output. `OP_CHECKSIGFROMSTACK` enables | |
`OP_TEMPLATEHASH` enables committing to the transaction spending an output. `OP_CHECKSIGFROMSTACK` enables |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
taken
|
||
The ability to push the Taproot internal key on the stack is a natural and extremely simple optimisation for rebindable | ||
signatures. | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This BIP appears to be missing a Specification section.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added mini implementation section for completeness
rules: there is no block that is valid under the rules proposed in this BIP but not under the existing Bitcoin consensus | ||
rules. As a consequence these changes are backward-compatible with non-upgraded node software. That said, the authors | ||
strongly encourage node operators to upgrade in order to fully validate all consensus rules. | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If there is a reference implementation for this opcode package, could you please add a section that links to the corresponding code?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no current one exists, we haven't moved forward on bitcoin-inquisition with templatehash and likely won't until the BIP becomes stable
adf18c9
to
d359ceb
Compare
Seems out of scope here, unless I'm missing some key motivation. |
You came up with the motivation yourself afaik (for LN-Symmetry). It's about the whole annex/nulldata/paircommit data availability thing. It is also mildly useful for MATT stuff in a more expressive future script, otherwise it gives us a safe pair-commitment for two stack elements. Assuming this proposal is activated as-is (as a thought experiment), even if core would be reluctant to, as proven by precedents, it would be a trivial exercise for an LSP to break the annex "filter" using the libre strategy with just a few tens of nodes. Forcing core to "face the reality of the network" and relax policy. Hence our comments on this proposal being predictive of annex use. I don't want to waste too much time on this topic. But this is exactly why after much agonizing PC was included in LNhance. |
Opening this here for wider discussion and feedback.
edit: Probably need to add implementation section: https://github.com/instagibbs/bitcoin/tree/2025-07-op_templatehash