diff --git a/conf.py b/conf.py index ae8f159..19bd5b5 100644 --- a/conf.py +++ b/conf.py @@ -185,3 +185,37 @@ # A list of files that should not be packed into the epub file. epub_exclude_files = ['search.html'] + +# These ReST entries are included in every doc and are used to build +# aliases to commonly-referenced items. For example, the use of +# :ref:`p2p-network-header` would produce the link text "P2P Network" +# despite general use in documentation being "network". These aliases +# improve readability and prevent unexpected content changes. +#rst_prolog = '' +rst_epilog = ''' +.. reference aliases +.. |network| replace:: :ref:`network` +.. |peer-to-peer network| replace:: :ref:`peer-to-peer network` + +.. frequently-used terms with links +.. |ECDSA| replace:: `ECDSA `__ +.. |secp256k1| replace:: `secp256k1 `__ + +.. OP codes +.. |OP_CHECKMULTISIG| replace:: :term:`"OP_CHECKMULTISIG" ` +.. |OP_CHECKSIG| replace:: :term:`"OP_CHECKSIG" ` +.. |OP_DUP| replace:: :term:`"OP_DUP" ` +.. |OP_EQUAL| replace:: :term:`"OP_EQUAL" ` +.. |OP_EQUALVERIFY| replace:: :term:`"OP_EQUALVERIFY" ` +.. |OP_HASH160| replace:: :term:`"OP_HASH160" ` +.. |OP_RETURN| replace:: :term:`"OP_RETURN" ` +.. |OP_VERIFY| replace:: :term:`"OP_VERIFY" ` + +.. other +.. |Bitcoin URI| replace:: :term:`"bitcoin:" URI ` +.. |Bitcoin URIs| replace:: :term:`"bitcoin:" URIs ` +.. |PaymentDetails| replace:: :term:`PaymentDetails` +.. |PaymentRequest| replace:: :term:`PaymentRequest` +.. |PaymentRequests| replace:: :term:`PaymentRequests ` +.. |r| replace:: :term:`"r" ` +''' diff --git a/devguide/block_chain.rst b/devguide/block_chain.rst index 4bbfbb3..51d7717 100644 --- a/devguide/block_chain.rst +++ b/devguide/block_chain.rst @@ -1,3 +1,5 @@ +.. _dev-block-chain-header: + Block Chain =========== @@ -6,14 +8,14 @@ The block chain provides Bitcoin's public ledger, an ordered and timestamped rec Introduction ------------ -Each full node in the Bitcoin `network <../devguide/p2p_network.html>`__ independently stores a block chain containing only blocks validated by that node. When several nodes all have the same blocks in their block chain, they are considered to be in :term:`consensus `. The validation rules these nodes follow to maintain consensus are called :term:`consensus rules `. This section describes many of the consensus rules used by Bitcoin Core. +Each full node in the Bitcoin |network| independently stores a block chain containing only blocks validated by that node. When several nodes all have the same blocks in their block chain, they are considered to be in :term:`consensus`. The validation rules these nodes follow to maintain consensus are called :term:`consensus rules`. This section describes many of the consensus rules used by Bitcoin Core. .. figure:: /img/dev/en-blockchain-overview.svg :alt: Block Chain Overview Block Chain Overview -The illustration above shows a simplified version of a block chain. A :term:`block ` of one or more new transactions is collected into the transaction data part of a block. Copies of each transaction are hashed, and the hashes are then paired, hashed, paired again, and hashed again until a single hash remains, the :term:`merkle root ` of a merkle tree. +The illustration above shows a simplified version of a block chain. A :term:`block` of one or more new transactions is collected into the transaction data part of a block. Copies of each transaction are hashed, and the hashes are then paired, hashed, paired again, and hashed again until a single hash remains, the :term:`merkle root` of a merkle tree. The merkle root is stored in the block header. Each block also stores the hash of the previous block’s header, chaining the blocks together. This ensures a transaction cannot be modified without modifying the block that records it and all following blocks. @@ -30,22 +32,22 @@ Outputs are tied to :term:`transaction identifiers (TXIDs) `, which are th Because each output of a particular transaction can only be spent once, the outputs of all transactions included in the block chain can be categorized as either :term:`Unspent Transaction Outputs (UTXOs) ` or spent transaction outputs. For a payment to be valid, it must only use UTXOs as inputs. -Ignoring coinbase transactions (described later), if the value of a transaction’s outputs exceed its inputs, the transaction will be rejected—but if the inputs exceed the value of the outputs, any difference in value may be claimed as a :term:`transaction fee ` by the Bitcoin :term:`miner ` who creates the block containing that transaction. For example, in the illustration above, each transaction spends 10,000 satoshis fewer than it receives from its combined inputs, effectively paying a 10,000 satoshi transaction fee. +Ignoring coinbase transactions (described later), if the value of a transaction’s outputs exceed its inputs, the transaction will be rejected—but if the inputs exceed the value of the outputs, any difference in value may be claimed as a :term:`transaction fee` by the Bitcoin :term:`miner` who creates the block containing that transaction. For example, in the illustration above, each transaction spends 10,000 satoshis fewer than it receives from its combined inputs, effectively paying a 10,000 satoshi transaction fee. Proof Of Work ------------- -The block chain is collaboratively maintained by anonymous peers on the `network <../devguide/p2p_network.html>`__, so Bitcoin requires that each block prove a significant amount of work was invested in its creation to ensure that untrustworthy peers who want to modify past blocks have to work harder than honest peers who only want to add new blocks to the block chain. +The block chain is collaboratively maintained by anonymous peers on the |network|, so Bitcoin requires that each block prove a significant amount of work was invested in its creation to ensure that untrustworthy peers who want to modify past blocks have to work harder than honest peers who only want to add new blocks to the block chain. Chaining blocks together makes it impossible to modify transactions included in any block without modifying all subsequent blocks. As a result, the cost to modify a particular block increases with every new block added to the block chain, magnifying the effect of the proof of work. -The :term:`proof of work ` used in Bitcoin takes advantage of the apparently random nature of cryptographic hashes. A good cryptographic hash algorithm converts arbitrary data into a seemingly random number. If the data is modified in any way and the hash re-run, a new seemingly random number is produced, so there is no way to modify the data to make the hash number predictable. +The :term:`proof of work` used in Bitcoin takes advantage of the apparently random nature of cryptographic hashes. A good cryptographic hash algorithm converts arbitrary data into a seemingly random number. If the data is modified in any way and the hash re-run, a new seemingly random number is produced, so there is no way to modify the data to make the hash number predictable. To prove you did some extra work to create a block, you must create a hash of the block header which does not exceed a certain value. For example, if the maximum possible hash value is 2256 − 1, you can prove that you tried up to two combinations by producing a hash value less than 2255. In the example given above, you will produce a successful hash on average every other try. You can even estimate the probability that a given hash attempt will generate a number below the :term:`target ` threshold. Bitcoin assumes a linear probability that the lower it makes the target threshold, the more hash attempts (on average) will need to be tried. -New blocks will only be added to the block chain if their hash is at least as challenging as a :term:`difficulty ` value expected by the consensus protocol. Every 2,016 blocks, the `network <../devguide/p2p_network.html>`__ uses timestamps stored in each block header to calculate the number of seconds elapsed between generation of the first and last of those last 2,016 blocks. The ideal value is 1,209,600 seconds (two weeks). +New blocks will only be added to the block chain if their hash is at least as challenging as a :term:`difficulty` value expected by the consensus protocol. Every 2,016 blocks, the |network| uses timestamps stored in each block header to calculate the number of seconds elapsed between generation of the first and last of those last 2,016 blocks. The ideal value is 1,209,600 seconds (two weeks). - If it took fewer than two weeks to generate the 2,016 blocks, the expected difficulty value is increased proportionally (by as much as 300%) so that the next 2,016 blocks should take exactly two weeks to generate if hashes are checked at the same rate. @@ -53,25 +55,25 @@ New blocks will only be added to the block chain if their hash is at least as ch (Note: an off-by-one error in the Bitcoin Core implementation causes the difficulty to be updated every 2,01\ *6* blocks using timestamps from only 2,01\ *5* blocks, creating a slight skew.) -Because each block header must hash to a value below the target threshold, and because each block is linked to the block that preceded it, it requires (on average) as much hashing power to propagate a modified block as the entire Bitcoin `network <../devguide/p2p_network.html>`__ expended between the time the original block was created and the present time. Only if you acquired a majority of the `network’s <../devguide/p2p_network.html>`__ hashing power could you reliably execute such a :term:`51 percent attack` against transaction history (although, it should be noted, that even less than 50% of the hashing power still has a good chance of performing such attacks). +Because each block header must hash to a value below the target threshold, and because each block is linked to the block that preceded it, it requires (on average) as much hashing power to propagate a modified block as the entire Bitcoin |network| expended between the time the original block was created and the present time. Only if you acquired a majority of the |network|’s hashing power could you reliably execute such a :term:`51 percent attack` against transaction history (although, it should be noted, that even less than 50% of the hashing power still has a good chance of performing such attacks). The block header provides several easy-to-modify fields, such as a dedicated nonce field, so obtaining new hashes doesn’t require waiting for new transactions. Also, only the 80-byte block header is hashed for proof-of-work, so including a large volume of transaction data in a block does not slow down hashing with extra I/O, and adding additional transaction data only requires the recalculation of the ancestor hashes in the merkle tree. Block Height And Forking ------------------------ -Any Bitcoin miner who successfully hashes a block header to a value below the target threshold can add the entire block to the block chain (assuming the block is otherwise valid). These blocks are commonly addressed by their :term:`block height `—the number of blocks between them and the first Bitcoin block (block 0, most commonly known as the :term:`genesis block `). For example, block 2016 is where difficulty could have first been adjusted. +Any Bitcoin miner who successfully hashes a block header to a value below the target threshold can add the entire block to the block chain (assuming the block is otherwise valid). These blocks are commonly addressed by their :term:`block height`—the number of blocks between them and the first Bitcoin block (block 0, most commonly known as the :term:`genesis block`). For example, block 2016 is where difficulty could have first been adjusted. .. figure:: /img/dev/en-blockchain-fork.svg :alt: Common And Uncommon Block Chain Forks Common And Uncommon Block Chain Forks -Multiple blocks can all have the same block height, as is common when two or more miners each produce a block at roughly the same time. This creates an apparent :term:`fork ` in the block chain, as shown in the illustration above. +Multiple blocks can all have the same block height, as is common when two or more miners each produce a block at roughly the same time. This creates an apparent :term:`fork` in the block chain, as shown in the illustration above. When miners produce simultaneous blocks at the end of the block chain, each node individually chooses which block to accept. In the absence of other considerations, discussed below, nodes usually use the first block they see. -Eventually a miner produces another block which attaches to only one of the competing simultaneously-mined blocks. This makes that side of the fork stronger than the other side. Assuming a fork only contains valid blocks, normal peers always follow the most difficult chain to recreate and throw away :term:`stale blocks ` belonging to shorter forks. (Stale blocks are also sometimes called orphans or orphan blocks, but those terms are also used for true orphan blocks without a known parent block.) +Eventually a miner produces another block which attaches to only one of the competing simultaneously-mined blocks. This makes that side of the fork stronger than the other side. Assuming a fork only contains valid blocks, normal peers always follow the most difficult chain to recreate and throw away |stale blocks| belonging to shorter forks. (|stale blocks| are also sometimes called orphans or |orphan blocks|, but those terms are also used for true |orphan blocks| without a known parent block.) Long-term forks are possible if different miners work at cross-purposes, such as some miners diligently working to extend the block chain at the same time other miners are attempting a 51 percent attack to revise transaction history. @@ -88,7 +90,7 @@ Blocks are not required to include any non-coinbase transactions, but miners alm All transactions, including the coinbase transaction, are encoded into blocks in binary raw transaction format. -The raw transaction format is hashed to create the transaction identifier (txid). From these txids, the :term:`merkle tree ` is constructed by pairing each txid with one other txid and then hashing them together. If there are an odd number of txids, the txid without a partner is hashed with a copy of itself. +The raw transaction format is hashed to create the transaction identifier (txid). From these txids, the :term:`merkle tree` is constructed by pairing each txid with one other txid and then hashing them together. If there are an odd number of txids, the txid without a partner is hashed with a copy of itself. The resulting hashes themselves are each paired with one other hash and hashed together. Any hash without a partner is hashed with itself. The process repeats until only one hash remains, the merkle root. @@ -113,20 +115,20 @@ Note: If identical txids are found within the same block, there is a possibility Consensus Rule Changes ---------------------- -To maintain consensus, all full nodes validate blocks using the same consensus rules. However, sometimes the consensus rules are changed to introduce new features or prevent `network <../devguide/p2p_network.html>`__ abuse. When the new rules are implemented, there will likely be a period of time when non-upgraded nodes follow the old rules and upgraded nodes follow the new rules, creating two possible ways consensus can break: +To maintain consensus, all full nodes validate blocks using the same consensus rules. However, sometimes the consensus rules are changed to introduce new features or prevent |network| abuse. When the new rules are implemented, there will likely be a period of time when non-upgraded nodes follow the old rules and upgraded nodes follow the new rules, creating two possible ways consensus can break: 1. A block following the new consensus rules is accepted by upgraded nodes but rejected by non-upgraded nodes. For example, a new transaction feature is used within a block: upgraded nodes understand the feature and accept it, but non-upgraded nodes reject it because it violates the old rules. 2. A block violating the new consensus rules is rejected by upgraded nodes but accepted by non-upgraded nodes. For example, an abusive transaction feature is used within a block: upgraded nodes reject it because it violates the new rules, but non-upgraded nodes accept it because it follows the old rules. -In the first case, rejection by non-upgraded nodes, mining software which gets block chain data from those non-upgraded nodes refuses to build on the same chain as mining software getting data from upgraded nodes. This creates permanently divergent chains—one for non-upgraded nodes and one for upgraded nodes—called a :term:`hard fork `. +In the first case, rejection by non-upgraded nodes, mining software which gets block chain data from those non-upgraded nodes refuses to build on the same chain as mining software getting data from upgraded nodes. This creates permanently divergent chains—one for non-upgraded nodes and one for upgraded nodes—called a :term:`hard fork`. .. figure:: /img/dev/en-hard-fork.svg :alt: Hard Fork Hard Fork -In the second case, rejection by upgraded nodes, it’s possible to keep the block chain from permanently diverging if upgraded nodes control a majority of the hash rate. That’s because, in this case, non-upgraded nodes will accept as valid all the same blocks as upgraded nodes, so the upgraded nodes can build a stronger chain that the non-upgraded nodes will accept as the best valid block chain. This is called a :term:`soft fork `. +In the second case, rejection by upgraded nodes, it’s possible to keep the block chain from permanently diverging if upgraded nodes control a majority of the hash rate. That’s because, in this case, non-upgraded nodes will accept as valid all the same blocks as upgraded nodes, so the upgraded nodes can build a stronger chain that the non-upgraded nodes will accept as the best valid block chain. This is called a :term:`soft fork`. .. figure:: /img/dev/en-soft-fork.svg :alt: Soft Fork @@ -135,9 +137,9 @@ In the second case, rejection by upgraded nodes, it’s possible to keep the blo Although a fork is an actual divergence in block chains, changes to the consensus rules are often described by their potential to create either a hard or soft fork. For example, “increasing the block size above 1 MB requires a hard fork.” In this example, an actual block chain fork is not required—but it is a possible outcome. -Consensus rule changes may be activated in various ways. During Bitcoin’s first two years, Satoshi Nakamoto performed several soft forks by just releasing the backwards-compatible change in a client that began immediately enforcing the new rule. Multiple soft forks such as `BIP30 `__ have been activated via a flag day where the new rule began to be enforced at a preset time or block height. Such forks activated via a flag day are known as :term:`User Activated Soft Forks ` (UASF) as they are dependent on having sufficient users (nodes) to enforce the new rules after the flag day. +Consensus rule changes may be activated in various ways. During Bitcoin’s first two years, Satoshi Nakamoto performed several soft forks by just releasing the backwards-compatible change in a client that began immediately enforcing the new rule. Multiple soft forks such as `BIP30 `__ have been activated via a flag day where the new rule began to be enforced at a preset time or block height. Such forks activated via a flag day are known as :term:`User-Activated Soft Forks ` (UASF) as they are dependent on having sufficient users (nodes) to enforce the new rules after the flag day. -Later soft forks waited for a majority of hash rate (typically 75% or 95%) to signal their readiness for enforcing the new consensus rules. Once the signalling threshold has been passed, all nodes will begin enforcing the new rules. Such forks are known as :term:`Miner Activated Soft Forks ` (MASF) as they are dependent on miners for activation. +Later soft forks waited for a majority of hash rate (typically 75% or 95%) to signal their readiness for enforcing the new consensus rules. Once the signalling threshold has been passed, all nodes will begin enforcing the new rules. Such forks are known as :term:`Miner-Activated Soft Forks ` (MASF) as they are dependent on miners for activation. **Resources:** `BIP16 `__, `BIP30 `__, and `BIP34 `__ were implemented as changes which might have lead to soft forks. `BIP50 `__ describes both an accidental hard fork, resolved by temporary downgrading the capabilities of upgraded nodes, and an intentional hard fork when the temporary downgrade was removed. A document from Gavin Andresen outlines `how future rule changes may be implemented `__. @@ -152,6 +154,10 @@ Full nodes can also check block and transaction version numbers. If the block or In either case, block and transaction data should not be relied upon if it comes from a node that apparently isn’t using the current consensus rules. -SPV clients which connect to full nodes can detect a likely hard fork by connecting to several full nodes and ensuring that they’re all on the same chain with the same block height, plus or minus several blocks to account for transmission delays and stale blocks. If there’s a divergence, the client can disconnect from nodes with weaker chains. +SPV clients which connect to full nodes can detect a likely hard fork by connecting to several full nodes and ensuring that they’re all on the same chain with the same block height, plus or minus several blocks to account for transmission delays and |stale blocks| +. If there’s a divergence, the client can disconnect from nodes with weaker chains. + +SPV clients should also monitor for block and :term:`transaction version number` increases to ensure they process received transactions and create new transactions using the current consensus rules. -SPV clients should also monitor for block and :ref:`transaction version number ` increases to ensure they process received transactions and create new transactions using the current consensus rules. +.. |stale blocks| replace:: :term:`stale blocks ` +.. |orphan blocks| replace:: :term:`orphan blocks ` \ No newline at end of file diff --git a/devguide/contracts.rst b/devguide/contracts.rst index 77832b4..db8709a 100644 --- a/devguide/contracts.rst +++ b/devguide/contracts.rst @@ -1,3 +1,5 @@ +.. _dev-contracts-header: + Contracts ========= @@ -17,9 +19,9 @@ Charlie-the-customer wants to buy a product from Bob-the-businessman, but neithe A simple contract could say that Charlie will spend satoshis to an output which can only be spent if Charlie and Bob both sign the input spending it. That means Bob won’t get paid unless Charlie gets his merchandise, but Charlie can’t get the merchandise and keep his payment. -This simple contract isn’t much help if there’s a dispute, so Bob and Charlie enlist the help of Alice-the-arbitrator to create an :term:`escrow contract `. Charlie spends his satoshis to an output which can only be spent if two of the three people sign the input. Now Charlie can pay Bob if everything is ok, Bob can `refund <../devguide/payment_processing.html#issuing-refunds>`__ Charlie’s money if there’s a problem, or Alice can arbitrate and decide who should get the satoshis if there’s a dispute. +This simple contract isn’t much help if there’s a dispute, so Bob and Charlie enlist the help of Alice-the-arbitrator to create an :term:`escrow contract`. Charlie spends his satoshis to an output which can only be spent if two of the three people sign the input. Now Charlie can pay Bob if everything is ok, Bob can `refund <../devguide/payment_processing.html#issuing-refunds>`_ Charlie’s money if there’s a problem, or Alice can arbitrate and decide who should get the satoshis if there’s a dispute. -To create a multiple-signature (:term:`multisig `) output, they each give the others a public key. Then Bob creates the following :term:`P2SH multisig ` redeem script: +To create a multiple-signature (:term:`multisig`) output, they each give the others a public key. Then Bob creates the following :term:`P2SH multisig` redeem script: :: @@ -31,9 +33,9 @@ To create a multiple-signature (:term:`multisig `) output, they each g Bob gives the redeem script to Charlie, who checks to make sure his public key and Alice’s public key are included. Then he hashes the redeem script to create a P2SH redeem script and pays the satoshis to it. Bob sees the payment get added to the block chain and ships the merchandise. -Unfortunately, the merchandise gets slightly damaged in transit. Charlie wants a full `refund <../devguide/payment_processing.html#issuing-refunds>`__, but Bob thinks a 10% `refund <../devguide/payment_processing.html#issuing-refunds>`__ is sufficient. They turn to Alice to resolve the issue. Alice asks for photo evidence from Charlie along with a copy of the redeem script Bob created and Charlie checked. +Unfortunately, the merchandise gets slightly damaged in transit. Charlie wants a full refund_, but Bob thinks a 10% refund_ is sufficient. They turn to Alice to resolve the issue. Alice asks for photo evidence from Charlie along with a copy of the redeem script Bob created and Charlie checked. -After looking at the evidence, Alice thinks a 40% `refund <../devguide/payment_processing.html#issuing-refunds>`__ is sufficient, so she creates and signs a transaction with two outputs, one that spends 60% of the satoshis to Bob’s public key and one that spends the remaining 40% to Charlie’s public key. +After looking at the evidence, Alice thinks a 40% refund_ is sufficient, so she creates and signs a transaction with two outputs, one that spends 60% of the satoshis to Bob’s public key and one that spends the remaining 40% to Charlie’s public key. In the signature script Alice puts her signature and a copy of the unhashed serialized redeem script that Bob created. She gives a copy of the incomplete transaction to both Bob and Charlie. Either one of them can complete it by adding his signature to create the following signature script: @@ -41,9 +43,9 @@ In the signature script Alice puts her signature and a copy of the unhashed seri OP_0 [A's signature] [B's or C's signature] [serialized redeem script] -(Opcodes to push the signatures and redeem script onto the stack are not shown. ``OP_0`` is a workaround for an off-by-one error in the original implementation which must be preserved for compatibility. Note that the signature script must provide signatures in the same order as the corresponding public keys appear in the redeem script. See the description in :ref:`“OP_CHECKMULTISIG” ` for details.) +(Opcodes to push the signatures and redeem script onto the stack are not shown. ``OP_0`` is a workaround for an off-by-one error in the original implementation which must be preserved for compatibility. Note that the signature script must provide signatures in the same order as the corresponding public keys appear in the redeem script. See the description in |OP_CHECKMULTISIG| for details.) -When the transaction is broadcast to the `network <../devguide/p2p_network.html>`__, each peer checks the signature script against the P2SH output Charlie previously paid, ensuring that the redeem script matches the redeem script hash previously provided. Then the redeem script is evaluated, with the two signatures being used as input data. Assuming the redeem script validates, the two transaction outputs show up in Bob’s and Charlie’s wallets as spendable balances. +When the transaction is broadcast to the |network|, each peer checks the signature script against the P2SH output Charlie previously paid, ensuring that the redeem script matches the redeem script hash previously provided. Then the redeem script is evaluated, with the two signatures being used as input data. Assuming the redeem script validates, the two transaction outputs show up in Bob’s and Charlie’s wallets as spendable balances. However, if Alice created and signed a transaction neither of them would agree to, such as spending all the satoshis to herself, Bob and Charlie can find a new arbitrator and sign a transaction spending the satoshis to another 2-of-3 multisig redeem script hash, this one including a public key from that second arbitrator. This means that Bob and Charlie never need to worry about their arbitrator stealing their money. @@ -52,26 +54,26 @@ However, if Alice created and signed a transaction neither of them would agree t Micropayment Channel -------------------- -Alice also works part time moderating forum posts for Bob. Every time someone posts to Bob’s busy forum, Alice skims the post to make sure it isn’t offensive or spam. Alas, Bob often forgets to pay her, so Alice demands to be paid immediately after each post she approves or rejects. Bob says he can’t do that because hundreds of small payments will cost him thousands of satoshis in transaction fees, so Alice suggests they use a :ref:`micropayment channel `. +Alice also works part time moderating forum posts for Bob. Every time someone posts to Bob’s busy forum, Alice skims the post to make sure it isn’t offensive or spam. Alas, Bob often forgets to pay her, so Alice demands to be paid immediately after each post she approves or rejects. Bob says he can’t do that because hundreds of small payments will cost him thousands of satoshis in transaction fees, so Alice suggests they use a :term:`micropayment channel`. Bob asks Alice for her public key and then creates two transactions. The first transaction pays 100 millibitcoins to a P2SH output whose 2-of-2 multisig redeem script requires signatures from both Alice and Bob. This is the bond transaction. Broadcasting this transaction would let Alice hold the millibitcoins hostage, so Bob keeps this transaction private for now and creates a second transaction. -The second transaction spends all of the first transaction’s millibitcoins (minus a transaction fee) back to Bob after a 24 hour delay enforced by locktime. This is the `refund <../devguide/payment_processing.html#issuing-refunds>`__ transaction. Bob can’t sign the `refund <../devguide/payment_processing.html#issuing-refunds>`__ transaction by himself, so he gives it to Alice to sign, as shown in the illustration below. +The second transaction spends all of the first transaction’s millibitcoins (minus a transaction fee) back to Bob after a 24 hour delay enforced by locktime. This is the refund_ transaction. Bob can’t sign the refund_ transaction by himself, so he gives it to Alice to sign, as shown in the illustration below. .. figure:: /img/dev/en-micropayment-channel.svg :alt: Micropayment Channel Example Micropayment Channel Example -Alice checks that the `refund <../devguide/payment_processing.html#issuing-refunds>`__ transaction’s locktime is 24 hours in the future, signs it, and gives a copy of it back to Bob. She then asks Bob for the bond transaction and checks that the `refund <../devguide/payment_processing.html#issuing-refunds>`__ transaction spends the output of the bond transaction. She can now broadcast the bond transaction to the `network <../devguide/p2p_network.html>`__ to ensure Bob has to wait for the time lock to expire before further spending his millibitcoins. Bob hasn’t actually spent anything so far, except possibly a small transaction fee, and he’ll be able to broadcast the `refund <../devguide/payment_processing.html#issuing-refunds>`__ transaction in 24 hours for a full `refund <../devguide/payment_processing.html#issuing-refunds>`__. +Alice checks that the refund_ transaction’s locktime is 24 hours in the future, signs it, and gives a copy of it back to Bob. She then asks Bob for the bond transaction and checks that the refund_ transaction spends the output of the bond transaction. She can now broadcast the bond transaction to the |network| to ensure Bob has to wait for the time lock to expire before further spending his millibitcoins. Bob hasn’t actually spent anything so far, except possibly a small transaction fee, and he’ll be able to broadcast the refund_ transaction in 24 hours for a full refund_. -Now, when Alice does some work worth 1 millibitcoin, she asks Bob to create and sign a new version of the `refund <../devguide/payment_processing.html#issuing-refunds>`__ transaction. Version two of the transaction spends 1 millibitcoin to Alice and the other 99 back to Bob; it does not have a locktime, so Alice can sign it and spend it whenever she wants. (But she doesn’t do that immediately.) +Now, when Alice does some work worth 1 millibitcoin, she asks Bob to create and sign a new version of the refund_ transaction. Version two of the transaction spends 1 millibitcoin to Alice and the other 99 back to Bob; it does not have a locktime, so Alice can sign it and spend it whenever she wants. (But she doesn’t do that immediately.) -Alice and Bob repeat these work-and-pay steps until Alice finishes for the day, or until the time lock is about to expire. Alice signs the final version of the `refund <../devguide/payment_processing.html#issuing-refunds>`__ transaction and broadcasts it, paying herself and refunding any remaining balance to Bob. The next day, when Alice starts work, they create a new :ref:`micropayment channel `. +Alice and Bob repeat these work-and-pay steps until Alice finishes for the day, or until the time lock is about to expire. Alice signs the final version of the refund_ transaction and broadcasts it, paying herself and refunding any remaining balance to Bob. The next day, when Alice starts work, they create a new :term:`micropayment channel`. -If Alice fails to broadcast a version of the `refund <../devguide/payment_processing.html#issuing-refunds>`__ transaction before its time lock expires, Bob can broadcast the first version and receive a full `refund <../devguide/payment_processing.html#issuing-refunds>`__. This is one reason :ref:`micropayment channels ` are best suited to small payments—if Alice’s Internet service goes out for a few hours near the time lock expiry, she could be cheated out of her payment. +If Alice fails to broadcast a version of the refund_ transaction before its time lock expires, Bob can broadcast the first version and receive a full refund_. This is one reason :term:`micropayment channels ` are best suited to small payments—if Alice’s Internet service goes out for a few hours near the time lock expiry, she could be cheated out of her payment. -Transaction malleability, discussed above in the Transactions section, is another reason to limit the value of :ref:`micropayment channels `. If someone uses transaction malleability to break the link between the two transactions, Alice could hold Bob’s 100 millibitcoins hostage even if she hadn’t done any work. +Transaction malleability, discussed above in the Transactions section, is another reason to limit the value of :term:`micropayment channels `. If someone uses transaction malleability to break the link between the two transactions, Alice could hold Bob’s 100 millibitcoins hostage even if she hadn’t done any work. For larger payments, Bitcoin transaction fees are very low as a percentage of the total transaction value, so it makes more sense to protect payments with immediately-broadcast separate transactions. diff --git a/devguide/index.rst b/devguide/index.rst index ec7c71a..c79aa7b 100644 --- a/devguide/index.rst +++ b/devguide/index.rst @@ -1,3 +1,5 @@ +.. _dev-index-header: + Developer Guides ================ diff --git a/devguide/mining.rst b/devguide/mining.rst index b28fa47..d9ba509 100644 --- a/devguide/mining.rst +++ b/devguide/mining.rst @@ -1,3 +1,5 @@ +.. _dev-mining-header: + Mining ====== @@ -15,7 +17,7 @@ Mining today takes on two forms: Solo Mining ----------- -As illustrated below, solo miners typically use ``bitcoind`` to get new transactions from the `network <../devguide/p2p_network.html>`__. Their mining software periodically polls ``bitcoind`` for new transactions using the `“getblocktemplate” RPC <../reference/rpc/getblocktemplate.html>`__, which provides the list of new transactions plus the public key to which the coinbase transaction should be sent. +As illustrated below, solo miners typically use ``bitcoind`` to get new transactions from the |network|. Their mining software periodically polls ``bitcoind`` for new transactions using the `“getblocktemplate” RPC <../reference/rpc/getblocktemplate.html>`__, which provides the list of new transactions plus the public key to which the coinbase transaction should be sent. .. figure:: /img/dev/en-solo-mining-overview.svg :alt: Solo Bitcoin Mining @@ -26,25 +28,25 @@ The mining software constructs a block using the template (described below) and If none of the hashes are below the threshold, the mining hardware gets an updated block header with a new merkle root from the mining software; this new block header is created by adding extra nonce data to the coinbase field of the coinbase transaction. -On the other hand, if a hash is found below the target threshold, the mining hardware returns the block header with the successful nonce to the mining software. The mining software combines the header with the block and sends the completed block to ``bitcoind`` to be broadcast to the `network <../devguide/p2p_network.html>`__ for addition to the block chain. +On the other hand, if a hash is found below the target threshold, the mining hardware returns the block header with the successful nonce to the mining software. The mining software combines the header with the block and sends the completed block to ``bitcoind`` to be broadcast to the |network| for addition to the block chain. Pool Mining ----------- -Pool miners follow a similar workflow, illustrated below, which allows mining pool operators to pay miners based on their share of the work done. The mining pool gets new transactions from the `network <../devguide/p2p_network.html>`__ using ``bitcoind``. Using one of the methods discussed later, each miner’s mining software connects to the pool and requests the information it needs to construct block headers. +Pool miners follow a similar workflow, illustrated below, which allows mining pool operators to pay miners based on their share of the work done. The mining pool gets new transactions from the |network| using ``bitcoind``. Using one of the methods discussed later, each miner’s mining software connects to the pool and requests the information it needs to construct block headers. .. figure:: /img/dev/en-pooled-mining-overview.svg :alt: Pooled Bitcoin Mining Pooled Bitcoin Mining -In pooled mining, the mining pool sets the target threshold a few orders of magnitude higher (less difficult) than the `network <../devguide/p2p_network.html>`__ difficulty. This causes the mining hardware to return many block headers which don’t hash to a value eligible for inclusion on the block chain but which do hash below the pool’s target, proving (on average) that the miner checked a percentage of the possible hash values. +In pooled mining, the mining pool sets the target threshold a few orders of magnitude higher (less difficult) than the |network| difficulty. This causes the mining hardware to return many block headers which don’t hash to a value eligible for inclusion on the block chain but which do hash below the pool’s target, proving (on average) that the miner checked a percentage of the possible hash values. The miner then sends to the pool a copy of the information the pool needs to validate that the header will hash below the target and that the block of transactions referred to by the header merkle root field is valid for the pool’s purposes. (This usually means that the coinbase transaction must pay the pool.) -The information the miner sends to the pool is called a share because it proves the miner did a share of the work. By chance, some shares the pool receives will also be below the `network <../devguide/p2p_network.html>`__ target—the mining pool sends these to the `network <../devguide/p2p_network.html>`__ to be added to the block chain. +The information the miner sends to the pool is called a share because it proves the miner did a share of the work. By chance, some shares the pool receives will also be below the |network| target—the mining pool sends these to the |network| to be added to the block chain. -The block reward and transaction fees that come from mining that block are paid to the mining pool. The mining pool pays out a portion of these proceeds to individual miners based on how many shares they generated. For example, if the mining pool’s target threshold is 100 times lower than the `network <../devguide/p2p_network.html>`__ target threshold, 100 shares will need to be generated on average to create a successful block, so the mining pool can pay 1/100th of its payout for each share received. Different mining pools use different reward distribution systems based on this basic share system. +The block reward and transaction fees that come from mining that block are paid to the mining pool. The mining pool pays out a portion of these proceeds to individual miners based on how many shares they generated. For example, if the mining pool’s target threshold is 100 times lower than the |network| target threshold, 100 shares will need to be generated on average to create a successful block, so the mining pool can pay 1/100th of its payout for each share received. Different mining pools use different reward distribution systems based on this basic share system. Block Prototypes ---------------- @@ -67,7 +69,7 @@ An improved method is the Bitcoin Core `“getblocktemplate” RPC <../reference 3. Other information necessary to construct a block header for the next block: the block version, previous block hash, and bits (target). -4. The mining pool’s current target threshold for accepting shares. (For solo miners, this is the `network <../devguide/p2p_network.html>`__ target.) +4. The mining pool’s current target threshold for accepting shares. (For solo miners, this is the |network| target.) Using the transactions received, the mining software adds a nonce to the coinbase extra nonce field and then converts all the transactions into a merkle tree to derive a merkle root it can use in a block header. Whenever the extra nonce field needs to be changed, the mining software rebuilds the necessary parts of the merkle tree and updates the time and merkle root fields in the block header. diff --git a/devguide/operating_modes.rst b/devguide/operating_modes.rst index ec6eb3f..f2462fa 100644 --- a/devguide/operating_modes.rst +++ b/devguide/operating_modes.rst @@ -1,3 +1,5 @@ +.. _dev-operating-modes-header: + Operating Modes =============== @@ -11,9 +13,9 @@ Currently there are two primary methods of validating the block chain as a clien Full Node --------- -The first and most secure model is the one followed by Bitcoin Core, also known as a “thick” or “full chain” client. This security model assures the validity of the block chain by downloading and validating blocks from the genesis block all the way to the most recently discovered block. This is known as using the *height* of a particular block to verify the client’s view of the `network <../devguide/p2p_network.html>`__. +The first and most secure model is the one followed by Bitcoin Core, also known as a “thick” or “full chain” client. This security model assures the validity of the block chain by downloading and validating blocks from the genesis block all the way to the most recently discovered block. This is known as using the *height* of a particular block to verify the client’s view of the |network|. -For a client to be fooled, an adversary would need to give a complete alternative block chain history that is of greater difficulty than the current “true” chain, which is computationally expensive (if not impossible) due to the fact that the chain with the most cumulative proof of work is by definition the “true” chain. Due to the computational difficulty required to generate a new block at the tip of the chain, the ability to fool a full node becomes very expensive after 6 confirmations. This form of verification is highly resistent to sybil attacks—only a single honest `network <../devguide/p2p_network.html>`__ peer is required in order to receive and verify the complete state of the “true” block chain. +For a client to be fooled, an adversary would need to give a complete alternative block chain history that is of greater difficulty than the current “true” chain, which is computationally expensive (if not impossible) due to the fact that the chain with the most cumulative proof of work is by definition the “true” chain. Due to the computational difficulty required to generate a new block at the tip of the chain, the ability to fool a full node becomes very expensive after 6 confirmations. This form of verification is highly resistent to sybil attacks—only a single honest |network| peer is required in order to receive and verify the complete state of the “true” block chain. .. figure:: /img/dev/en-block-height-vs-depth.svg :alt: Block Height Compared To Block Depth @@ -34,7 +36,7 @@ Potential SPV Weaknesses If implemented naively, an SPV client has a few important weaknesses. -First, while the SPV client can not be easily fooled into thinking a transaction is in a block when it is not, the reverse is not true. A full node can simply lie by omission, leading an SPV client to believe a transaction has not occurred. This can be considered a form of Denial of Service. One mitigation strategy is to connect to a number of full nodes, and send the requests to each node. However this can be defeated by `network <../devguide/p2p_network.html>`__ partitioning or Sybil attacks, since identities are essentially free, and can be bandwidth intensive. Care must be taken to ensure the client is not cut off from honest nodes. +First, while the SPV client can not be easily fooled into thinking a transaction is in a block when it is not, the reverse is not true. A full node can simply lie by omission, leading an SPV client to believe a transaction has not occurred. This can be considered a form of Denial of Service. One mitigation strategy is to connect to a number of full nodes, and send the requests to each node. However this can be defeated by |network| partitioning or Sybil attacks, since identities are essentially free, and can be bandwidth intensive. Care must be taken to ensure the client is not cut off from honest nodes. Second, the SPV client only requests transactions from full nodes corresponding to keys it owns. If the SPV client downloads all blocks and then discards unneeded ones, this can be extremely bandwidth intensive. If they simply ask full nodes for blocks with specific transactions, this allows full nodes a complete view of the public addresses that correspond to the user. This is a large privacy leak, and allows for tactics such as denial of service for clients, users, or addresses that are disfavored by those running full nodes, as well as trivial linking of funds. A client could simply spam many fake transaction requests, but this creates a large strain on the SPV client, and can end up defeating the purpose of thin clients altogether. @@ -69,7 +71,7 @@ Bloom filters were standardized for use via `BIP37 `__. +There are future proposals such as Unspent Transaction Output (UTXO) commitments in the block chain to find a more satisfactory middle-ground for clients between needing a complete copy of the block chain, or trusting that a majority of your connected peers are not lying. UTXO commitments would enable a very secure client using a finite amount of storage using a data structure that is authenticated in the block chain. These type of proposals are, however, in very early stages, and will require soft forks in the |network|. Until these types of operating modes are implemented, modes should be chosen based on the likely threat model, computing and bandwidth constraints, and liability in bitcoin value. diff --git a/devguide/p2p_network.rst b/devguide/p2p_network.rst index 66896cd..06102c4 100644 --- a/devguide/p2p_network.rst +++ b/devguide/p2p_network.rst @@ -1,3 +1,5 @@ +.. _dev-p2p-network-header: + P2P Network =========== @@ -6,7 +8,7 @@ The Bitcoin network protocol allows full nodes (peers) to collaboratively mainta Introduction ------------ -Full nodes download and verify every block and transaction prior to relaying them to other nodes. Archival nodes are full nodes which store the entire blockchain and can serve historical blocks to other nodes. Pruned nodes are full nodes which do not store the entire blockchain. Many SPV clients also use the Bitcoin `network <../devguide/p2p_network.html>`__ protocol to connect to full nodes. +Full nodes download and verify every block and transaction prior to relaying them to other nodes. Archival nodes are full nodes which store the entire blockchain and can serve historical blocks to other nodes. Pruned nodes are full nodes which do not store the entire blockchain. Many SPV clients also use the Bitcoin |network| protocol to connect to full nodes. Consensus rules do not cover networking, so Bitcoin programs may use alternative networks and protocols, such as the `high-speed block relay network `__ used by some miners and the `dedicated transaction information servers `__ used by some wallets that provide SPV-level security. @@ -28,13 +30,13 @@ When started for the first time, programs don’t know the IP addresses of any a seed.bitcoin.sipa.be. 60 IN A 203.0.113.183 [...] -The DNS seeds are maintained by Bitcoin community members: some of them provide dynamic DNS seed servers which automatically get IP addresses of active nodes by scanning the `network <../devguide/p2p_network.html>`__; others provide static DNS seeds that are updated manually and are more likely to provide IP addresses for inactive nodes. In either case, nodes are added to the DNS seed if they run on the default Bitcoin ports of 8333 for mainnet or 18333 for testnet. +The DNS seeds are maintained by Bitcoin community members: some of them provide dynamic DNS seed servers which automatically get IP addresses of active nodes by scanning the |network|; others provide static DNS seeds that are updated manually and are more likely to provide IP addresses for inactive nodes. In either case, nodes are added to the DNS seed if they run on the default Bitcoin ports of 8333 for mainnet or 18333 for testnet. -DNS seed results are not authenticated and a malicious seed operator or `network <../devguide/p2p_network.html>`__ `man-in-the-middle `__ attacker can return only IP addresses of nodes controlled by the attacker, isolating a program on the attacker’s own `network <../devguide/p2p_network.html>`__ and allowing the attacker to feed it bogus transactions and blocks. For this reason, programs should not rely on DNS seeds exclusively. +DNS seed results are not authenticated and a malicious seed operator or |network| `man-in-the-middle `__ attacker can return only IP addresses of nodes controlled by the attacker, isolating a program on the attacker’s own |network| and allowing the attacker to feed it bogus transactions and blocks. For this reason, programs should not rely on DNS seeds exclusively. -Once a program has connected to the `network <../devguide/p2p_network.html>`__, its peers can begin to send it ``addr`` (address) messages with the IP addresses and port numbers of other peers on the `network <../devguide/p2p_network.html>`__, providing a fully decentralized method of peer discovery. Bitcoin Core keeps a record of known peers in a persistent on-disk database which usually allows it to connect directly to those peers on subsequent startups without having to use DNS seeds. +Once a program has connected to the |network|, its peers can begin to send it ``addr`` (address) messages with the IP addresses and port numbers of other peers on the |network|, providing a fully decentralized method of peer discovery. Bitcoin Core keeps a record of known peers in a persistent on-disk database which usually allows it to connect directly to those peers on subsequent startups without having to use DNS seeds. -However, peers often leave the `network <../devguide/p2p_network.html>`__ or change IP addresses, so programs may need to make several different connection attempts at startup before a successful connection is made. This can add a significant delay to the amount of time it takes to connect to the `network <../devguide/p2p_network.html>`__, forcing a user to wait before sending a transaction or checking the status of payment. +However, peers often leave the |network| or change IP addresses, so programs may need to make several different connection attempts at startup before a successful connection is made. This can add a significant delay to the amount of time it takes to connect to the |network|, forcing a user to wait before sending a transaction or checking the status of payment. To avoid this possible delay, `BitcoinJ `__ always uses dynamic DNS seeds to get IP addresses for nodes believed to be currently active. Bitcoin Core also tries to strike a balance between minimizing delays and avoiding unnecessary DNS seed use: if Bitcoin Core has entries in its peer database, it spends up to 11 seconds attempting to connect to at least one of them before falling back to seeds; if a connection is made within that time, it does not query any seeds. @@ -81,14 +83,14 @@ The first time a node is started, it only has a single block in its local best b In the header hashes field of the `“getblocks” message <../reference/p2p_networking.html#getblocks>`__, this new node sends the header hash of the only block it has, the genesis block (6fe2…0000 in internal byte order). It also sets the stop hash field to all zeroes to request a maximum-size response. -Upon :ref:`receipt ` of the `“getblocks” message <../reference/p2p_networking.html#getblocks>`__, the sync node takes the first (and only) header hash and searches its local best block chain for a block with that header hash. It finds that block 0 matches, so it replies with 500 block inventories (the maximum response to a `“getblocks” message <../reference/p2p_networking.html#getblocks>`__) starting from block 1. It sends these inventories in the `“inv” message <../reference/p2p_networking.html#inv>`__ illustrated below. +Upon :term:`receipt` of the `“getblocks” message <../reference/p2p_networking.html#getblocks>`__, the sync node takes the first (and only) header hash and searches its local best block chain for a block with that header hash. It finds that block 0 matches, so it replies with 500 block inventories (the maximum response to a `“getblocks” message <../reference/p2p_networking.html#getblocks>`__) starting from block 1. It sends these inventories in the `“inv” message <../reference/p2p_networking.html#inv>`__ illustrated below. .. figure:: /img/dev/en-ibd-inv.svg :alt: First Inv Message Sent During IBD First Inv Message Sent During IBD -Inventories are unique identifiers for information on the `network <../devguide/p2p_network.html>`__. Each inventory contains a type field and the unique identifier for an instance of the object. For blocks, the unique identifier is a hash of the block’s header. +Inventories are unique identifiers for information on the |network|. Each inventory contains a type field and the unique identifier for an instance of the object. For blocks, the unique identifier is a hash of the block’s header. The block inventories appear in the `“inv” message <../reference/p2p_networking.html#inv>`__ in the same order they appear in the block chain, so this first `“inv” message <../reference/p2p_networking.html#inv>`__ contains inventories for blocks 1 through 501. (For example, the hash of block 1 is 4860…0000 as seen in the illustration above.) @@ -101,7 +103,7 @@ The IBD node uses the received inventories to request 128 blocks from the sync n It’s important to blocks-first nodes that the blocks be requested and sent in order because each block header references the header hash of the preceding block. That means the IBD node can’t fully validate a block until its parent block has been received. Blocks that can’t be validated because their parents haven’t been received are called orphan blocks; a subsection below describes them in more detail. -Upon :ref:`receipt ` of the `“getdata” message <../reference/p2p_networking.html#getdata>`__, the sync node replies with each of the blocks requested. Each block is put into serialized block format and sent in a separate `“block” message <../reference/p2p_networking.html#block>`__. The first `“block” message <../reference/p2p_networking.html#block>`__ sent (for block 1) is illustrated below. +Upon :term:`receipt` of the `“getdata” message <../reference/p2p_networking.html#getdata>`__, the sync node replies with each of the blocks requested. Each block is put into serialized block format and sent in a separate `“block” message <../reference/p2p_networking.html#block>`__. The first `“block” message <../reference/p2p_networking.html#block>`__ sent (for block 1) is illustrated below. .. figure:: /img/dev/en-ibd-block.svg :alt: First Block Message Sent During IBD @@ -115,7 +117,7 @@ The IBD node downloads each block, validates it, and then requests the next bloc Second GetBlocks Message Sent During IBD -Upon :ref:`receipt ` of the second `“getblocks” message <../reference/p2p_networking.html#getblocks>`__, the sync node searches its local best block chain for a block that matches one of the header hashes in the message, trying each hash in the order they were received. If it finds a matching hash, it replies with 500 block inventories starting with the next block from that point. But if there is no matching hash (besides the stopping hash), it assumes the only block the two nodes have in common is block 0 and so it sends an ``inv`` starting with block 1 (the same `“inv” message <../reference/p2p_networking.html#inv>`__ seen several illustrations above). +Upon :term:`receipt` of the second `“getblocks” message <../reference/p2p_networking.html#getblocks>`__, the sync node searches its local best block chain for a block that matches one of the header hashes in the message, trying each hash in the order they were received. If it finds a matching hash, it replies with 500 block inventories starting with the next block from that point. But if there is no matching hash (besides the stopping hash), it assumes the only block the two nodes have in common is block 0 and so it sends an ``inv`` starting with block 1 (the same `“inv” message <../reference/p2p_networking.html#inv>`__ seen several illustrations above). This repeated search allows the sync node to send useful inventories even if the IBD node’s local block chain forked from the sync node’s local block chain. This fork detection becomes increasingly useful the closer the IBD node gets to the tip of the block chain. @@ -155,7 +157,7 @@ All of these problems are addressed in part or in full by the headers-first IBD Headers-First ~~~~~~~~~~~~~ -`Bitcoin Core 0.10.0 `__ uses an initial block download (IBD) method called *headers-first*. The goal is to download the headers for the best :term:`header chain
`, partially validate them as best as possible, and then download the corresponding blocks in parallel. This solves several problems with the older blocks-first IBD method. +`Bitcoin Core 0.10.0 `__ uses an initial block download (IBD) method called *headers-first*. The goal is to download the headers for the best :term:`header chain`, partially validate them as best as possible, and then download the corresponding blocks in parallel. This solves several problems with the older blocks-first IBD method. .. figure:: /img/dev/en-headers-first-flowchart.svg :alt: Overview Of Headers-First Method @@ -171,7 +173,7 @@ The first time a node is started, it only has a single block in its local best b In the header hashes field of the `“getheaders” message <../reference/p2p_networking.html#getheaders>`__, the new node sends the header hash of the only block it has, the genesis block (6fe2…0000 in internal byte order). It also sets the stop hash field to all zeroes to request a maximum-size response. -Upon :ref:`receipt ` of the `“getheaders” message <../reference/p2p_networking.html#getheaders>`__, the sync node takes the first (and only) header hash and searches its local best block chain for a block with that header hash. It finds that block 0 matches, so it replies with 2,000 header (the maximum response) starting from block 1. It sends these header hashes in the `“headers” message <../reference/p2p_networking.html#headers>`__ illustrated below. +Upon :term:`receipt` of the `“getheaders” message <../reference/p2p_networking.html#getheaders>`__, the sync node takes the first (and only) header hash and searches its local best block chain for a block with that header hash. It finds that block 0 matches, so it replies with 2,000 header (the maximum response) starting from block 1. It sends these header hashes in the `“headers” message <../reference/p2p_networking.html#headers>`__ illustrated below. .. figure:: /img/dev/en-ibd-headers.svg :alt: First headers message @@ -218,9 +220,9 @@ Block Broadcasting When a miner discovers a new block, it broadcasts the new block to its peers using one of the following methods: -- :ref:`Unsolicited Block Push `\ **:** the miner sends a `“block” message <../reference/p2p_networking.html#block>`__ to each of its full node peers with the new block. The miner can reasonably bypass the standard relay method in this way because it knows none of its peers already have the just-discovered block. +- :term:`Unsolicited Block Push`\ **:** the miner sends a `“block” message <../reference/p2p_networking.html#block>`__ to each of its full node peers with the new block. The miner can reasonably bypass the standard relay method in this way because it knows none of its peers already have the just-discovered block. -- :ref:`Standard Block Relay `\ **:** the miner, acting as a standard relay node, sends an `“inv” message <../reference/p2p_networking.html#inv>`__ to each of its peers (both full node and SPV) with an inventory referring to the new block. The most common responses are: +- :term:`Standard Block Relay`\ **:** the miner, acting as a standard relay node, sends an `“inv” message <../reference/p2p_networking.html#inv>`__ to each of its peers (both full node and SPV) with an inventory referring to the new block. The most common responses are: - Each blocks-first (BF) peer that wants the block replies with a `“getdata” message <../reference/p2p_networking.html#getdata>`__ requesting the full block. @@ -234,9 +236,9 @@ When a miner discovers a new block, it broadcasts the new block to its peers usi This protocol for block broadcasting was proposed in BIP 130 and has been implemented in Bitcoin Core since version 0.12. -By default, Bitcoin Core broadcasts blocks using direct headers announcement to any peers that have signalled with `“sendheaders” <../reference/p2p_networking.html#sendheaders>`__ and uses :ref:`standard block relay ` for all peers that have not. Bitcoin Core will accept blocks sent using any of the methods described above. +By default, Bitcoin Core broadcasts blocks using direct headers announcement to any peers that have signalled with `“sendheaders” <../reference/p2p_networking.html#sendheaders>`__ and uses :term:`standard block relay` for all peers that have not. Bitcoin Core will accept blocks sent using any of the methods described above. -Full nodes validate the received block and then advertise it to their peers using the :ref:`standard block relay ` method described above. The condensed table below highlights the operation of the messages described above (Relay, BF, HF, and SPV refer to the relay node, a blocks-first node, a headers-first node, and an SPV client; *any* refers to a node using any block retrieval method.) +Full nodes validate the received block and then advertise it to their peers using the :term:`standard block relay` method described above. The condensed table below highlights the operation of the messages described above (Relay, BF, HF, and SPV refer to the relay node, a blocks-first node, a headers-first node, and an SPV client; *any* refers to a node using any block retrieval method.) +------------------------------------------------------------------+---------------+------------------------------------------------------------------------------------------+ | Message | From→To | Payload | @@ -259,7 +261,7 @@ Full nodes validate the received block and then advertise it to their peers usin Orphan Blocks ~~~~~~~~~~~~~ -Blocks-first nodes may download orphan blocks—blocks whose :ref:`previous block header hash ` field refers to a block header this node hasn’t seen yet. In other words, orphan blocks have no known parent (unlike stale blocks, which have known parents but which aren’t part of the best block chain). +Blocks-first nodes may download orphan blocks—blocks whose :term:`previous block header hash` field refers to a block header this node hasn’t seen yet. In other words, orphan blocks have no known parent (unlike stale blocks, which have known parents but which aren’t part of the best block chain). .. figure:: /img/dev/en-orphan-stale-definition.svg :alt: Difference Between Orphan And Stale Blocks @@ -270,7 +272,7 @@ When a blocks-first node downloads an orphan block, it will not validate it. Ins Headers-first nodes avoid some of this complexity by always requesting block headers with the `“getheaders” message <../reference/p2p_networking.html#getheaders>`__ before requesting a block with the `“getdata” message <../reference/p2p_networking.html#getdata>`__. The broadcasting node will send a `“headers” message <../reference/p2p_networking.html#headers>`__ containing all the block headers (up to 2,000) it thinks the downloading node needs to reach the tip of the best header chain; each of those headers will point to its parent, so when the downloading node receives the `“block” message <../reference/p2p_networking.html#block>`__, the block shouldn’t be an orphan block—all of its parents should be known (even if they haven’t been validated yet). If, despite this, the block received in the `“block” message <../reference/p2p_networking.html#block>`__ is an orphan block, a headers-first node will discard it immediately. -However, orphan discarding does mean that headers-first nodes will ignore orphan blocks sent by miners in an :ref:`unsolicited block push `. +However, orphan discarding does mean that headers-first nodes will ignore orphan blocks sent by miners in an :term:`unsolicited block push`. Transaction Broadcasting ------------------------ @@ -282,7 +284,7 @@ Memory Pool Full peers may keep track of unconfirmed transactions which are eligible to be included in the next block. This is essential for miners who will actually mine some or all of those transactions, but it’s also useful for any peer who wants to keep track of unconfirmed transactions, such as peers serving unconfirmed transaction information to SPV clients. -Because unconfirmed transactions have no permanent status in Bitcoin, Bitcoin Core stores them in non-persistent memory, calling them a memory pool or mempool. When a peer shuts down, its memory pool is lost except for any transactions stored by its wallet. This means that never-mined unconfirmed transactions tend to slowly disappear from the `network <../devguide/p2p_network.html>`__ as peers restart or as they purge some transactions to make room in memory for others. +Because unconfirmed transactions have no permanent status in Bitcoin, Bitcoin Core stores them in non-persistent memory, calling them a memory pool or mempool. When a peer shuts down, its memory pool is lost except for any transactions stored by its wallet. This means that never-mined unconfirmed transactions tend to slowly disappear from the |network| as peers restart or as they purge some transactions to make room in memory for others. Transactions which are mined into blocks that later become stale blocks may be added back into the memory pool. These re-added transactions may be re-removed from the pool almost immediately if the replacement blocks include them. This is the case in Bitcoin Core, which removes stale blocks from the chain one by one, starting with the tip (highest block). As each block is removed, its transactions are added back to the memory pool. After all of the stale blocks are removed, the replacement blocks are added to the chain one by one, ending with the new tip. As each block is added, any transactions it confirms are removed from the memory pool. @@ -298,4 +300,4 @@ Alerts *Removed in*\ `Bitcoin Core 0.13.0 `__ -Earlier versions of Bitcoin Core allowed developers and trusted community members to issue `Bitcoin alerts `__ to notify users of critical `network <../devguide/p2p_network.html>`__-wide issues. This messaging system `was retired `__ in Bitcoin Core v0.13.0; however, internal alerts, partition detection warnings and the ``-alertnotify`` option features remain. +Earlier versions of Bitcoin Core allowed developers and trusted community members to issue `Bitcoin alerts `__ to notify users of critical |network|-wide issues. This messaging system `was retired `__ in Bitcoin Core v0.13.0; however, internal alerts, partition detection warnings and the ``-alertnotify`` option features remain. diff --git a/devguide/payment_processing.rst b/devguide/payment_processing.rst index 8b75923..537a178 100644 --- a/devguide/payment_processing.rst +++ b/devguide/payment_processing.rst @@ -1,3 +1,5 @@ +.. _dev-payment-processing-header: + Payment Processing ================== @@ -6,7 +8,7 @@ Payment processing encompasses the steps spenders and receivers perform to make Introduction ------------ -This section will explain how receivers and spenders can, respectively, request and make payments using Bitcoin—and how they can deal with complications such as `refunds <../devguide/payment_processing.html#issuing-refunds>`__ and `recurrent rebilling <../devguide/payment_processing.html#rebilling-recurring-payments>`__. +This section will explain how receivers and spenders can, respectively, request and make payments using Bitcoin—and how they can deal with complications such as `refunds <../devguide/payment_processing.html#issuing-refunds>`_ and `recurrent rebilling <../devguide/payment_processing.html#rebilling-recurring-payments>`__. .. figure:: /img/dev/en-payment-processing.svg :alt: Bitcoin Payment Processing @@ -20,7 +22,7 @@ It is worth mentioning that each of these steps can be outsourced by using third Pricing Orders -------------- -Because of exchange rate variability between satoshis and national currencies (:ref:`fiat `), many Bitcoin orders are priced in :ref:`fiat ` but paid in satoshis, necessitating a price conversion. +Because of exchange rate variability between satoshis and national currencies (:term:`fiat`), many Bitcoin orders are priced in :term:`fiat` but paid in satoshis, necessitating a price conversion. Exchange rate data is widely available through HTTP-based APIs provided by currency exchanges. Several organizations also aggregate data from multiple exchanges to create index prices, which are also available using HTTP-based APIs. @@ -30,11 +32,11 @@ To minimize problems, your applications may want to collect data from at least t You may also want to program your applications to enter a safe mode if exchange rates are rapidly increasing or decreasing, indicating a possible problem in the Bitcoin market which could make it difficult to spend any satoshis received today. -Exchange rates lie outside the control of Bitcoin and related technologies, so there are no new or planned technologies which will make it significantly easier for your program to correctly convert order totals from :ref:`fiat ` into satoshis. +Exchange rates lie outside the control of Bitcoin and related technologies, so there are no new or planned technologies which will make it significantly easier for your program to correctly convert order totals from :term:`fiat` into satoshis. -Because the exchange rate fluctuates over time, order totals pegged to :ref:`fiat ` must expire to prevent spenders from delaying payment in the hope that satoshis will drop in price. Most widely-used payment processing systems currently expire their invoices after 10 to 20 minutes. +Because the exchange rate fluctuates over time, order totals pegged to :term:`fiat` must expire to prevent spenders from delaying payment in the hope that satoshis will drop in price. Most widely-used payment processing systems currently expire their invoices after 10 to 20 minutes. -Shorter expiration periods increase the chance the invoice will expire before payment is received, possibly necessitating manual intervention to request an additional payment or to issue a `refund <../devguide/payment_processing.html#issuing-refunds>`__. Longer expiration periods increase the chance that the exchange rate will fluctuate a significant amount before payment is received. +Shorter expiration periods increase the chance the invoice will expire before payment is received, possibly necessitating manual intervention to request an additional payment or to issue a refund_. Longer expiration periods increase the chance that the exchange rate will fluctuate a significant amount before payment is received. Requesting Payments ------------------- @@ -47,11 +49,11 @@ The next subsections will describe in detail the following four compatible ways 1. All wallet software lets its users paste in or manually enter an address and amount into a payment screen. This is, of course, inconvenient—but it makes an effective fallback option. -2. Almost all desktop wallets can associate with :ref:`“bitcoin:” URIs `, so spenders can click a link to pre-fill the payment screen. This also works with many mobile wallets, but it generally does not work with web-based wallets unless the spender installs a browser extension or manually configures a URI handler. +2. Almost all desktop wallets can associate with |Bitcoin URIs|, so spenders can click a link to pre-fill the payment screen. This also works with many mobile wallets, but it generally does not work with web-based wallets unless the spender installs a browser extension or manually configures a URI handler. -3. Most mobile wallets support scanning :ref:`“bitcoin:” URIs ` encoded in a QR code, and almost all wallets can display them for accepting payment. While also handy for online orders, QR Codes are especially useful for in-person purchases. +3. Most mobile wallets support scanning |Bitcoin URIs| encoded in a QR code, and almost all wallets can display them for accepting payment. While also handy for online orders, QR Codes are especially useful for in-person purchases. -4. Recent wallet updates add support for the new payment protocol providing increased security, authentication of a receiver’s identity using `X.509 `__ certificates, and other important features such as `refunds <../devguide/payment_processing.html#issuing-refunds>`__. +4. Recent wallet updates add support for the new payment protocol providing increased security, authentication of a receiver’s identity using `X.509 `__ certificates, and other important features such as refunds_. |Warning icon| **Warning:** Special care must be taken to avoid the theft of incoming payments. In particular, private keys should not be stored on web servers, and payment requests should be sent over HTTPS or other secure methods to prevent `man-in-the-middle `__ attacks from replacing your Bitcoin address with the attacker’s address. @@ -84,7 +86,7 @@ Bitcoins Unit (Abbreviation) bitcoin: URI ~~~~~~~~~~~~ -The :ref:`“bitcoin:” URI ` scheme defined in `BIP21 `__ eliminates denomination confusion and saves the spender from copying and pasting two separate values. It also lets the payment request provide some additional information to the spender. An example: +The |Bitcoin URI| scheme defined in `BIP21 `__ eliminates denomination confusion and saves the spender from copying and pasting two separate values. It also lets the payment request provide some additional information to the spender. An example: :: @@ -92,7 +94,7 @@ The :ref:`“bitcoin:” URI ` scheme defined in `BIP21 ` parameter is generally used to provide wallet software with the recipient’s name. The :ref:`“message” ` parameter is generally used to describe the payment request to the spender. Both the label and the message are commonly stored by the spender’s wallet software—but they are never added to the actual transaction, so other Bitcoin users cannot see them. Both the label and the message must be `URI encoded `__. +Two other parameters are widely supported. The :term:`“label”