docs: add troubleshooting guide with first section on the batch-hash-…

…mismatch error

README.md

@@ -157,6 +157,8 @@ An optional `--eigenda.confirmation-depth` flag can be provided to specify a num
 `0`: Verify the cert immediately upon blob confirmation and return the cert
 `N where 0<N<64`: Wait `N` blocks before returning the cert to the batcher (default value is 5)
 
+The default value is 5. Using 0 is dangerous: see [troubleshooting the batch-hash-mismatch error](./docs/troubleshooting_v1.md#batch-hash-mismatch-error).
+
 ### In-Memory Backend
 
 An ephemeral memory store backend can be used for faster feedback testing when testing rollup integrations. To target this feature, use the CLI flags `--memstore.enabled`, `--memstore.expiration`.

docs/troubleshooting_v1.md

@@ -0,0 +1,51 @@
+# Troubleshooting V1
+
+List of common bugs and issues that you may encounter while using eigenda-proxy.
+
+## "batch hash mismatch" error
+
+This error returned [here](https://github.com/Layr-Labs/eigenda-proxy/blob/97b71bda6e2136a6c0d71be2f7e9642dc840fefc/verify/cert.go#L125) happens when verifying a v1 cert, where the batch hash in the cert does not match the hash that was computed when the cert was bridged onchain by the EigenDA disperser.
+
+This typically results from an L1 reorg while running the eigenda-proxy with the `--eigenda.confirmation-depth` flag set to 0. In this setting, the disperser bridges the cert onchain via the confirmBatch function, then the proxy (while polling the GetBlobStatus) endpoint receives the cert, and without waiting for the cert to have been onchain for a few blocks (to prevent reorgs) immediately returns it to the batcher, which then sends it onchain. At this point the L1 reorgs, forcing the disperser to resent a new confirmBatch transaction, which lands in a new block. The cert for the blob then gets updated with a different [confirmation_block_number](https://github.com/Layr-Labs/eigenda/blob/f305e046ae3e611e19c15e134571cc2ec83062b4/api/proto/disperser/disperser.proto#L242). This makes the cert in the batcher inbox outdated and no longer valid. Hence, the rollup nodes, after reading the cert from the batcher-inbox, submits it to the proxy via a GET request, which hashes the batchHeader contained in the cert and compares it to the batch hash in the onchain. Because their `confirmation_block_number` differ, the hash is naturally different, hence the observed `batch hash mismatch` error.
+
+To troubleshoot this error, you will first need the [request_id](https://github.com/Layr-Labs/eigenda/blob/f305e046ae3e611e19c15e134571cc2ec83062b4/api/proto/disperser/disperser.proto#L112) of the blob. This can be found in the proxy's logs, and is also used as the URL slug for our blob explorer: in https://blobs-holesky.eigenda.xyz/blobs/05cb90531098964cc73f0e6b782f52f3b4387e3aed3ef6b89cd9a3c118dd781e-313733393032373134323137353330363738362f302f33332f312f33332fe3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855, the request_id is `05cb90531098964cc73f0e6b782f52f3b4387e3aed3ef6b89cd9a3c118dd781e-313733393032373134323137353330363738362f302f33332f312f33332fe3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855`. From this request_id, you can query for the cert from the disperser:
+```bash
+REQUEST_ID=05cb90531098964cc73f0e6b782f52f3b4387e3aed3ef6b89cd9a3c118dd781e-313733393032373134323137353330363738362f302f33332f312f33332fe3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
+REQUEST_ID_BASE64=$(echo -n $REQUEST_ID | base64)
+grpcurl -d @ disperser-holesky.eigenda.xyz:443 disperser.Disperser.GetBlobStatus <<EOM
+{
+  "request_id": "$REQUEST_ID_BASE64"
+}
+EOM
+```
+And then you can compare the confirmation_block_number returned from the disperser to the one in the cert that was posted to the rollup's batcher inbox:
+```bash
+OP_ALTDA_COMMITMENT=010000f901d8f852f842a02da35fdbe4fed916eb80fafb6e12711a35327e1e7055b78fc9085812e9d6e28fa021c98859127f075fc90785a730ce1b3894586badee45a698a5cce9c2d6d36269820200cac480213701c401213708f901818302270581c3f873eba0eea9df513a56cbdc8a8c045ddaa61a2a7f86097ad0b99b5b08baa7119065effe82000182636283328734a0c7c898eb39076fbc6b367c51c1908a68bbcac3d81f739057aec168aa3a78e1b20083328792a0c3169896044de8cee3270fdd643157665353a6963149cb158b63137ac405251ab90100c3904bddb63b2be1f2e779b9ecf289fe40693eb943cb260a7d5c95ac6021f35883ade0e4d82e071b86158510fc83cd2c2d61f4a72e6bc6d5c533bacd44eb6a6b97f742d7518f6475f65fb0656fe58d8ad2cc8961c770680c9538593c5a8bc3861fc11e38477076f06911e6de93d9b641b535115700bcdcc863b58b021a7a177b3343e80ab645867061216e191740e4558754e7d13fc3f235743f8d865c0f976a91a3c7912d03e56bb955ddec854c2d661edfc43d5c366040296eac657960a8b40ec6e01b17624d18f32e930b8712cc1a00a2f696229ce10b5b03b3c7c992b601c8ef46390b2fd016687cd1ed4cd70310018dfa97418c7819a0cda7322a7eefa4820001
+RLP_ENCODED_CERT=${OP_ALTDA_COMMITMENT:6}
+cast --from-rlp $RLP_ENCODED_CERT | jq
+```
+Unfortunately `cast --from-rlp` doesn't currently support schemas, so you will have to compare the arrays-of-hex-strings output to the [BlobInfo](https://github.com/Layr-Labs/eigenda/blob/f305e046ae3e611e19c15e134571cc2ec83062b4/api/proto/disperser/disperser.proto#L178) schema (note that [CertV1 === BlobInfo](https://github.com/Layr-Labs/eigenda-proxy/blob/97b71bda6e2136a6c0d71be2f7e9642dc840fefc/verify/certificate.go#L31)).
+
+And here's a full script to compare both (replace `REQUEST_ID`, `OP_ALTDA_COMMITMENT`, and `DISPERSER_ENDPOINT` with your values):
+```bash
+DISPERSER_ENDPOINT=disperser-holesky.eigenda.xyz:443
+# Get the request_id from your batcher's proxy logs
+REQUEST_ID=05cb90531098964cc73f0e6b782f52f3b4387e3aed3ef6b89cd9a3c118dd781e-313733393032373134323137353330363738362f302f33332f312f33332fe3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
+# Get the cert from the rollup's batcher inbox
+OP_ALTDA_COMMITMENT=010000f901d8f852f842a02da35fdbe4fed916eb80fafb6e12711a35327e1e7055b78fc9085812e9d6e28fa021c98859127f075fc90785a730ce1b3894586badee45a698a5cce9c2d6d36269820200cac480213701c401213708f901818302270581c3f873eba0eea9df513a56cbdc8a8c045ddaa61a2a7f86097ad0b99b5b08baa7119065effe82000182636283328734a0c7c898eb39076fbc6b367c51c1908a68bbcac3d81f739057aec168aa3a78e1b20083328792a0c3169896044de8cee3270fdd643157665353a6963149cb158b63137ac405251ab90100c3904bddb63b2be1f2e779b9ecf289fe40693eb943cb260a7d5c95ac6021f35883ade0e4d82e071b86158510fc83cd2c2d61f4a72e6bc6d5c533bacd44eb6a6b97f742d7518f6475f65fb0656fe58d8ad2cc8961c770680c9538593c5a8bc3861fc11e38477076f06911e6de93d9b641b535115700bcdcc863b58b021a7a177b3343e80ab645867061216e191740e4558754e7d13fc3f235743f8d865c0f976a91a3c7912d03e56bb955ddec854c2d661edfc43d5c366040296eac657960a8b40ec6e01b17624d18f32e930b8712cc1a00a2f696229ce10b5b03b3c7c992b601c8ef46390b2fd016687cd1ed4cd70310018dfa97418c7819a0cda7322a7eefa4820001
+
+REQUEST_ID_BASE64=$(echo -n $REQUEST_ID | base64)
+DISPERSER_CERT_CONFIRMATION_BLOCK_NUM=$((
+    grpcurl -d @ $DISPERSER_ENDPOINT disperser.Disperser.GetBlobStatus <<EOM
+{
+  "request_id": "$REQUEST_ID_BASE64"
+}
+EOM
+) | jq .info.blobVerificationProof.batchMetadata.confirmationBlockNumber)
+
+RLP_ENCODED_CERT=${OP_ALTDA_COMMITMENT:6}
+BATCHER_INBOX_CERT_CONFIRMATION_BLOCK_NUM=$(cast --from-rlp $RLP_ENCODED_CERT | jq -r '.[1][2][3]' | cast --to-dec)
+
+echo "Disperser cert confirmation block number: $DISPERSER_CERT_CONFIRMATION_BLOCK_NUM"
+echo "Batcher inbox cert confirmation block number: $BATCHER_INBOX_CERT_CONFIRMATION_BLOCK_NUM"
+```