THE ASSET HUB MIGRATION (Please Review)#856
THE ASSET HUB MIGRATION (Please Review)#856fellowship-merge-bot[bot] merged 216 commits intomainfrom
Conversation
This PR is not intended to be merged into the master branch but into a dedicated Asset Hub migration working branch. Please create such a branch if you have the necessary permissions and change the base for this PR. The PR introduces two pallets designed for the Asset Hub migration, Relay Chain migrator (rc-migrator) and Asset Hub migrator (ah-migrator). Please refer to the code base for more docs. Currently, these pallets handle only the migration of Relay Chain accounts. They are intended to support the migration of all migrating pallets and act as managers for the overall migration process. It might be helpful to merge this PR into the dev branch to use this initial setup as a base for further development. [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
To be merged into the AHM dev branch. Changes: - Create `polkadot-integration-tests-ahm` crate that pulls in the Relay and AH runtimes - Load snapshot for both chains and run trivial DMP testing - Add stage to initialize the account migration - TODO: Investigate failing debug assertion on the account migration - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
👉 To be merged into the AHM working branch. Changes: - Clean up AH and RC account migration code - Add error logging - Mint Relay blocks until no more account can be moved - Mint AH blocks until all DMP messages are consumed - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
👉 To be merged into the AHM working branch. Changes: - Add Multisig module to Relay and AH - Migrate multisigs out (TODO investigate weird accs with ggwpez/pdu#5) - Unlock deposits on the AH side since we dont actually migrate multisigs since they contain a call hash, making them impossible to migrate. - TODO docs - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
👉 To be merged into the AHM dev branch. quote from the `proxy.md` doc: ## Pallet Proxy The proxy pallet consists of two storage variables. ## Storage: Proxies The [Proxies](https://github.com/paritytech/polkadot-sdk/blob/7c5224cb01710d0c14c87bf3463cc79e49b3e7b5/substrate/frame/proxy/src/lib.rs#L564-L579) storage map maps a delegator to its delegates. It can be translated one-to-one by mapping the `ProxyType` and `Delay` fields. ### Proxy Type Translation The different kinds that are possible for a proxy are a [runtime injected type](https://github.com/paritytech/polkadot-sdk/blob/7c5224cb01710d0c14c87bf3463cc79e49b3e7b5/substrate/frame/proxy/src/lib.rs#L119-L125). Since these are different for each runtime, we need a converter that maps the Relay to AH `ProxyType` as close as possible to keep the original intention. The Relay kind is defined [here](https://github.com/polkadot-fellows/runtimes/blob/dde99603d7dbd6b8bf541d57eb30d9c07a4fce32/relay/polkadot/src/lib.rs#L1000-L1010) and the AH version [here](https://github.com/polkadot-fellows/runtimes/blob/fd8d0c23d83a7b512e721b1fde2ba3737a3478d5/system-parachains/asset-hubs/asset-hub-polkadot/src/lib.rs#L453-L468). This is done by injecting a `RcToProxyType` converter into the Asset Hub migration pallet. This is not bullet proof since it relies on some copy&paste code instead of pulling in the Polkadot runtime into the AH runtime but it is the simplest solution. Mapping from Relay to AH looks as follows: - Any: same - NonTransfer: same - Governance: newly added - Staking: newly added - Variant 4: ignore as it is a historic remnant - Variant 5: ignore ditto - CancelProxy: same - Auction: dropped - NominationPools: newly added All variants that serve no purpose anymore on the Relay Chain are deleted from there. For example `Staking`. The ones that are still usable on the relay like `NonTransfer` are **also deleted** since there is no storage deposit taken anymore. (TODO think about what is best here) ### Translation of the Delay The [delay of a ProxyDefinition](https://github.com/paritytech/polkadot-sdk/blob/7c5224cb01710d0c14c87bf3463cc79e49b3e7b5/substrate/frame/proxy/src/lib.rs#L77) is measured in blocks. These are currently 6 seconds Relay blocks. To translate them to 12s AH blocks, we can divide the number by two. ## Storage: Announcements The [Announcements](https://github.com/paritytech/polkadot-sdk/blob/7c5224cb01710d0c14c87bf3463cc79e49b3e7b5/substrate/frame/proxy/src/lib.rs#L581-L592) storage maps proxy account IDs to [Accouncement](https://github.com/paritytech/polkadot-sdk/blob/7c5224cb01710d0c14c87bf3463cc79e49b3e7b5/substrate/frame/proxy/src/lib.rs#L80-L89). Since an announcement contains a call hash, we cannot translate them for the same reason as with the Multisigs; the preimage of the hash would be either undecodable, decode to something else (security issue) or accidentally decode to the same thing. We therefore do not migrate the announcements. ## User Impact - Announcements need to be re-created - Proxies of type `Auction` are not migrated and need to be re-created on the Relay - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
👉 To be merged into the Asset Hub Migration working branch. Changes: - Factor out XCM sending into common function - Cleanup - Add docs for the preimage pallet that I started with - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Meant to be merged into `dev-asset-hub-migration` branch. Some improvements and fixes to the accounts migration.
👉 To be merged into the AHM dev branch. quote from Preimage.md: The preimage pallet consists of three storage maps, one of which is a legacy item. The interesting one are `RequestStatusFor` and `PreimageFor`. ## Storage: PreimageFor [Maps](https://github.com/paritytech/polkadot-sdk/blob/00946b10ab18331f959f5cbced7c433b6132b1cb/substrate/frame/preimage/src/lib.rs#L185) a hash and the length of a preimage to its preimage data. Preimages can be migrated rather easily by sending them in chunks from the Relay and appending them on the Asset Hub. The preimages are often used to store large governance calls. Q: One question here would be whether or not we want to translate these calls. I think we can and should. But I am not sure about the best time point to do so. We can translate the preimages calls upon arrival on the Asset Hub, although there is a small chance that a preimage that was not intended to be decoded as a call would be translated. After all, the preimage pallet is a universal data storage without any implied semantics about its content. It could therefore be better to translate the preimages once we are migrating the Referenda pallet and then only translate all preimages that occur as hash lookups in a referenda. However, since the scheduler is the only way to request a preimage, all requested preimages should probably be valid calls. But the translation still needs to happen in accord with the Referenda pallet as to not invalidate its hashes. Basically: loop referenda -> load referenda -> load preimage of referenda -> translate preimage -> calculate new preimage hash -> update preimage with new hash -> update referenda with new hash. One further note on the preimage translation: If the length of a preimage is increased by the translation, then we should not reconsider the deposit but keep the original deposit as to not punish users for this. The cases that the translation increases the size of a preimage past the 4MiB hard limit should be negligible. ## Storage: RequestStatusFor This maps preimage hashes to [RequestStatus](https://github.com/paritytech/polkadot-sdk/blob/00946b10ab18331f959f5cbced7c433b6132b1cb/substrate/frame/preimage/src/lib.rs#L81-L89). The RequestStatus contains a consideration that will be re-considered upon import. This would unreserve funds for all users that noted preimages. Possible up to 20 DOT per preimage. The migration of this should be straighforward this but we need to remember that it must be updated if we start translating preimage calls. ## Storage: StatusFor Deprecated. Will not be migrated but funds will be unreserved. ## User Impact For anyone who has registered a preimage: - If the preimage was in the new RequestStatusFor: Some unlocked funds 😎. We cannot calculate a list of affected accounts in advance since users can still influence this. - If the preimage was in the old StatusFor: will be removed and funds unlocked. [Exhaustive list](https://github.com/ggwpez/substrate-scripts/blob/master/ahm-preimage-statusfor-accounts.py) of all 166 Polkadot accounts that are affected by this and will have **UP TO** these funds unlocked (not a legally binding statement): <details><summary>Details</summary> <p> - `16LKv69ct6xDzSiUjuz154vCg62dkyysektHFCeJe85xb6X`: 1256.897 DOT - `15ynbcMgPf7HbQErRz66RDLMuBVdcWVuURhR4SLPiqa6B8jx`: 633.121 DOT - `12jW7jTPVKWahgRvxZL8ZCKKAwzy4kbrkHhzhafNjHEJXfw9`: 633.121 DOT - `13BD4q9RYQtxkUQLvyCksnN9Pa7sC5fGj5dcdxpojxGkoHMp`: 40.229 DOT - `13NRkBCD7NLkppxoHpYrUQ6GcjNpZEWCeXFjXDgDctANBG9j`: 40.193 DOT - `14VwUiNPMN2T9jGvWaSm5pwcUr5ziqLjTomRm6xUxwy3Urjm`: 40.17 DOT - `14TBcRgp166DXvMv9ZCJbKSqanUGP6tguryPQcaBqjQp8d4m`: 40.152 DOT - `14TBcRgp166DXvMv9ZCJbKSqanUGP6tguryPQcaBqjQp8d4m`: 40.143 DOT - `1eK9SC7Z2QFi4es2aKQHAehcZqY7bEiprkpsT483PvVK8KE`: 40.143 DOT - `15YLDvV6Q2NUVEFBN26kRgHyyeH1Bu91NKTwBg3xW3hEVfoj`: 40.108 DOT - `13q3NEbcSepgVbCyN6XLQtEvyZuAEqDUPLiuX2iydaQrwDCU`: 40.107 DOT - `14M94kYk31k2hY8MpnfNPRviJ4VcsFFjBhq7V2Fs9DzCVhXc`: 40.107 DOT - `1316cTZeHz8HtEjaJRHu8sHbp9brtUmy2LiP74KZXgXhifry`: 40.107 DOT - `16maYYXg9chsfsBVoiSbSWzmFveamERwShPZv3SB5hVnYTmT`: 40.107 DOT - `12ow3eJ3vbjeNRahUUrBnc98mWeJTSQ7rJCAVqiFQDEnzbu8`: 40.107 DOT - `13g4yRs3NbtaXyu1Uww8AXd4uvrqXyR1hPR4jejRLv8rBUyB`: 40.107 DOT - `12rpF7eUC59kU7itRe3NpSTQJroK5YiHfn5c4bT21BZxp257`: 40.107 DOT - `16maYYXg9chsfsBVoiSbSWzmFveamERwShPZv3SB5hVnYTmT`: 40.107 DOT - `1fN87Fgj5BUhezFgbLiGbXTMrBVggnmYBX9anzMBky8KaJ5`: 40.107 DOT - `14PiQ7uar36zPMgEckA7qWUahYheavRL6NHCbUCkXXRNrFSc`: 40.107 DOT - `13SceNt2ELz3ti4rnQbY1snpYH4XE4fLFsW8ph9rpwJd6HFC`: 40.107 DOT - `1hzs7HJ4teyvX9cwFsxCaJBSNQcPAWHixQT4fem5h66cogb`: 40.107 DOT - `15ho9t317QDvod18gCoTNe9yoiMjTXHwVxd5RC2iWyzEEby1`: 40.107 DOT - `13SceNt2ELz3ti4rnQbY1snpYH4XE4fLFsW8ph9rpwJd6HFC`: 40.107 DOT - `12bMyzdtiT2V9iNJ7BzQXPmzZ4KTzqFmZPSNeBmg97mFP5F4`: 40.107 DOT - `1QjuTEKebQ3au8bxQC6iwYSPCA2iZn3YHwX8VABCauKtwRk`: 40.107 DOT - `15fvwi77dujPz9Mk9U792gNa2Mg5z6489DPwErwCZwu7EpLE`: 40.107 DOT - `1WgB9o954mkQi97f36azSwDt7SfRUQuJ1kCyb7Sv1WAUcSe`: 40.107 DOT - `14zU4FXuYU2wmi2PfXLADZW92NRYEw8nfUEvi7sqiJLafJ3A`: 40.107 DOT - `15cfSaBcTxNr8rV59cbhdMNCRagFr3GE6B3zZRsCp4QHHKPu`: 40.107 DOT - `133VgJJgp1s9wxLqgCFxYx6T873hZQNchJM9tmbU6NkYaaqW`: 40.107 DOT - `13YMTEPKAxPRiyaZdMKrozeNT9x1Pa5h7aExebCdi6nc3Qqd`: 40.107 DOT - `14M94kYk31k2hY8MpnfNPRviJ4VcsFFjBhq7V2Fs9DzCVhXc`: 40.107 DOT - `13Ghf2T883ZobjngC1BAgR1BWvK2P7qP37gGxHDVFf3fjbmw`: 40.107 DOT - `15cfSaBcTxNr8rV59cbhdMNCRagFr3GE6B3zZRsCp4QHHKPu`: 40.107 DOT - `15qz4ZLeyXp1i4Jbx7AXiUQVCCLWVXu3dLjcTPHY3v9KGAvL`: 40.107 DOT - `14TBcRgp166DXvMv9ZCJbKSqanUGP6tguryPQcaBqjQp8d4m`: 40.107 DOT - `13zTcqasJT4DnDgNjmsceACcuSjt4q2geEjtMprnGXCnuuh1`: 40.107 DOT - `12eWtdVxQ9ScYD9AzyMuSsX8B9iEikWtUGiirJ1YJtDCCuwu`: 40.107 DOT - `1342Xpqiwwmxnhugnp91d21xR7s8V6uxXQJ1xYBQfUwbvgDB`: 40.107 DOT - `16JA2pWJ7rXhKAq9xaCpSvVgWf6MaPLYvtSVpj7ZWjTkhYoB`: 40.107 DOT - `1j5YyEGdcPd9BxkzVNNjKkqdi5f7g3Dd7JMgaGUhsMrZ6dZ`: 40.107 DOT - `1njGozmydXftj6KYFPGLPN7Qq3kgmFqxsRdF5hWJAschp1S`: 40.107 DOT - `12pdBf9NJ2jqRHdVmtqSZMRvWQoiH81AfaACgiMuXLeySNzc`: 40.107 DOT - `1njGozmydXftj6KYFPGLPN7Qq3kgmFqxsRdF5hWJAschp1S`: 40.107 DOT - `16MF8p8KfktKazPiQEqTXJq1CtYuZ9aNrBShXQNRdhckctC5`: 40.107 DOT - `197nLd2rFoesjmvTfMpkFhHde7ngKzpLaA8xsbdWyeaJwzx`: 40.107 DOT - `12BYYgmRb5BjHjZf7nykJDB1C6FXTfqr9QSmrav8RHt19ahj`: 40.107 DOT - `12BJTP99gUerdvBhPobiTvrWwRaj1i5eFHN9qx51JWgrBtmv`: 40.107 DOT - `1333zsMafds2sKAr8nG3zwXTCHPYv2Nm6CRgakpu6YVGt7nM`: 40.107 DOT - `15qz4ZLeyXp1i4Jbx7AXiUQVCCLWVXu3dLjcTPHY3v9KGAvL`: 40.107 DOT - `13EDmaUe89xXocPppFmuoAZaCsckaJy3deAyVyiykk1zKQbF`: 40.107 DOT - `13Ghf2T883ZobjngC1BAgR1BWvK2P7qP37gGxHDVFf3fjbmw`: 40.107 DOT - `1ZVYsze5Ls3osofU6wWSp5dphr62Rj7YiL4NsXiZU3a298F`: 40.107 DOT - `14j9cWtbvYid754crk6ieQABGYHtGZozzeavT1jc11bt32ZM`: 40.107 DOT - `14fcqMPHhCtwnbPAHxjsf3JiGsDuLQPGMpndrWawuiAiiCqE`: 40.107 DOT - `12dt664RtnYbeiR1D45CUPyHk1Ufv1NEHFXkuRLy47FktR31`: 40.107 DOT - `131JKfT9kNvKjp5NJY2jHZmb32wjbr6xDHuCt4zHapWVtDde`: 40.107 DOT - `15cfSaBcTxNr8rV59cbhdMNCRagFr3GE6B3zZRsCp4QHHKPu`: 40.107 DOT - `1f1wZcBaJrPHkBkzx2S7KXFbjtT7KMg7fDaV47P6157KRWo`: 40.107 DOT - `14Q5M6LWDVCPm47sVvz6M6YAEsEi5u3Rszh8z5eC2bhL9Upk`: 40.107 DOT - `1k5ddMCPuLbu9Hax12EdKRmPwGigUKQW1ab6tRAWPxKygRF`: 40.107 DOT - `15YLDvV6Q2NUVEFBN26kRgHyyeH1Bu91NKTwBg3xW3hEVfoj`: 40.107 DOT - `14fhPR28n9EHZitNyf6wjYZVBPwKgcgogVjJPTzvCcb8qi9G`: 40.107 DOT - `1RYjrCKUmvM8D9QDKCNbWJYUe49h6ZfkgXvEAtkHgvzxbGB`: 40.107 DOT - `15wznkm7fMaJLFaw7B8KrJWkNcWsDziyTKVjrpPhRLMyXsr5`: 40.107 DOT - `14PiQ7uar36zPMgEckA7qWUahYheavRL6NHCbUCkXXRNrFSc`: 40.107 DOT - `14cFTN4jFFiiL1qszmGKZjokAdNr4YSD7Gf5rhZRA62TrtMb`: 40.107 DOT - `12bqgqerfH21x5hv85AJ9AiNFWXVmBLDoCvmz78MD4fgEP7Y`: 40.107 DOT - `15oXzySe6tjF2MumHfUodH8pFQWjy2hraRmXUJXXMKKY6p3F`: 40.107 DOT - `1ZVYsze5Ls3osofU6wWSp5dphr62Rj7YiL4NsXiZU3a298F`: 40.107 DOT - `12NCX9ZK1z9fxBfRraD6L4V86EmPipSerHnPcsj1k4hSkszg`: 40.107 DOT - `126X27SbhrV19mBFawys3ovkyBS87SGfYwtwa8J2FjHrtbmA`: 40.106 DOT - `15DL1EU6TpGDvL8HCNNU2ZDZdbcDUPiHYr1DBHBerUWMkJnT`: 40.106 DOT - `1bqBkjrbVc6nFbpZ2oqnbEKAs99CYSf2XVAwtGVWBRxDvNY`: 40.106 DOT - `13Ghf2T883ZobjngC1BAgR1BWvK2P7qP37gGxHDVFf3fjbmw`: 40.106 DOT - `152wswWPnwr1uLxqyENaesqjFtJcMwLT3dmrpb7KNt1PZ1PX`: 40.106 DOT - `14M94kYk31k2hY8MpnfNPRviJ4VcsFFjBhq7V2Fs9DzCVhXc`: 40.106 DOT - `16kkgkzjyJZL91WaL6GAUJnTZjiaowZcFyHAs5GWCNVqJimJ`: 40.106 DOT - `14mZVYo7jy13aHTiNMQZJzsii5CPsVEaMQwLXTEMLzkmxKH2`: 40.106 DOT - `13uvpozMRF7PCGbgPutm852Jt58nNBVUPdMFEQg5m7d1w8J8`: 40.106 DOT - `15cfSaBcTxNr8rV59cbhdMNCRagFr3GE6B3zZRsCp4QHHKPu`: 40.106 DOT - `123jNGxHk9ZV7oVVhFWFtMghNpmnmmTWxSpNxf8TTKzmCSQ2`: 40.106 DOT - `16MJX8HEwhbJwN9LCKLymW812eD9N97c5EkRNVjWzhFTwhBN`: 40.106 DOT - `13uvpozMRF7PCGbgPutm852Jt58nNBVUPdMFEQg5m7d1w8J8`: 40.106 DOT - `14mg5GK7RoiafH7djdKgZKxKewuhj8ds19bqjioaEHR6WhQ4`: 40.106 DOT - `1pzhyYR9gLk3GmwRtQESLkJCUXazFsAESgcbTRLc9q9hNuy`: 40.106 DOT - `16ZhiPmAt65atW7uvNSqyK1qitQL4FQUvYz8yYXfV1EGwVP1`: 40.106 DOT - `15kgSF6oSMFeaN7xYAykihoyQFZLRu1cF5FaBdiSDHJ233H5`: 40.106 DOT - `1L3j12S8rmd5GvJsxzBQzFKypYX5yV2kLrPJhacUYVrLvus`: 40.106 DOT - `13Ghf2T883ZobjngC1BAgR1BWvK2P7qP37gGxHDVFf3fjbmw`: 40.106 DOT - `13mEX6UD8t4L8YfsUxE8QjYFDkfEkAg2QpKWqKEfg5gZw3et`: 40.106 DOT - `13uvpozMRF7PCGbgPutm852Jt58nNBVUPdMFEQg5m7d1w8J8`: 40.106 DOT - `16Q4cR5vHLkoNqtqCZcdeKnZhY9a8AiXZAtemRJmMCpeiu82`: 40.106 DOT - `133uT5bf5xz8xMkCmwVBWpeHjN4NyfvfqwdpXu2oZnn29kEG`: 40.106 DOT - `13mm8mjuALSbyvfjfso22eexuFwL4MqMrcw1w5To9L52yb5h`: 40.106 DOT - `16kkgkzjyJZL91WaL6GAUJnTZjiaowZcFyHAs5GWCNVqJimJ`: 40.106 DOT - `12mRyiCp9zdh1wEVW5gLLiFBxDPKks72rRXmSupyEK3VAMLf`: 40.106 DOT - `12mRyiCp9zdh1wEVW5gLLiFBxDPKks72rRXmSupyEK3VAMLf`: 40.106 DOT - `13mm8mjuALSbyvfjfso22eexuFwL4MqMrcw1w5To9L52yb5h`: 40.106 DOT - `167vWTbKWmJhWUitgP1hGRZfaActDyZufCVu6vqUzrhQ2pS3`: 40.106 DOT - `15V75NT7bvs9YuVF6NTJynpTCswRshzwvcqPJZoaEJsBVxHi`: 40.106 DOT - `15VgqbuZGdwrpGjKkJMA9nE2gqLMHyQpWmE7k6dc4fQdRMXa`: 40.106 DOT - `12eMZTAnXEsyedXmsB6jDVRnF9Mq8ZrhLefRGhxPE4JwrPAS`: 40.106 DOT - `121k35TZKEpoQeKURnEgt2zqWsyDKxUJkTFuwpZeLoSYUe7o`: 40.106 DOT - `14PiQ7uar36zPMgEckA7qWUahYheavRL6NHCbUCkXXRNrFSc`: 40.106 DOT - `16ad3ehm2XsVQbQgqYPxicRB5nGinQU9zEKiCJ7ZVhRN9CyG`: 40.106 DOT - `123LuJKS65HaBbLSdDS46ByeC7bvQwA1iUhTpmjigQAfUKpK`: 40.106 DOT - `1316cTZeHz8HtEjaJRHu8sHbp9brtUmy2LiP74KZXgXhifry`: 40.106 DOT - `1316cTZeHz8HtEjaJRHu8sHbp9brtUmy2LiP74KZXgXhifry`: 40.106 DOT - `1QgMmM5QyTBVkC9cBNPVQszCTHjCBskFG1pny8zVprPSd1J`: 40.106 DOT - `1dwxEFdaRzBF1fpZqbXz71nLhJHvPi6a8eETjPSyC3Wrvom`: 40.106 DOT - `12wWLUd5qMzLFGqBsMnHLVFeTuYJwuo5ygMAxuSywrBX1XSF`: 40.106 DOT - `19C7X2ayEGaHbRb7obTd7u2crJhYm6W47XpyLC2jQBGdpif`: 40.106 DOT - `1316cTZeHz8HtEjaJRHu8sHbp9brtUmy2LiP74KZXgXhifry`: 40.106 DOT - `14DsLzVyTUTDMm2eP3czwPbH53KgqnQRp3CJJZS9GR7yxGDP`: 40.106 DOT - `1xgDfXcNuB94dDcKmEG8rE9x9JVoqozCBnnitkN9nAe3Nyx`: 40.106 DOT - `16kkgkzjyJZL91WaL6GAUJnTZjiaowZcFyHAs5GWCNVqJimJ`: 40.106 DOT - `16aQb7rHLB8UXzd2YSh56vjAELyyq8jYaj5QdAHjVjsA3ey9`: 40.106 DOT - `14jHouxT1VbhBDw93VW8Z89p139Qgu7ECHz3zxM2CpQEDJDB`: 40.106 DOT - `15fHj7Q7SYxqMgZ38UpjXS8cxdq77rczTP3JgY9JVi5piMPN`: 40.106 DOT - `149FXUmHgg75z4sk2LzFDyctNLHhzf2YxGMFHT7TakkbeQ7F`: 40.106 DOT - `12hAtDZJGt4of3m2GqZcUCVAjZPALfvPwvtUTFZPQUbdX1Ud`: 40.106 DOT - `13GtCixw3EZARj52CVbKLrsAzyc7dmmYhDV6quS5yeVCfnh1`: 40.106 DOT - `15ixta6FiXTBE8gXCTUNP3ahdYWcTuateHgB2czGg5EGDVMA`: 40.106 DOT - `15kgSF6oSMFeaN7xYAykihoyQFZLRu1cF5FaBdiSDHJ233H5`: 40.106 DOT - `13GtCixw3EZARj52CVbKLrsAzyc7dmmYhDV6quS5yeVCfnh1`: 40.106 DOT - `13Ghf2T883ZobjngC1BAgR1BWvK2P7qP37gGxHDVFf3fjbmw`: 40.106 DOT - `13Ghf2T883ZobjngC1BAgR1BWvK2P7qP37gGxHDVFf3fjbmw`: 40.106 DOT - `139Vbu9X3h4v7NTBVSpLijAvpWUoGhYwKmeuxaSJ9kQsD2SG`: 40.106 DOT - `128fHaGJDKeXNNjqamUTaLe5dpU41zpbBaQA6BW9VsPKpkH6`: 40.106 DOT - `15DL1EU6TpGDvL8HCNNU2ZDZdbcDUPiHYr1DBHBerUWMkJnT`: 40.106 DOT - `16agh1vhJ78MiJ7tjuTd9RzreMwBwTEu15x8kCDfJy1xBYUs`: 40.106 DOT - `16kkgkzjyJZL91WaL6GAUJnTZjiaowZcFyHAs5GWCNVqJimJ`: 40.106 DOT - `13SceNt2ELz3ti4rnQbY1snpYH4XE4fLFsW8ph9rpwJd6HFC`: 40.106 DOT - `1zhukWzj6pTskKUhDmyCaoJLuaHp5AVMDn5uLoNXTrw2gDR`: 40.106 DOT - `15fHj7Q7SYxqMgZ38UpjXS8cxdq77rczTP3JgY9JVi5piMPN`: 40.106 DOT - `12mRyiCp9zdh1wEVW5gLLiFBxDPKks72rRXmSupyEK3VAMLf`: 40.106 DOT - `13u5odFdy7uFmRLpbgtYGWeFy8rFkcD3bYfad49B81C31pwL`: 40.106 DOT - `16fUfF5mqL3cGGL3ai1CTL45UyNVTBHcbMkmuh5Va5M2yJ5p`: 40.106 DOT - `14mZVYo7jy13aHTiNMQZJzsii5CPsVEaMQwLXTEMLzkmxKH2`: 40.106 DOT - `1uamkTsQk6TwVAm6FvD7optu9fDPUh7GojEc2mZHym13Kcf`: 40.106 DOT - `14DsLzVyTUTDMm2eP3czwPbH53KgqnQRp3CJJZS9GR7yxGDP`: 40.106 DOT - `12CHAK3YxJG5pGW6JAGp6Daj8ruRfPwCNbPM7jU8mC2zh2qD`: 40.106 DOT - `123LuJKS65HaBbLSdDS46ByeC7bvQwA1iUhTpmjigQAfUKpK`: 40.106 DOT - `16k8FBUzGaAScYvewFB9g6WGt8Zms9oygPVKt7GioG4gimRp`: 40.106 DOT - `14QQcaXERr6kzwW55L4GKmN8tC8NJRoGt1jF5D8GMWoXdyaz`: 40.106 DOT - `13EAhGcpe93mqSFZQrQ4P2cfpdAo5txWc5UQVTfEKDoqZjhw`: 40.106 DOT - `15SN9iNKxCJJjQ5f6JXEDxiaS6bRHxxTZtsfm3wCSSjyoENg`: 40.106 DOT - `12mRyiCp9zdh1wEVW5gLLiFBxDPKks72rRXmSupyEK3VAMLf`: 40.106 DOT - `15SN9iNKxCJJjQ5f6JXEDxiaS6bRHxxTZtsfm3wCSSjyoENg`: 40.106 DOT - `15oXuEfGte2HPoxxWwz18er7LNFuLNEdXtNNk53dggkfFgCR`: 40.106 DOT - `16agh1vhJ78MiJ7tjuTd9RzreMwBwTEu15x8kCDfJy1xBYUs`: 40.106 DOT - `1EpEiYpWRAWmte4oPLtR5B1TZFxcBShBdjK4X9wWnq2KfLK`: 40.101 DOT - `13Ghf2T883ZobjngC1BAgR1BWvK2P7qP37gGxHDVFf3fjbmw`: 40.1 DOT - `13SceNt2ELz3ti4rnQbY1snpYH4XE4fLFsW8ph9rpwJd6HFC`: 40.099 DOT - `14DsLzVyTUTDMm2eP3czwPbH53KgqnQRp3CJJZS9GR7yxGDP`: 40.087 DOT - `1481qDmGELXNaeDi3jsLqHUSXLpSkaEg3euUX8Ya3SPoDLmt`: 40.075 DOT - `16Drp38QW5UXWMHT7n5d5mPPH1u5Qavuv6aYAhbHfN3nzToe`: 40.074 DOT - `14onpjYNgzDZwY57Y3w5cwwnFyp6K62mNNbgq4Xhw7zNG9iX`: 40.07 DOT - `15nKYvAm8Yu9QVK65JWrhfyabhHkWywg21X9gX4GFJo3v4cT`: 40.069 DOT - `138MRRCFovYvetAhv37SnNsZoCVyghYoUArhBzMzKFfFGeMP`: 40.067 DOT - `13u5odFdy7uFmRLpbgtYGWeFy8rFkcD3bYfad49B81C31pwL`: 40.067 DOT - `12NGmpotx1WxkZ6RrqZeMBerBUB2aa2fBCrhSPvbAJWAcF33`: 40.067 DOT - `1EpEiYpWRAWmte4oPLtR5B1TZFxcBShBdjK4X9wWnq2KfLK`: 40.067 DOT - `13YMTEPKAxPRiyaZdMKrozeNT9x1Pa5h7aExebCdi6nc3Qqd`: 40.067 DOT </p> </details> - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
To be merged into the AHM working branch. Changes: - Configure the RC Migrator pallet as call filter for the Polkadot Relay chain - Test - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Dónal Murray <donalm@seadanda.dev>
To be merged into the AHM working branch. # Pallet Nomination Pools The nomination pools pallet has 15 storage items of which 14 can be migrated without any translation. # Storage Values All nine storage values are migrated as is in a single message. # Storage Maps The `BondedPools` map needs translation for its commission logic. The Commission struct contains an absolute relay timestamp [throttle_from](https://github.com/paritytech/polkadot-sdk/blob/bf20a9ee18f7215210bbbabf79e955c8c35b3360/substrate/frame/nomination-pools/src/lib.rs#L737) and the commission change [min_delay](https://github.com/paritytech/polkadot-sdk/blob/bf20a9ee18f7215210bbbabf79e955c8c35b3360/substrate/frame/nomination-pools/src/lib.rs#L922). The translation for both happens upon arrival on the Asset Hub. Ideally, it would be done on the Relay chain side since we have more compute power there but it is not possible since the timestamp translation depends on the Relay number upon arrival on Asset Hub. The timestamp is translated in [rc_to_ah_timestamp](https://github.com/polkadot-fellows/runtimes/blob/5af776e1443b5e7eb17b6e9d87ef40311afaf6f9/pallets/ah-migrator/src/staking/nom_pools.rs#L127) and the `min_delay` is directly passed to `RcToAhDelay`. The other five storage maps are migrated as it. ## User Impact Impact here is negligible and only for pool operators - not members: - Pool commission change rate (measured in blocks) could be decreased by one block. - Pool operators may be able to change the commission rate one block later than anticipated. This is due to the nature or translating blocks of two different blockchains which does not yield unambiguous results. - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
meant to be merged into `dev-asset-hub-migration` branch referenda pallet migration --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
To be merged into the AHM working branch. Changes: - Clippy cleanup - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
meant to be merged into `dev-asset-hub-migration` The migration for the Scheduler pallet from Relay Chain to Asset Hub. --------- Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
To be merged into the AHM dev branch. There should be no user impact here - both pallets have their storage migrated exactly as is and the used times are always in Epoch. - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
WIP to be merged into the AHM dev branch. TODO write user impact. - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
meant to be merged into ahm dev branch The data migration for the conviction-voting pallet. --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
To be merged into the AHM working branch - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
meant to be merged into AHM dev branch Integrates asset rate pallet into Polkadot Asset Hub and introduces the data migration for the pallet. No user impact, no security concerns. --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
meant to be merged into ahm dev branch Migration bounties pallet data from RC to AH. --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Merging into the AHM working branch. Depends on #579 # Pallet Vesting Pallet vesting has one storage map to hold the vesting schedules and one storage value to track the current version of the pallet. The version can be easily migrated, but for the schedules it is a bit difficult. ## Storage: Vesting The vesting schedules are already measured in relay blocks, as can be seen [here](https://github.com/polkadot-fellows/runtimes/blob/b613b54d94af5f4702533a56c6260651a14bdccb/system-parachains/asset-hubs/asset-hub-polkadot/src/lib.rs#L297). This means that we can just integrate the existing schedules. The only possibly issue is when there are lots of pre-existing schedules. The maximal number of schedules is 28; both on Relay and AH. We cannot use the merge functionality of the vesting pallet since that can be used as an attack vector: anyone can send 28 vested transfers with very large unlock duration and low amount to force all other schedules to adapt this long unlock period. This would reduce the rewards per block, which is bad. For now, we are writing all colliding AH schedules into a storage item for manual inspection later. It could still happen that unmalicious users will have more than 28 schedules, but as nobody has used the vested transfers on AH yet. Q: Maybe we should disable vested transfers with the next runtime upgrade on AH. ## Storage: StorageVersion The vesting pallet is not using the proper FRAME version tracking; rather, it tracks its version in the `StorageVersion` value. It does this incorrectly though, with Asset Hub reporting version 0 instead of 1. We ignore and correct this by writing 1 to the storage. ## User Impact This affects users that have vesting schedules on the Relay chain or on Asset Hub. There exists a risk that the number of total schedules exceeds 28, which means that they will not fit into the storage anymore. We then prioritize the schedules from AH and pause and stash all schedules that do not fit (up to 28). - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
The [configuration of the allowed_call_filters](https://github.com/polkadot-fellows/runtimes/blob/daf6723d4390db875c8e3d4a8bf8818f5e837351/relay/polkadot/src/lib.rs#L1582C1-L1583C70) doesn't match [their pallet documentation](https://github.com/polkadot-fellows/runtimes/blob/daf6723d4390db875c8e3d4a8bf8818f5e837351/pallets/rc-migrator/src/lib.rs#L338C3-L343C84) - judging by the names and documentation they should be reversed. Same for the [filter implementation](https://github.com/polkadot-fellows/runtimes/blob/daf6723d4390db875c8e3d4a8bf8818f5e837351/pallets/rc-migrator/src/lib.rs#L1116C4-L1122C4).
To be merged into the AHM working branch. Quote from `accounts.md`: # Account Migration Accounts are migrated with all their balance, locks and reserves at the beginning of the Asset Hub migration. ## User Impact Users need to be aware that all of their funds will be moved from the Relay chain to the Asset Hub. The Account ID will stay the same. This ensures that normal user accounts will be to control their funds on Asset Hub. ## Sovereign Account Translation For parachain sovereign accounts, it is not possible to just use the same account ID. The sovereign account address of a parachain is calculated differently, depending on whether it is the account on the Relay or a parachain (like Asset Hub). There are different kinds of sovereign accounts. In this context, we only focus on these parachain sovereign accounts: - On the Relay: derived from `"para" ++ para_id ++ 00..` - On the Asset Hub and all other sibling parachains: derived from `"sibl" ++ para_id ++ 00..` Our translation logic inverts the derivation and changes the prefix from `"para"` to `"sibl"` for all accounts that match the pattern `"para" ++ para_id ++ 00..`. The full list of translated accounts is in [this CSV file](./sovereign_account_translation.csv). It is advised that parachains check that they can control their account on Asset Hub. They can also forego this check if they do not need control thereof - for example when they are not holding any funds on their relay sovereign account. However, please note that someone could still send funds to that address before or after the migration. Example for Bifrost: this is the [relay sovereign account](https://polkadot.subscan.io/account/13YMK2eeopZtUNpeHnJ1Ws2HqMQG6Ts9PGCZYGyFbSYoZfcm) and it gets translated to this [sibling sovereign account](https://assethub-polkadot.subscan.io/account/13cKp89TtYknbyYnqnF6dWN75q5ZosvFSuqzoEVkUAaNR47A). --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
To be merged into the AHM branch TODO: write user impact and add tests - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: muharem <ismailov.m.h@gmail.com>
Set up a leaner migration testing framework for single pallets and implementing it for pallet `preimage`. This PR introduces the traits `RcMigrationChecks` and `AhMigrationChecks` for checks before and after a single pallet's migration. Upon implementing these traits for a pallet, you can use `rc_migrate` and `ah_migrate` functions to run a test on the correctness of the pallet's migration.
meant to be merged into ahm dev branch Stage management for the Asset Hub migration --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Merges into the AHM branch - [x] Does not require a CHANGELOG entry Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
WIP --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Migrate accounts with free balance less than RC ED but with enough balance for AH ED. It turns out the Kusama has around 38k accounts with free balance less than RC ED but more/equal to AH ED. current accounts migration ignores these accounts. With the change presented in this PR we can migrate those accounts too.
…mark (#916) This PR reverts the workaround introduced in #910 and bumps EPMB pallet version to `0.3.2` to include SDK fix paritytech/polkadot-sdk#9778
| Weight::from_parts(10_000_000, 1000) | ||
| } | ||
| fn set_unprocessed_msg_buffer() -> Weight { | ||
| Weight::from_parts(10_000_000, 1000) |
There was a problem hiding this comment.
@ggwpez seems like we never generated weights for this pallet
There was a problem hiding this comment.
we discussed that we can have one more release for Polkadot. so this is not a blocker.
|
Review required! Latest push from author must always be reviewed |
| /// Failed to send XCM message to AH. | ||
| XcmError, |
There was a problem hiding this comment.
What's the difference between this error and XcmSendError
| log::error!( | ||
| target: LOG_TARGET, | ||
| "Received error response for query id: {query_id}; response: {response:?}" | ||
| ); |
There was a problem hiding this comment.
I guess we just check the logs for failed ACKs. Else we could change the state of the pending message so we know which ones we should resend
| /// Resend a previously sent and unconfirmed XCM message. | ||
| #[pallet::call_index(4)] | ||
| #[pallet::weight(T::RcWeightInfo::resend_xcm())] | ||
| pub fn resend_xcm(origin: OriginFor<T>, query_id: u64) -> DispatchResultWithPostInfo { |
There was a problem hiding this comment.
Is it okay if we resend it in another migration stage? Or do we need to resend it in the same one?
|
|
||
| ### Translation of the Delay | ||
|
|
||
| The [delay of a ProxyDefinition](https://github.com/paritytech/polkadot-sdk/blob/7c5224cb01710d0c14c87bf3463cc79e49b3e7b5/substrate/frame/proxy/src/lib.rs#L77) is currently measured in Relay Chain blocks. This will change and be measured in Asset Hub blocks after the migration. The delays are translated by dividing them by two. |
There was a problem hiding this comment.
Should we not use relay block numbers for this? What happens when AH gets elastic scaling later and block times are reduced?
There was a problem hiding this comment.
this is actually not done this way. it uses RC block provider. you can see it in proxy pallet config setup
| It could still happen that unmalicious users will have more than 28 schedules, but as nobody has | ||
| used the vested transfers on AH yet. | ||
|
|
||
| Q: Maybe we should disable vested transfers with the next runtime upgrade on AH. |
There was a problem hiding this comment.
Was this answered? Seems reasonable to disable them before the migration if that would simplify things
There was a problem hiding this comment.
its filtered. you can see it AH runtimes setup
| if T::MaxAhWeight::get() | ||
| .any_lt(T::AhWeightInfo::receive_proxy_announcements(batch.len() + 1)) | ||
| { | ||
| log::info!( | ||
| target: LOG_TARGET, | ||
| "AH weight limit reached at batch length {}, stopping", | ||
| batch.len() | ||
| ); | ||
| if batch.is_empty() { | ||
| return Err(Error::OutOfWeight); | ||
| } else { | ||
| break; | ||
| } | ||
| } | ||
|
|
||
| if batch.len() > MAX_ITEMS_PER_BLOCK { | ||
| log::info!( | ||
| target: LOG_TARGET, | ||
| "Maximum number of items ({:?}) to migrate per block reached, current batch size: {}", | ||
| MAX_ITEMS_PER_BLOCK, | ||
| batch.len() | ||
| ); | ||
| break; | ||
| } |
There was a problem hiding this comment.
All of these checks also look like things that would benefit from some de-duplication
| - removed `LocalPlurality` from the list, | ||
| - kept System Parachains and local Root (which continue to get free delivery). | ||
| 3. Did NOT change the destination/beneficiary of XCM delivery/transport fees. They will continue to go | ||
| to the local Treasury account, even if the Treasury moves to Asset Hub. See 2nd TODO: @acatangiu above for details. |
There was a problem hiding this comment.
| to the local Treasury account, even if the Treasury moves to Asset Hub. See 2nd TODO: @acatangiu above for details. | |
| to the local Treasury account, even if the Treasury moves to Asset Hub. See TODO: @acatangiu above for details. |
| ## TODOs | ||
|
|
||
| -[ ] TODO: @acatangiu. Post migration we need to switch all system chains XCM transport fees beneficiary from | ||
| `RelayTreasuryLocation` to the new `AssetHubTreasuryLocation`. Does not necessarily need to be done |
There was a problem hiding this comment.
All extra funds generated between AHM and when the beneficiary change will be moved again? Is that the idea?
Notes on TS Migration TestsMerged and working in CIThe following pallets are covered with migration tests (though some functionality still needs to be implemented):
Work in ProgressPostponed
Some of these tests are 80–90% complete and will require another review pass, but they already provide solid guarantees for the covered pallets. Missing CoverageThe following pallets still lack migration test coverage:
|
Relates to the comment: #856 (comment) cc: @muharem
| /// | ||
| /// The base weight is only included for the first imported account. | ||
| pub fn weight_ah_receive_account(batch_len: u32, account: &AccountFor<T>) -> Weight { | ||
| let weight_of = if account.is_liquid() { |
There was a problem hiding this comment.
This looks off, at least conceptually:
item_weight_of subtracts the base weight for n == 0 if batch_len != 0. This seems only sensible if we can guarantee that the base weight of liquid and non liquid accounts is the same.
There was a problem hiding this comment.
the base weights should be similar. but even if they are not, this is only used to make sure we do not send more than AH (MaxAhWeight) can process in a single block. {MaxAhWeight = 80% of AH_MessageQueueServiceWeight}, so there is 20% buffer. Plus these messages batched. For accounts its 4 messages per block. They should weight around 20% of that capacity. We have a problem if there is a miscalculation that leads for that 20% to be > AH_MessageQueueServiceWeight. And we should catch this at our dry run tests.
| Err(_) => TransactionOutcome::Rollback(Err(account)), | ||
| } | ||
| }) | ||
| .expect("Always returning Ok; qed"); |
There was a problem hiding this comment.
Not really a proof. We can still have an error if we reached max transaction level.
There was a problem hiding this comment.
Hm true. I think that is the only case where it would error.
| .ok_or(Error::<T>::FailedToCalculateCheckingAccount)? | ||
| .checked_sub(balances_before.total_issuance) | ||
| .ok_or(Error::<T>::FailedToCalculateCheckingAccount)? | ||
| .checked_sub(balances_before.checking_account) |
There was a problem hiding this comment.
Overall this makes sense to me. For the balances_before.checking_account value ... I would like to verify my understanding: What value are we expecting here? What was in the AH checking account before the migration?
There was a problem hiding this comment.
For the balances_before.checking_account value ... I would like to verify my understanding: What value are we expecting here? What was in the AH checking account before the migration?
yes
you can see a more detailed explanation here (including a visual representation/drawing) https://github.com/polkadot-fellows/runtimes/blob/oty-dev-asset-hub-migration-2506/pallets/rc-migrator/src/accounts.md#tracking-total-issuance-post-migration
There was a problem hiding this comment.
Maybe I am blind, but I did not see an explanation on ah_checking_before specifically. Will look another time, later.
Good catch Basti! TLDR: parachains and dapps need to disable KSM/DOT transfers during migration and switch reserve location of KSM/DOT after migration. @franciscoaguirre while I can find guides for parachains and dapps to prepare for migration in terms of KSM/DOT reserves:
I don't see any explicit communication on how dapps and parachains should disable KSM/DOT transfers during migration, and/or guides on how to do it. (I expect Paraspell SDK to do this automatically for dapps using it) L.E.: I found some info in the official AHM support article saying how XCM transfers of KSM/DOT are not working during migration, but still can't find any guide for making dapps and parachains to actually limit/disable them and have a local error rather than them failing along the way resulting in trapped funds. |
eskimor
left a comment
There was a problem hiding this comment.
I focused mostly on balances in the review and all seems to make sense/ has the right safe guards. Transactional behavior and weight tracking seems also ok (except for one minor conceptual incorrectness, which is likely fine in practice).
I understand that we have done the migration on Paseo already - have we upgraded the Paseo runtimes from assethub since then? I saw tests for runtime upgrades after the fact, but this can't be tested enough really.
Likewise, double-checking that a runtime upgrade works in a failed migration state is equally crucial.
Will check on a few more things, but approving for now.
Hmm, very good point, the runtime upgrade can be done on-chain by OpenGov (either Root track or WhitelistedCaller track),
OpenGov (Referenda and conviction voting) depends on accounts/balances and total/inactive issuance, so basically, when the account migrations is done There is also possibility to So iiuc, if the migration is stopped on the RC before If the migration is stopped on the RC after |
yes
yes
we should have e2e tests for this. @rockbmb can you please confirm? |
acatangiu
left a comment
There was a problem hiding this comment.
Covered in my review(s):
- general migration logic: command&control, flow, state machine, etc
- XCM communication for migration
- balances migration
- RC and AH configs (XCM, governance, treasury, etc) before/during/after migration
- emulated tests
- integration tests
@muharem we don't; PET has an open issue for this (open-web3-stack/polkadot-ecosystem-tests#298), but it was not possible to do it in time for Kusama. Aiming for Polkadot now. |
can we just make a manual test first with Kusama? |
// only to let all CIs to execute for now - [x] Does not require a CHANGELOG entry --------- Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io> Co-authored-by: muharem <ismailov.m.h@gmail.com> Co-authored-by: Branislav Kontur <bkontur@gmail.com> Co-authored-by: GitHub Action <action@github.com>
Refund tx fee for schedule migration call
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
|
/merge |
|
Enabled Available commands
For more information see the documentation |
|
This is already merged, but some context on testing: Aside from the many Rust and integration tests added in this PR, PET was also used to test migration dry-runs. Here is a descriptive lists of the kinds of tests run on the KAH runtime. Here is an example run of PET in the dry-running repo's CI. TL;DR aside from a few false positives (correcting them is WIP), runtimes look solid, modulo Dijkstra's quote on testing. |
A high-level look of the migration is this:
You can see the main components being the
pallet_rc_migratorandpallet_ah_migrator. The RC migrator pallet takes data out of the RC storage and enqueues it for AH through XCM::Transact calls to the AH migrator pallet.Each pallet on the RC side has a migrator module that handles the removal of the old data and creation of the messages for AH. AH then processes these messages through the MQ pallet and acknowledges back to RC that they were processed (not shown in the graphic).
There is rate-limiting code in place to prevent it from overloading the DMP queue or Asset Hub.
For Reviewers
The review comments are being addressed in #915.
We need eyes on the Core part that is described below.
Core STM
The main stage transitions look like this. The only manual part is the scheduling call by the fellowship that kicks off the procedure:
Pendingwait for Fellowship to callschedule_migration(when)-> ScheduledScheduledwaiting for scheduled block ->WaitingForAHWaitingForAHAH replies that it is ready ->WarmUpWarmUpwaiting for warm-up end ->StartingStartingdoing some startup initialization ->DataMigrationDataMigrationMAIN PART all pallet migrations finish ->CoolOffCoolOffwaiting for cool-off to end ->SignalMigrationFinishSignalMigrationFinishsend signal to AH that we are done ->MigrationDoneMigrationDonefinished, unlock all Pallets.State Machine
The core State Machine lives in the
RcMigratorpallet in itson_initialize:runtimes/pallets/rc-migrator/src/lib.rs
Lines 1338 to 1342 in 8ef1cc7
On each block the Rc Migrator checks the current state and possibly progresses to the next one. It will only do a single state transition per block to reduce complexity of the code. Generally, there are three states per pallet (called stages in the code). Some pallets have more then three, but it is generally still a multiple of three and always follows the pattern of
PalletInit,PalletOngoing { cursor },PalletDone:runtimes/pallets/rc-migrator/src/lib.rs
Lines 239 to 244 in 8ef1cc7
Each pallet then has one migrator struct (eg.
MultisigMigrator<T>) that migrates the storage items from the Relay side. It lives in the module of its namerc-migrator/src/multisig.rsrespectively.Taking Multisigs as an example; you can find two functions in the migrator struct:
migrate_manyandmigrate_single. The former one is from thePalletMigrationtrait. Themigrate_singlefunction is done by convention to keep the code readable. Themigrate_manydoes the following:XcmBatchAndMeterAH Side
The main logic on Asset Hub lives in the
AhMigratorpallet. It contains at least one extrinsic per pallet to receive the incoming data from the RC. We are using extrinsics since RC is wrapping the data into XCM::Transacts:runtimes/pallets/ah-migrator/src/lib.rs
Lines 650 to 653 in 9a20b12
This calls into the
multisigmodule to integrate the data into the storage. Per convention we are emitting events for each item and the whole batch.The main task of the AH side is to integrate the data. In the example of the multisig it is special since we do not migrate the multisig, but only the reserved account which will be unreserved on AH.
Messaging Logic
The messaging is ACK based. The Relay sends message to Asset Hub up to a specified limit (50). The limit can be adjusted on the fly through a dedicated dispatchable call, restricted to Admin and Manager roles. The limit may be overstepped by the configured batch-size (10), so in total there should never be more than 60 DMPs queued. Each DMP is set to at most 50KB.
The Relay Chain migrator creates a query for each XCM data message using the XCM query–response primitives and attaches a response status request from Asset Hub to the message. The XCM executor on Asset Hub then sends back the response status for every message with such a request.
The Relay migrator stores each data message sent to Asset Hub until it receives confirmation of successful processing. It also provides a dispatchable call, available only to Admin and Manager roles, to resend any unconfirmed messages.
A bug that prevented certain messages from being accounted for during a particulat pallet migration caused the Westend AHM stall. The system was queuing too many DMPs, which forced WAH to author an overweight block while attempting to process them all at once.
Call Filtering
Pallet calls are filtered during the migration for three reasons:
Both RC and AH provide call filters for
Before,DuringandAfterthe migration. As you can see, most things are disabled during the migration on both chains. After the migration, Relay chain keeps pretty much everything disabled since it was now enabled on AH:This is implemented by returning three bools, one for each phase:
runtimes/system-parachains/asset-hubs/asset-hub-polkadot/src/ah_migration/call_filter.rs
Lines 81 to 83 in 88f4aaf
Testing and Dry-Runs
https://github.com/paritytech/ahm-dryrun is the repository for testing infrastructure for AHM. As of now, the following is being tested there (as a part of "AHM Flow (all steps)" nightly action):
Each action produces a table with the summary of the test results:
And 4 snapshot files (2 real databases, 2 try-runtime snapshots) which can be used to re-run any of the tests, or run the pre/post migration chain, locally.
Full Changelog
AccountStatetype update ([AHM] Missing TODOs andAccountStatetype update #742)RootLocationfor WaivedLocation for AHP ([AHM] MissingRootLocationfor WaivedLocation for AHP #798)SecretaryEntitiesfor AHP ([AHM] MissingSecretaryEntitiesfor AHP #797)mockedmigration across all involved pallets ([AHM / ah-migrator]: Implement RC->AH account translationmockedmigration across all involved pallets #807)RelayTreasuryPalletAccount(instead of AH, it should be BH) ([AHM] Fix snowbridge test - wrong import ofRelayTreasuryPalletAccount(instead of AH, it should be BH) #831)update_ah_msg_processed_count(or missing benchmark?) #892// only to let all CIs to execute for now