Support account code updates

[Fumuran]

This discussion is started to... well, discuss two things:

1. Explain in detail how the account code is (not) updated in the current implementation (or, more precisely, what preliminary actions have already been done to support this functionality).
2. And, which is much more important, come up with the mechanism which will actually allow us to update the account code.

---

How is the account code being updated right now

The general rule which is followed during the transaction is that the code commitment stored in the account data block could be changed only in the epilogue, after all the user code has been executed (as opposed to the other commitments which could be updated during the execution).

In fact we have only two actual values for the code commitment: it is NEW_CODE_ROOT stored in the bookkeeping section and the ACCOUNT_CODE_COMMITMENT stored in the account data block.

In memory.masm file we have the following getter/setter pairs:

• For the "new" code commitment: get_new_acct_code_commitment + set_new_acct_code_commitment -> work with the NEW_CODE_ROOT value.
• For the "current" code commitment: get_acct_code_commitment + set_acct_code_commitment -> work with the code commitment in the account data block.
• For the initial code commitment: get_init_account_code_commitment (set_acct_code_commitment procedure is used as a setter here) -> work with the code commitment in the account data block.

Note

Since the code commitment could be updated only in the epilogue, the getters and setters for the current and initial values are doing exactly the same, returning the exact same value.

The user facing API (in miden::account) consists of two procedures: get_initial_code_commitment and get_code_commitment, which return the same value from the account data block. Our api.masm file contains two corresponding procedures for them. For now user doesn't have an ability to modify the account code and, hence, obtain the "current" code commitment.

The pipeline of the code update looks like so:

• In the prologue.masm we set the new code commitment (NEW_CODE_ROOT) to the value from the current (initial) code commitment (ACCOUNT_CODE_COMMITMENT), making the new code commitment to have an initial commitment as a "default" value.
• During the transaction execution the code could be updated with the (unimplemented) account::set_code procedure. Looks like inside this procedure the new code commitment had to be calculated and the NEW_CODE_ROOT had to be set to this newly computed value. The ACCOUNT_CODE_COMMITMENT value remains unchanged.
• In the epilogue.masm during the transaction finalization we update the code commitment: we set the code commitment form the account data block (ACCOUNT_CODE_COMMITMENT) to the potentially updated new code commitment (NEW_CODE_ROOT), so that the new code commitment will be initial in the next transaction.

Refactor proposals

1. Update the NEW_CODE_ROOT

IMO it will be less confusing if we leave only the "initial" and "current" commitment notions, so the NEW_CODE_ROOT could become the CURRENT_CODE_ROOT (with the renaming of the corresponding getters/setters) — it will contain the latest code commitment and will be updated the same way as the account storage commitment (using the caching feature with the dirty flag).

2. Merge the "initial" and the "current" procedures

The procedures, which now have "Initial" and "current" meaning (but essentially doing the same) should be merged so that there will be only one pair of the "initial" getters/setters in the memory and just one "initial" getter in the API (CURRENT_CODE_ROOT will play the role of the "current" value instead).

---

The saddest thing here is that everything I wrote above could be useless, since any idea of how can we implement the code update I have will require to rework this setup.

For now the biggest (probably stupid) question is why do we have to preserve the account code commitment in the account data block during the transaction? I think understanding of this rule will help me to understand what else could be reworked and what should remain the same.

Important

We should remember that because any account procedure potentially could have the access to the account storage, any addition/deletion/update of the procedure should be followed by the corresponding storage change.

[bobbinth]

As far as code updates go, I think there are a few things to think through:

1. Which accounts should be able to update code? Right now, we have 4 types of accounts: fungible/non-fungible faucet, regular account with immutable code, and regular account with mutable code. Faucets are considered to be immutable. There is an argument to be made that maybe we don't need this distinction at all: all accounts should be updatable and it is up to the account to specify whether it is updatable or not (via its interface).
2. What types of updates do we want to support. For example:
   a. Should users be able to only add new procedures? Or should they be able to remove procedures as well? (note that modifying procedures in our case doesn't make sense because changing underlying procedure code changes its MAST root and in a way is equivalent to removing an old procedure an adding a new one).
   b. Should the users be able to "swap" the authentication procedure? This may be useful in some contexts - e.g., a network account starts out with one auth procedure so that owners can fix bugs, but after some time, the owners can make the account immutable and effectively give up control by swapping the auth procedure to something else (though, maybe there are other ways to achieve the same functionality).
3. Assuming we've answered the above questions, what should the mechanism be for updating account code? As mentioned in the OP, adding procedures frequently would require adding more storage slots. Similarly, removing procedures may require removing storage slots. So, we'll need to come up with a set of kernel APIs that make sure we can both update the code and update the storage layout. Maybe we'd need something like add_component procedure that takes information about new procedures/storage slots to be added and performs the update. And if we decide to support procedure removals, we'll need to have something similar for that too.

[mmagician]

There is an argument to be made that maybe we don't need this distinction at all: all accounts should be updatable and it is up to the account to specify whether it is updatable or not (via its interface).

I think this would simplify things a lot. E.g., our current definition of what a faucet is seems quite rigid: it only supports one type of an asset (e.g. programmable assets would likely require a new account type). Furthermore, if the user starts with the FungibleFaucet interface but adds more procedures to it - what is the value in still calling it FungibleFaucet?

Or should they be able to remove procedures as well?

Good question - I wonder when removing procedures would ever be useful (aside from cleaning up storage)?

Should the users be able to "swap" the authentication procedure?

I think upgrading the authentication procedure is a pretty common use case for applications as they progress from training wheels stage to production. (Actually, I think this applies to any procedure - and while under the hood it's equivalent to add+remove as you write, maybe it makes sense to expose "upgrade" at the user level)

[bobbinth]

I wonder when removing procedures would ever be useful (aside from cleaning up storage)?

while under the hood it's equivalent to add+remove as you write, maybe it makes sense to expose "upgrade" at the user level

The main issue with removing procedures is that it may break accounts/notes that rely on these procedures. But the authentication procedure is different because it is not called directly by user code. So, swapping (or upgrading) authentication procedures is pretty straight-forward, but "upgrading" other procedures is a bit more tricky as it may have wide-reaching implications.

[PhilippGackstatter]

There is an argument to be made that maybe we don't need this distinction at all: all accounts should be updatable and it is up to the account to specify whether it is updatable or not (via its interface).

I agree this would be very nice and simpler to express at the interface level rather than at the AccountType level.

I think upgrading the authentication procedure is a pretty common use case for applications as they progress from training wheels stage to production. (Actually, I think this applies to any procedure [...]

I agree. I think we need to support removal of procedures even just to allow removing procedures that have been discovered to contain a security bug.
As it is now, the auth procedure is part of account code, and so supporting removal/adding of auth procedure probably means we can easily support adding/removing non-auth account procedure as well.

3. So, we'll need to come up with a set of kernel APIs that make sure we can both update the code and update the storage layout. Maybe we'd need something like add_component procedure that takes information about new procedures/storage slots to be added and performs the update. And if we decide to support procedure removals, we'll need to have something similar for that too.

Agreed, we need both code and storage updates. Since we don't have "account components" at the protocol/tx kernel level, only "raw" storage slots, I probably wouldn't call the API anything "component" related, though.

On a high-level, the question is at which point during the transaction lifecycle we should allow upgrading code & storage. The prologue requires that the account data provided via the advice provider matches the initial account commitment from the transaction stack input. Therefore, at the beginning of a transaction, we always have to provide the "previous" state of the account and cannot already provide an updated one. That leaves any point after the prologue for upgrades.

I imagine upgrading network accounts would have to be done via note scripts, since tx scripts cannot easily be provided. For example, a network account may restrict who can upgrade it by requiring a special note to be sent to it (which has some authentication mechanism), while it allows everyone else to call its other APIs (e.g. swap assets).

For local accounts, on the other hand, the tx script seems like the best place to run upgrades. In both cases, we'd probably want to let the upgrade only come into effect at the end of the transaction. The reason is that we'd otherwise have to support swapping account code during a transaction, and similarly for storage which would be a significant amount of complexity for the transaction kernel and the host. At the same time, the benefit of having an upgrade take effect immediately is not clear to me, so only letting it take effect in the epilogue seems fine.

Assuming we agree on this, we can think about what APIs would look like. Starting simply, I wonder if account_add_procedure and account_remove_procedure as well as account_add_storage_slot and account_remove_storage_slot would be sufficient.

[bobbinth]

As it is now, the auth procedure is part of account code, and so supporting removal/adding of auth procedure probably means we can easily support adding/removing non-auth account procedure as well.

Agreed that that once we support updates of auth procedures, supporting removal of other procedures won't be difficult as well. I am still worried about wider implications of allowing removal of procedures from accounts, and at the very least I'd do that as a follow up to the basic code update capabilities.

Assuming we agree on this, we can think about what APIs would look like. Starting simply, I wonder if account_add_procedure and account_remove_procedure as well as account_add_storage_slot and account_remove_storage_slot would be sufficient.

This could work but one comment: we should probably have a special procedure for updating auth procedures. It could be something like set_auth_procedure (to set any of the existing procedures as as the auth procedure for the account), or probably a better approach would be to have something like replace_auth_procedure to explicitly replace the auth procedure with a new procedure.

Also, I'm assuming these procedures would just record additions/removals and we'd actually process them in bulk in the epilogue (after the auth procedure has been executed).

[PhilippGackstatter]

we should probably have a special procedure for updating auth procedures

If we have per-procedure updates, then I agree, that seems safer.

From the user perspective, I actually wonder if we need dedicated kernel APIs at all. Adding code and storage could look like this (assuming local account & transaction for now):

• Consider the BasicWallet account component with its two procedures (and no storage). Adding just one of its procedures to the account code may be useful in some scenarios, but I think we should generally consider per-account component additions (or later removals). So, either adding the entire BasicWallet or not.
• Once a user has selected a component to add, they would:
  • call TransactionArgs::add_account_component(&mut self, component: AccountComponent) which:
    • stores the components procedure MAST roots into the advice inputs under a well-known ADD_CODE_PROCEDURES_KEY, which is the sequential hash of the to-be-added MAST roots.
    • store the components storage slot names into the advice inputs under ADD_STORAGE_SLOTS_KEY. Additionally, for every slot name, it inserts the slot value or the map entries into the advice inputs under the (hash? of the) slot name key as well.
  • After note and tx scripts have executed, in finalize_transaction, before execute_auth_procedure, the epilogue:
    • looks up the well-known keys ADD_CODE_PROCEDURES_KEY and ADD_STORAGE_SLOTS_KEY in the advice inputs and adds the procedures and slot keys to the in-kernel account delta, looking up the storage slot values and map entries as appropriate.
  • When the auth procedure runs next:
    • it can introspect the transaction's account delta (we don't have this yet) to check whether procedures/slots have been added (or later removed). This would allow certain account auth components to disallow updates by default, for example.
    • When it computes the account delta commitment for signing, the constructed TransactionSummary would contain the added procedures/storage slots. This would allow a client/wallet to implement logic that disallows updates (if not already done in the auth procedure) or for a user to check if the changes are as expected. Therefore, the procedure/slot data in the advice inputs is covered by a user signature, effectively preventing a malicious remote prover from changing the advice inputs.
  • At some point during this process, the transaction host would have to look up the procedure MAST roots in the MastForestStore to be able to add the "MASM instructions" / part of MastForest to the host-delta. The in-kernel delta only ever needs to see the MAST roots, since it only needs the commitment to them. I think this is relevant for all accounts, but especially for public accounts whose newly added procedures would have to be part of the delta so the network has the latest code.

Notes:

• I'm not sure whether making users add the data to the advice inputs (or hardcoding it into a tx script) and then calling a kernel API to add procedures/slots would have any benefits compared to the above. The tx script isn't directly covered by a user signature anyway, only its effects are (e.g. modifications to the account or notes). So the outcome should effectively be the same but the above should be much nicer to use.
• Depending on how easy it is to enforce that an account has exactly one auth procedure with the above, we could alternatively add another well-known REPLACE_AUTH_PROCEDURE_KEY.
• For network transactions, the note script would have to store the procedure MAST and slot data somehow.
  • The slot data could be part of the note inputs/payload and insert it into the advice inputs from there.
  • Perhaps the code could be made part of the underlying MastForest in the NoteScript, which would automatically be added to the ScriptMastForestStore in the transaction. The alternative is to store it in its note inputs/payload, and then insert it into the advice inputs from there. I think this would require serializing parts of a MastForest to felts, which seems a bit ugly.
• The above should similarly allow for removals with a corresponding API.
• Supporting per procedure and per storage slot operations could be done by adding dedicated functions to TransactionArgs, with the component ones being recommended.

Edit: I think avoiding well-known keys in the advice provider is probably better (less chance for later conflicts) and so we could also put the updates on the advice stack and then load them into kernel memory during the prologue.