-
Notifications
You must be signed in to change notification settings - Fork 104
feat: Single-word Array component
#2203
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mmagician
wants to merge
10
commits into
next
Choose a base branch
from
mmagician-array-via-map
base: next
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 7 commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
2796585
feat: working array
mmagician e708a51
chore: add changelog entry for Array component (#2204)
Copilot 22cf4a9
fix: set_map_item no longer returns old root
mmagician 84e9baf
chore: change Try to TryFrom for AccountComponent
mmagician 8b2e127
chore: correct the docs
mmagician 737f04b
chore: masm doc corrections
mmagician 20478b7
chore: use BeWord instead of Word
mmagician 8d0b7f3
chore: adjust masm comment about max len
mmagician 9cf1fb5
Merge branch 'next' into mmagician-array-via-map
mmagician 573a110
Merge branch 'next' into mmagician-array-via-map
mmagician File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| # The MASM code template for the Array Account Component. | ||
| # | ||
| # NOTE: This file is used as a template. The placeholder {{DATA_SLOT}} is substituted | ||
| # at construction time with the actual slot name provided by the user. | ||
| # | ||
| # This component provides an array data structure backed by a StorageMap. | ||
| # It supports "set" and "get" operations for storing and retrieving words by index. | ||
| # The array can store up to 2^64 - 2^32 elements (indices 0 to 2^64 - 2^32). | ||
mmagician marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| # | ||
| # See the `Array` Rust type's documentation for more details. | ||
|
|
||
| use miden::protocol::active_account | ||
| use miden::protocol::native_account | ||
|
|
||
| type BeWord = struct @bigendian { a: felt, b: felt, c: felt, d: felt } | ||
|
|
||
| # CONSTANTS | ||
| # ================================================================================================= | ||
|
|
||
| # The slot where the array data is stored as a map. | ||
| # Keys are [index, 0, 0, 0], values are the stored words. | ||
| # This value is substituted at construction time. | ||
| const DATA_SLOT = word("{{DATA_SLOT}}") | ||
|
|
||
| # PROCEDURES | ||
| # ================================================================================================= | ||
|
|
||
| #! Sets a word in the array at the specified index. | ||
| #! | ||
| #! Inputs: [index, VALUE, pad(11)] | ||
| #! Outputs: [OLD_VALUE, pad(12)] | ||
| #! | ||
| #! Where: | ||
| #! - index is the index at which to store the value (0 to 2^64 - 2^32). | ||
mmagician marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| #! - VALUE is the word to store at the specified index. | ||
| #! | ||
| #! Invocation: call | ||
| pub proc set(index: felt, value: BeWord) -> BeWord | ||
| # Build KEY = [index, 0, 0, 0] | ||
| push.0.0.0 movup.3 | ||
| # => [index, 0, 0, 0, VALUE, pad(11)] | ||
|
|
||
| # Push slot IDs | ||
| push.DATA_SLOT[0..2] | ||
| # => [slot_prefix, slot_suffix, KEY, VALUE, pad(11)] | ||
|
|
||
| exec.native_account::set_map_item | ||
| # => [OLD_VALUE, pad(11)] | ||
|
|
||
| # (auto-padding to 16 elements) | ||
| # => [OLD_VALUE, pad(12)] | ||
partylikeits1983 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| end | ||
|
|
||
| #! Gets a word from the array at the specified index. | ||
| #! | ||
| #! Inputs: [index, pad(15)] | ||
| #! Outputs: [VALUE, pad(12)] | ||
| #! | ||
| #! Where: | ||
| #! - index is the index of the element to retrieve (0 to 2^64 - 2^32). | ||
| #! - VALUE is the word stored at the specified index (zero if not set). | ||
| #! | ||
| #! Invocation: call | ||
| pub proc get(index: felt) -> BeWord | ||
| # Build KEY = [index, 0, 0, 0] | ||
| push.0.0.0 movup.3 | ||
| # => [index, 0, 0, 0, pad(15)] | ||
|
|
||
| # Push slot IDs | ||
| push.DATA_SLOT[0..2] | ||
| # => [slot_prefix, slot_suffix, KEY, pad(15)] | ||
|
|
||
| exec.active_account::get_map_item | ||
| # => [VALUE, pad(15)] | ||
| repeat.3 movup.4 drop end | ||
| end | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,285 @@ | ||
| use alloc::string::String; | ||
| use alloc::vec::Vec; | ||
|
|
||
| use miden_protocol::account::{AccountComponent, StorageMap, StorageSlot, StorageSlotName}; | ||
| use miden_protocol::assembly::Library; | ||
| use miden_protocol::assembly::diagnostics::NamedSource; | ||
| use miden_protocol::transaction::TransactionKernel; | ||
| use miden_protocol::{Felt, FieldElement, StorageMapError, Word}; | ||
|
|
||
| // CONSTANTS | ||
| // ================================================================================================ | ||
|
|
||
| /// The maximum number of elements the array can store. | ||
| /// | ||
| /// Since indices are represented as a single [`Felt`] element in the key `[index, 0, 0, 0]`, | ||
| /// the array can store up to 2^64 - 2^32 + 1 elements (indices 0 to 2^64 - 2^32). | ||
| /// | ||
| /// [`Felt`]: miden_protocol::Felt | ||
| pub const ARRAY_MAX_ELEMENTS: u64 = 0xffffffff00000001; | ||
|
|
||
| /// The MASM template for the Array component. | ||
| /// The placeholder `{{DATA_SLOT}}` is substituted with the actual slot name at construction time. | ||
| const ARRAY_MASM_TEMPLATE: &str = include_str!("../../asm/account_components/array.masm"); | ||
|
|
||
| // ARRAY COMPONENT | ||
| // ================================================================================================ | ||
|
|
||
| /// An [`AccountComponent`] providing an array data structure for storing words. | ||
| /// | ||
| /// This component provides a sparse array backed by a StorageMap that can store up to | ||
| /// [`ARRAY_MAX_ELEMENTS`] elements. It supports `set` and `get` operations for storing and | ||
| /// retrieving words by index. | ||
| /// | ||
| /// When linking against this component, the `miden` library (i.e. | ||
| /// [`ProtocolLib`](miden_protocol::ProtocolLib)) must be available to the assembler which is the | ||
| /// case when using [`CodeBuilder`][builder]. | ||
| /// | ||
| /// The procedures of this component are: | ||
| /// - `set`: Stores a word at the specified index. Returns the old value. | ||
| /// - `get`: Retrieves a word at the specified index. Returns zero if not set. | ||
| /// | ||
| /// This component supports all account types. | ||
| /// | ||
| /// ## Capacity | ||
| /// | ||
| /// The array can store up to [`ARRAY_MAX_ELEMENTS`] elements. | ||
| /// | ||
| /// ## Storage Layout | ||
| /// | ||
| /// - [`Self::data_slot`]: A StorageMap where key `[index, 0, 0, 0]` maps to the stored word | ||
| /// | ||
| /// ## Configurable Storage Slot | ||
| /// | ||
| /// The data slot can be configured at construction time, allowing multiple independent | ||
| /// arrays to coexist in the same account by using different slot names. | ||
| /// | ||
| /// [builder]: crate::code_builder::CodeBuilder | ||
| pub struct Array { | ||
| /// Initial elements to populate the array with, as (index, value) pairs. | ||
| initial_elements: Vec<(Felt, Word)>, | ||
| /// The storage slot name for the array data. | ||
| data_slot: StorageSlotName, | ||
| } | ||
|
|
||
| impl Array { | ||
| /// Creates a new [`Array`] component with the specified data slot and no initial elements. | ||
| /// | ||
| /// # Arguments | ||
| /// * `data_slot` - The storage slot name where the array data will be stored. | ||
| pub fn new(data_slot: StorageSlotName) -> Self { | ||
| Self { initial_elements: Vec::new(), data_slot } | ||
| } | ||
|
|
||
| /// Creates a new [`Array`] component with the given initial elements. | ||
| /// | ||
| /// Elements are provided as (index, value) pairs. Any index not specified will | ||
| /// return the zero word when accessed. | ||
| /// | ||
| /// # Arguments | ||
| /// * `data_slot` - The storage slot name where the array data will be stored. | ||
| /// * `elements` - Initial elements as (index, value) pairs. | ||
| pub fn with_elements( | ||
| data_slot: StorageSlotName, | ||
| elements: impl IntoIterator<Item = (Felt, Word)>, | ||
| ) -> Self { | ||
| Self { | ||
| initial_elements: elements.into_iter().collect(), | ||
| data_slot, | ||
| } | ||
| } | ||
|
|
||
| /// Creates a new [`Array`] component from a contiguous slice of words. | ||
| /// | ||
| /// The words are stored at indices 0, 1, 2, ... in order. | ||
| /// | ||
| /// # Arguments | ||
| /// * `data_slot` - The storage slot name where the array data will be stored. | ||
| /// * `elements` - Words to store at consecutive indices starting from 0. | ||
| pub fn from_slice(data_slot: StorageSlotName, elements: &[Word]) -> Self { | ||
| let initial_elements = elements | ||
| .iter() | ||
| .enumerate() | ||
| .map(|(i, word)| (Felt::new(i as u64), *word)) | ||
| .collect(); | ||
| Self { initial_elements, data_slot } | ||
| } | ||
|
|
||
| /// Returns a reference to the [`StorageSlotName`] where the array data is stored. | ||
| pub fn data_slot(&self) -> &StorageSlotName { | ||
| &self.data_slot | ||
| } | ||
|
|
||
| /// Generates the MASM source code for this array component by substituting | ||
| /// the data slot name into the template. | ||
| fn generate_masm_source(&self) -> String { | ||
| ARRAY_MASM_TEMPLATE.replace("{{DATA_SLOT}}", self.data_slot.as_str()) | ||
| } | ||
|
|
||
| /// Generates the compiled [`Library`] for this array component with the given component name. | ||
| /// | ||
| /// The `component_name` determines the module path used to reference the array's procedures. | ||
| /// For example, with `component_name = "myarray::data"`, the procedures would be called as | ||
| /// `myarray::data::get` and `myarray::data::set`. | ||
| /// | ||
| /// This can be used to link the array component's procedures into transaction scripts | ||
| /// or other code that needs to call the array's `get` and `set` procedures. | ||
| pub fn generate_library(&self, component_name: &str) -> Library { | ||
| let masm_source = self.generate_masm_source(); | ||
| let source = NamedSource::new(component_name, masm_source); | ||
| TransactionKernel::assembler() | ||
| .assemble_library([source]) | ||
| .expect("Array MASM template should be valid") | ||
| } | ||
| } | ||
|
|
||
| impl TryFrom<Array> for AccountComponent { | ||
| type Error = StorageMapError; | ||
|
|
||
| fn try_from(array: Array) -> Result<Self, Self::Error> { | ||
| // Generate the MASM source with the configured slot name | ||
| let masm_source = array.generate_masm_source(); | ||
|
|
||
| // Assemble the library dynamically | ||
| let source = NamedSource::new("array::component", masm_source); | ||
| let library = TransactionKernel::assembler() | ||
| .assemble_library([source]) | ||
| .expect("Array MASM template should be valid"); | ||
|
|
||
| // Data slot: StorageMap with initial elements | ||
| let map_entries = array | ||
| .initial_elements | ||
| .into_iter() | ||
| .map(|(index, value)| (Word::from([index, Felt::ZERO, Felt::ZERO, Felt::ZERO]), value)); | ||
|
|
||
| let storage_slots = | ||
| vec![StorageSlot::with_map(array.data_slot, StorageMap::with_entries(map_entries)?)]; | ||
|
|
||
| Ok(AccountComponent::new(library, storage_slots) | ||
| .expect("Array component should satisfy the requirements of a valid account component") | ||
| .with_supports_all_types()) | ||
| } | ||
| } | ||
|
|
||
| #[cfg(test)] | ||
| mod tests { | ||
| use miden_protocol::Word; | ||
| use miden_protocol::account::AccountBuilder; | ||
|
|
||
| use super::*; | ||
| use crate::account::auth::NoAuth; | ||
|
|
||
| #[test] | ||
| fn test_array_creation_empty() { | ||
| let slot = | ||
| StorageSlotName::new("myproject::myarray::data").expect("slot name should be valid"); | ||
| let array = Array::new(slot); | ||
| let component: AccountComponent = array.try_into().expect("should create component"); | ||
|
|
||
| // Verify component was created successfully with one storage slot (data) | ||
| assert_eq!(component.storage_slots().len(), 1); | ||
| } | ||
|
|
||
| #[test] | ||
| fn test_array_creation_with_elements() { | ||
| let elements = vec![ | ||
| (Felt::new(0), Word::from([1, 2, 3, 4u32])), | ||
| (Felt::new(5), Word::from([5, 6, 7, 8u32])), | ||
| (Felt::new(1000), Word::from([9, 10, 11, 12u32])), | ||
| ]; | ||
| let slot = | ||
| StorageSlotName::new("myproject::myarray::data").expect("slot name should be valid"); | ||
| let array = Array::with_elements(slot, elements); | ||
| let component: AccountComponent = array.try_into().expect("should create component"); | ||
|
|
||
| // Verify component was created successfully | ||
| assert_eq!(component.storage_slots().len(), 1); | ||
| } | ||
|
|
||
| #[test] | ||
| fn test_array_from_slice() { | ||
| let elements = vec![ | ||
| Word::from([1, 2, 3, 4u32]), | ||
| Word::from([5, 6, 7, 8u32]), | ||
| Word::from([9, 10, 11, 12u32]), | ||
| ]; | ||
| let slot = | ||
| StorageSlotName::new("myproject::myarray::data").expect("slot name should be valid"); | ||
| let array = Array::from_slice(slot, &elements); | ||
| let component: AccountComponent = array.try_into().expect("should create component"); | ||
|
|
||
| // Verify component was created successfully | ||
| assert_eq!(component.storage_slots().len(), 1); | ||
| } | ||
|
|
||
| #[test] | ||
| fn test_array_account_integration() { | ||
| let data_slot = | ||
| StorageSlotName::new("myproject::myarray::data").expect("slot name should be valid"); | ||
| let elements = vec![ | ||
| (Felt::new(0), Word::from([1, 2, 3, 4u32])), | ||
| (Felt::new(1), Word::from([5, 6, 7, 8u32])), | ||
| (Felt::new(1000), Word::from([9, 10, 11, 12u32])), | ||
| ]; | ||
| let array = Array::with_elements(data_slot.clone(), elements.clone()); | ||
| let array_component: AccountComponent = array.try_into().expect("should create component"); | ||
|
|
||
| // Build an account with the Array component | ||
| let account = AccountBuilder::new([0u8; 32]) | ||
| .with_auth_component(NoAuth) | ||
| .with_component(array_component) | ||
| .build() | ||
| .expect("account building should succeed"); | ||
|
|
||
| // Verify data elements are stored correctly | ||
| for (index, expected) in &elements { | ||
| let key = Word::from([*index, Felt::ZERO, Felt::ZERO, Felt::ZERO]); | ||
| let value = account | ||
| .storage() | ||
| .get_map_item(&data_slot, key) | ||
| .expect("data slot should contain element"); | ||
| assert_eq!(&value, expected, "element at index {} should match", index); | ||
| } | ||
| } | ||
|
|
||
| #[test] | ||
| fn test_multiple_arrays_different_slots() { | ||
| // Create two arrays with different slot names | ||
| let slot1 = | ||
| StorageSlotName::new("myproject::array1::data").expect("slot name should be valid"); | ||
| let slot2 = | ||
| StorageSlotName::new("myproject::array2::data").expect("slot name should be valid"); | ||
|
|
||
| let array1: AccountComponent = | ||
| Array::with_elements(slot1.clone(), [(Felt::new(0), Word::from([1, 1, 1, 1u32]))]) | ||
| .try_into() | ||
| .expect("should create component"); | ||
| let array2: AccountComponent = | ||
| Array::with_elements(slot2.clone(), [(Felt::new(0), Word::from([2, 2, 2, 2u32]))]) | ||
| .try_into() | ||
| .expect("should create component"); | ||
|
|
||
| // Build an account with both Array components | ||
| let account = AccountBuilder::new([0u8; 32]) | ||
| .with_auth_component(NoAuth) | ||
| .with_component(array1) | ||
| .with_component(array2) | ||
| .build() | ||
| .expect("account building should succeed"); | ||
|
|
||
| // Verify both arrays have their data stored correctly | ||
| let key = Word::from([0u32, 0, 0, 0]); | ||
|
|
||
| let value1 = account | ||
| .storage() | ||
| .get_map_item(&slot1, key) | ||
| .expect("slot1 should contain element"); | ||
| assert_eq!(value1, Word::from([1, 1, 1, 1u32])); | ||
|
|
||
| let value2 = account | ||
| .storage() | ||
| .get_map_item(&slot2, key) | ||
| .expect("slot2 should contain element"); | ||
| assert_eq!(value2, Word::from([2, 2, 2, 2u32])); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,6 @@ | ||
| use super::auth::AuthScheme; | ||
|
|
||
| pub mod array; | ||
| pub mod auth; | ||
| pub mod components; | ||
| pub mod faucets; | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Question: why make this a component rather than just a utility? That is, we could provide the storage slot name as a parameter to
get/setprocedures and it would work similar to how storage slots/maps work.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree this would be preferable. One consequence of "templating" is that the
Arrayaccount component as it is now would not be detectable in anAccountInterfacebecause it doesn't have a stable MAST root. Not sure if it's important for this component, but whenever possible, I think we should avoid "templating".