-
Notifications
You must be signed in to change notification settings - Fork 11
docs: align test DSL doc with current integration_test_lib interfaces #382
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Fibonacci747
wants to merge
1
commit into
zeko-labs:compatible
Choose a base branch
from
Fibonacci747:docs/update-test-dsl-to-current-intf
base: compatible
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -32,67 +32,77 @@ TODO: error severity ordering? seems tricky to get right, but super useful | |||||
| ##### Decorations | ||||||
|
|
||||||
| ```ocaml | ||||||
| val section : string -> node test -> node test | ||||||
| val section : string -> unit Malleable_error.t -> unit Malleable_error.t | ||||||
| val section_hard : string -> 'a Malleable_error.t -> 'a Malleable_error.t | ||||||
| ``` | ||||||
|
|
||||||
| The `section` function associates a portion of DSL code with a name. This is useful for tracking the high level of "why" the test is doing what it is doing. The test DSL is able to provide human descriptions of what it is doing most of the time, but sections allow the tests to also have high level logical groupings of "why" we are doing the specific operations being described. These names get propagated to fatal failures and errors that are accumulated. | ||||||
| The `section` function associates a portion of DSL code with a name. This is useful for tracking the high level of "why" the test is doing what it is doing. These names get propagated to failures and accumulated errors within the `Malleable_error.t` context. `section_hard` is a variant for sections that produce a value. | ||||||
|
|
||||||
| ##### Orchestra Interactions | ||||||
|
|
||||||
| ```ocaml | ||||||
| val spawn : node_config -> node_arguments -> node test | ||||||
| ``` | ||||||
| Node lifecycle and network orchestration are handled by the engine and network layers rather than DSL primitives. Tests interact with networks via `Engine.Network_manager` and `Engine.Network`, and individual nodes via `Engine.Network.Node.start/stop`. There are no `spawn`/`destroy` functions in the current DSL. | ||||||
|
|
||||||
| The `spawn` function communicates with the orchestrator to allocate and start a new node. This function takes a runtime config for the node and CLI arguments to execute the node with (not necessarily fed in via the CLI though, in the case of local tests). This function does not wait for the node to join the network; execution of the test will continue as soon as the node is started, regardless of errors. | ||||||
|
|
||||||
| ```ocaml | ||||||
| val destroy : node -> unit test | ||||||
| ``` | ||||||
| ##### Concurrent Tasks | ||||||
|
|
||||||
| The `destroy` function communicates with the orchestrator to destroy nodes that have been created. If the node attempting to be destroyed is already dead due to a crash, the test execution will continue, but the crash will accumulated as an error. If the node attempting to be destroyed has been previously destroyed via this function, an error is accumulated. Any nodes that were spawned and are not destroyed by the end of a test will cause errors to be accumulated (we want all spawned nodes to be asserted to have not crashed or to be expected to crash by the end of the test). | ||||||
| Concurrent task primitives (`task`, `stop`) are not part of the current DSL surface. | ||||||
|
|
||||||
| ##### Concurrent Tasks | ||||||
| ##### Synchronization | ||||||
|
|
||||||
| ```ocaml | ||||||
| type task | ||||||
| val task : (unit test -> unit test) -> task test | ||||||
| val stop : task -> unit test | ||||||
| val wait_for : t -> Wait_condition.t -> unit Malleable_error.t | ||||||
| ``` | ||||||
|
|
||||||
| ##### Synchronization | ||||||
| Waiting is expressed via `Wait_condition` constructors and optional timeouts: | ||||||
|
|
||||||
| ```ocaml | ||||||
| val wait_for : | ||||||
| (* exclusive arguments *) | ||||||
| ?status:node_status | ||||||
| -> ?blocks:int | ||||||
| -> ?epoch_reached:int | ||||||
| -> ... | ||||||
| (* non-exclusive arguments *) | ||||||
| -> ?timeout: | ||||||
| [ `Slots of int | ||||||
| | `Epochs of int | ||||||
| | `Milliseconds of int64 ] | ||||||
| -> unit | ||||||
| -> unit test | ||||||
| module Wait_condition : sig | ||||||
| type t | ||||||
| val blocks_to_be_produced : int -> t | ||||||
| val block_height_growth : height_growth:int -> t | ||||||
| val nodes_to_synchronize : Engine.Network.Node.t list -> t | ||||||
| val node_to_initialize : Engine.Network.Node.t -> t | ||||||
| val nodes_to_initialize : Engine.Network.Node.t list -> t | ||||||
| val signed_command_to_be_included_in_frontier : | ||||||
| txn_hash:Mina_transaction.Transaction_hash.t -> | ||||||
| node_included_in:[ `Any_node | `Node of Engine.Network.Node.t ] -> t | ||||||
| val ledger_proofs_emitted_since_genesis : | ||||||
| test_config:Test_config.t -> num_proofs:int -> t | ||||||
| val zkapp_to_be_included_in_frontier : | ||||||
| has_failures:bool -> zkapp_command:Mina_base.Zkapp_command.t -> t | ||||||
| val persisted_frontier_loaded : Engine.Network.Node.t -> t | ||||||
| val transition_frontier_loaded_from_persistence : | ||||||
| fresh_data:bool -> sync_needed:bool -> t | ||||||
| val with_timeouts : | ||||||
| ?soft_timeout:Network_time_span.t -> | ||||||
| ?hard_timeout:Network_time_span.t -> t -> t | ||||||
| end | ||||||
| ``` | ||||||
|
|
||||||
| ##### Analysis | ||||||
|
|
||||||
| ```ocaml | ||||||
| val assert : ... | ||||||
| val watch_log_errors : | ||||||
| logger:Logger.t -> | ||||||
| event_router:Event_router.t -> | ||||||
| on_fatal_error:(Logger.Message.t -> unit) -> | ||||||
| log_error_accumulator | ||||||
|
|
||||||
| val lift_accumulated_log_errors : | ||||||
| ?exit_code:int -> log_error_accumulator -> | ||||||
| Test_error.remote_error Test_error.Set.t | ||||||
| ``` | ||||||
|
|
||||||
| ```ocaml | ||||||
| type collection = Logger.Log.t -> Metric.t list | ||||||
| val collect : node -> collection -> unit test | ||||||
| ``` | ||||||
| Assertions and metric collection are handled via log/event processing (`Log_engine`, `Event_router`, `Network_state`). There is no `assert` or `collect` DSL primitive in the current surface. | ||||||
|
|
||||||
| ##### Node Interactions: TODO | ||||||
|
|
||||||
| Direct node interactions are performed via `Engine.Network` and `Engine.Network.Node` APIs rather than dedicated DSL primitives. | ||||||
|
|
||||||
| <!-- DSL DESIGN SECTION --> | ||||||
| #### Pseudo-code Examples | ||||||
|
|
||||||
| Note: The following examples reflect an earlier design proposal and do not match the current DSL API. They should be translated to use `Engine.Network_manager`/`Engine.Network` for orchestration and `Dsl.wait_for` with `Wait_condition` for synchronization. | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This note is very important for preventing users from trying to use outdated examples. To make it more prominent and ensure it's not missed, consider using a blockquote and bolding the 'Note' label.
Suggested change
|
||||||
|
|
||||||
| ##### Basic Bootstrap Test | ||||||
|
|
||||||
| ```ocaml | ||||||
|
|
||||||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The type
tin this function signature is not defined in this document. To improve clarity for readers, it would be helpful to add a brief explanation of whattrepresents (e.g., the test runner state). You could add a comment in the code block or a sentence in the description.