From b689ddf4dcf3ab12c818f33860817648a7721865 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Thu, 30 Apr 2026 14:06:17 +0800 Subject: [PATCH 01/18] Rename platform tests directory --- desk/tests/ph/{for => platform}/lure.hoon | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename desk/tests/ph/{for => platform}/lure.hoon (100%) diff --git a/desk/tests/ph/for/lure.hoon b/desk/tests/ph/platform/lure.hoon similarity index 100% rename from desk/tests/ph/for/lure.hoon rename to desk/tests/ph/platform/lure.hoon From 51a4a7a061a083f22daaa4cfb3a8805ce5631873 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Thu, 30 Apr 2026 14:44:21 +0800 Subject: [PATCH 02/18] agents: add basic backend instructions --- AGENTS.md | 45 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000..ffa6a1b161 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,45 @@ +# Tlon Messenger backend +The backend of the Tlon Messenger app is hosted on the Urbit platform. + +All the backend code is located in the desk/ directory, which +is deployed to an urbit ship. + +## Development +Interfacing with an Urbit ship for the purpose of development +can be done in a variety of ways. Currently, the two primary options +are: +1. Interfacing using clurd tooling +2. Interfacing through a running mcp server hosted on an Urbit ship + +Both of these require developer setup, and are not part of the +repository. + +Interfacing through clurd requires access to the clurd +directory, which contains tooling and useful documents for working +with a running Urbit development ship. + +Interfacing through an MCP server requires an Urbit development ship +with an enabled MCP + +When asked to perform backend development work, check with the +user for the preferred interface. + +## Deploying the desk +To deploy desk/ changes to a ship, simply use rsync to copy +the desk/ directory to a corresponding mounted desk in the ship's pier. +However, remember **not to delete** existing files in the mounted desk. +The deployed desk contains necessary files which are not preserved in +this repository. +The user should point to the correct location where the pier is located. + +## Compiling the desk +Once the changes in the repository have been copied over to the ship, it +can be compiled. +Issue `|commit groups` with long-enough timeout. For small changes, a timeout +of under 1 minute can be sufficient. For changes involving sur +files or libraries used by many agents, a timeout of 2 or 3 minutes might be necessary. + + + + + From 74d98ad6e1255b6441f0baa409c321f03864a005 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Thu, 30 Apr 2026 15:14:34 +0800 Subject: [PATCH 03/18] agents: add backend tests instructions --- AGENTS.md | 61 +++++++++++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 55 insertions(+), 6 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index ffa6a1b161..53c0cd0df6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -12,14 +12,14 @@ are: 2. Interfacing through a running mcp server hosted on an Urbit ship Both of these require developer setup, and are not part of the -repository. +repository. Interfacing through clurd requires access to the clurd directory, which contains tooling and useful documents for working with a running Urbit development ship. -Interfacing through an MCP server requires an Urbit development ship -with an enabled MCP +Interfacing through an MCP server requires an Urbit development ship +with an enabled MCP When asked to perform backend development work, check with the user for the preferred interface. @@ -35,11 +35,60 @@ The user should point to the correct location where the pier is located. ## Compiling the desk Once the changes in the repository have been copied over to the ship, it can be compiled. -Issue `|commit groups` with long-enough timeout. For small changes, a timeout -of under 1 minute can be sufficient. For changes involving sur -files or libraries used by many agents, a timeout of 2 or 3 minutes might be necessary. +Issue `|commit groups` with long-enough timeout. For small changes, a timeout +of under 1 minute can be sufficient. For changes involving sur +files or libraries used by many agents, a timeout of 2 or 3 minutes might be necessary. +## Rerunning commands with clurd +Clurd tooling is not able to capture an output of a command +past the specified timeout. When we need to rerun a command, +especially if we suspect output has been missed, we need to: +1. Interrupt the previous command, which could still be runnig +2. Repeat the command with a longer (or shorter) timeout +## Backend tests + +There are two kinds of backend tests in groups. The first kind uses the +`/lib/test-agent.hoon` library, which provides a monadic framework for +implementing gall agent tests. It works by simulating rudimentary gall +functionality, which allows testing of the agent core (which is a pure function of +agent state) under the variety of circumstances. + +test-agent tests are typically located in `/tests` directory, under the +corresponding desk entry. For example, tests for `/app/groups.hoon` +agent would be located at `/tests/app/groups.hoon`. + +The second kind of tests uses aqua ship virtualization technology. +Using aqua, a virtual fleet of ships can be run directly on the +development ship with little resource cost. While these ships are not +fully-featured and do not support every urbit runtime event, they +nonetheless allow testing of gall agents running on a virtualized ship. + +Aqua tests are located in `/tests/ph` directory. + +These two kinds of tests use different arm prefix to avoid having a +runner execute a wrong kind of test. test-agent test arms use a `test` +prefix, while aqua tests use `ph-test` prefix. + +## Running test-agent tests +The test runner is the `-test` thread, which will scan the directory +to find tests, and execute them one-by-one, reporting failures. +Use `-test /=groups=/tests` to run all tests in the directory. + +To target a specific file and test arm, use the `-test /=groups=/tests/path/arm` +syntax. + +## Running aqua tests +Aqua tests use a similar setup to test-agent tests. +There is a dedicated test runner `-ph-test`, which is located in the +groups desk. (To invoke the thread from a particular desk, we use the +`-desk!thread` syntax.) + +Use `-groups!ph-test /=groups=/tests/ph` to run all aqua tests in the +`/tests/ph` directory. + +To target a specific file and test arm, use the `-groups!ph-test +/=groups=/tests/ph/path/arm` syntax. From 543deaf2c478ee5e6cd3edfbd23910eba1f7798f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Tue, 5 May 2026 15:50:33 +0800 Subject: [PATCH 04/18] Aqua tests: agent instrumentation --- AGENTS.md | 82 ++++----------------- desk/lib/verb.hoon | 5 +- desk/ted/ph/fleet.hoon | 87 +++++++++++++++++++++++ desk/ted/ph/test.hoon | 96 ++++--------------------- desk/tests/ph/app/groups.hoon | 1 + desk/tests/ph/platform/lure.hoon | 14 ++-- docs/aqua/introduction.md | 118 +++++++++++++++++++++++++++++++ 7 files changed, 241 insertions(+), 162 deletions(-) create mode 100644 desk/ted/ph/fleet.hoon create mode 100644 docs/aqua/introduction.md diff --git a/AGENTS.md b/AGENTS.md index 53c0cd0df6..029a43acf1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,3 +1,5 @@ +# AGENTS.md + # Tlon Messenger backend The backend of the Tlon Messenger app is hosted on the Urbit platform. @@ -5,47 +7,11 @@ All the backend code is located in the desk/ directory, which is deployed to an urbit ship. ## Development -Interfacing with an Urbit ship for the purpose of development -can be done in a variety of ways. Currently, the two primary options -are: -1. Interfacing using clurd tooling -2. Interfacing through a running mcp server hosted on an Urbit ship - -Both of these require developer setup, and are not part of the -repository. - -Interfacing through clurd requires access to the clurd -directory, which contains tooling and useful documents for working -with a running Urbit development ship. - -Interfacing through an MCP server requires an Urbit development ship -with an enabled MCP - -When asked to perform backend development work, check with the -user for the preferred interface. - -## Deploying the desk -To deploy desk/ changes to a ship, simply use rsync to copy -the desk/ directory to a corresponding mounted desk in the ship's pier. -However, remember **not to delete** existing files in the mounted desk. -The deployed desk contains necessary files which are not preserved in -this repository. -The user should point to the correct location where the pier is located. - -## Compiling the desk -Once the changes in the repository have been copied over to the ship, it -can be compiled. -Issue `|commit groups` with long-enough timeout. For small changes, a timeout -of under 1 minute can be sufficient. For changes involving sur -files or libraries used by many agents, a timeout of 2 or 3 minutes might be necessary. - - -## Rerunning commands with clurd -Clurd tooling is not able to capture an output of a command -past the specified timeout. When we need to rerun a command, -especially if we suspect output has been missed, we need to: -1. Interrupt the previous command, which could still be runnig -2. Repeat the command with a longer (or shorter) timeout +Interface with a running urbit ship through a tmux session +running an urbit ship. Do not switch to that session, but interface +with it using tmux input and capture commands. +A typical command to verify connection is working is `%`, which will +display current identity, desk and time. ## Backend tests @@ -55,40 +21,18 @@ implementing gall agent tests. It works by simulating rudimentary gall functionality, which allows testing of the agent core (which is a pure function of agent state) under the variety of circumstances. -test-agent tests are typically located in `/tests` directory, under the +test-agent tests are located in `/tests` directory, under the corresponding desk entry. For example, tests for `/app/groups.hoon` agent would be located at `/tests/app/groups.hoon`. -The second kind of tests uses aqua ship virtualization technology. -Using aqua, a virtual fleet of ships can be run directly on the -development ship with little resource cost. While these ships are not +The second kind of tests uses aqua-based ship virtualization. +Using aqua, a virtual fleet of ships can be run directly on a +ship with little resource cost. While these ships are not fully-featured and do not support every urbit runtime event, they -nonetheless allow testing of gall agents running on a virtualized ship. +nonetheless allow testing of gall agents running on virtualized ship. Aqua tests are located in `/tests/ph` directory. -These two kinds of tests use different arm prefix to avoid having a -runner execute a wrong kind of test. test-agent test arms use a `test` -prefix, while aqua tests use `ph-test` prefix. - -## Running test-agent tests -The test runner is the `-test` thread, which will scan the directory -to find tests, and execute them one-by-one, reporting failures. -Use `-test /=groups=/tests` to run all tests in the directory. - -To target a specific file and test arm, use the `-test /=groups=/tests/path/arm` -syntax. - -## Running aqua tests -Aqua tests use a similar setup to test-agent tests. -There is a dedicated test runner `-ph-test`, which is located in the -groups desk. (To invoke the thread from a particular desk, we use the -`-desk!thread` syntax.) - -Use `-groups!ph-test /=groups=/tests/ph` to run all aqua tests in the -`/tests/ph` directory. - -To target a specific file and test arm, use the `-groups!ph-test -/=groups=/tests/ph/path/arm` syntax. +For details on how to work with aqua tests see documentation in `/docs/aqua` diff --git a/desk/lib/verb.hoon b/desk/lib/verb.hoon index 620e9abf8e..1d0e046167 100644 --- a/desk/lib/verb.hoon +++ b/desk/lib/verb.hoon @@ -152,9 +152,10 @@ =/ =echo:logs ?- -.event %fail - [leaf+"fail {}" trace.event] + [leaf+"[] fail {}" trace.event] :: - %tell echo.event + %tell + [leaf+"[]" echo.event] == %- %- %*(. slog pri val) echo diff --git a/desk/ted/ph/fleet.hoon b/desk/ted/ph/fleet.hoon new file mode 100644 index 0000000000..9361f45163 --- /dev/null +++ b/desk/ted/ph/fleet.hoon @@ -0,0 +1,87 @@ +:: prepare aqua fleet snapshot +:: +:: fleet=(list ship) +:: +/- spider, aquarium +/+ *strandio, ph-io, ph-test +=, strand=strand:spider +:: +sync-desk: sync aqua ship desk to host desk +:: +|% +++ sync-desk + |= [her=ship desk=@tas] + =/ m (strand ,~) + ^- form:m + ;< =bowl:strand bind:m get-bowl + =/ sab=path + /(scot %p our.bowl)/[desk]/(scot %da now.bowl) + =| =path + =| raw-files=(list [^path page:clay]) + =. raw-files + |- + =* loop $ + =+ .^(=arch %cy (weld sab path)) + =. raw-files + %+ roll ~(tap in ~(key by dir.arch)) + |= [dir=@ta =_raw-files] + (welp loop(path (snoc path dir)) raw-files) + ?~ fil.arch raw-files + =+ .^(=page:clay %cs (weld sab /blob/(scot %uv u.fil.arch))) + :_ raw-files + [path page] + =/ files + %+ turn raw-files + |= [=^path =page:clay] + =+ .^(=dais:clay %cb (snoc sab p.page)) + =+ .^(=tube:clay %cc (weld sab /[p.page]/mime)) + =+ !<(=mime (tube (vale:dais q.page))) + [path ~ mime] + =/ =beam [[her desk ud+1] /] + ;< ~ bind:m (send-events:ph-io [%event her /c/mount/0v1abc [%mont desk beam]]~) + =/ =task:clay + [%into desk & files] + ;< ~ bind:m (send-events:ph-io [%event her /c/sync/0v1abc task]~) + ;< ~ bind:m (sleep ~s0) + (pure:m ~) +-- +^- thread:spider +|= args=vase +=/ m (strand ,vase) +^- form:m +=+ !<(args=(unit [fleet=(list ship) sync=?]) args) +=/ [fleet=(list ship) sync=?] + ?~ args ~|(%no-args-found !!) + [fleet sync]:u.args +=. fleet + ^- (list ship) + :* ~loshut-lonreg :: bait provider + ~rivfur-livmet :: notify provider + ~dem :: bait provider galaxy + ~fen :: notify provider galaxy + fleet + == +;< =bowl:spider bind:m get-bowl +~> %slog.1^(crip "Booting fleet {}") +;< vane-tids=(map term tid:spider) bind:m start-simple:ph-io +;< ~ bind:m + =/ n (strand ,~) + |- + ?~ fleet (pure:n ~) + ;< ~ bind:n (init-ship:ph-io i.fleet &) + $(fleet t.fleet) +:: +;< ~ bind:m + =/ n (strand ,~) + ?. sync (pure:n ~) + ~> %slog.1^(crip "Syncing %groups desk to ships...") + |- + ?~ fleet (pure:n ~) + ;< ~ bind:n (sync-desk i.fleet %groups) + $(fleet t.fleet) +;< =bowl:spider bind:m get-bowl +=+ snap-id=(end 3^4 (sham eny.bowl)) +=/ snap=@t + (cat 3 'aqua-tests-' (scot %uv snap-id)) +~> %slog.1^(crip "Taking snapshot...") +;< ~ bind:m (send-events:ph-io [%snap-ships snap fleet]~) +(pure:m !>(snap)) diff --git a/desk/ted/ph/test.hoon b/desk/ted/ph/test.hoon index 3794f74153..2650490e12 100644 --- a/desk/ted/ph/test.hoon +++ b/desk/ted/ph/test.hoon @@ -1,4 +1,7 @@ -:: ph test runner +:: run aqua tests +:: +:: pax=(unit path) optional test path +:: snap=@t aqua fleet snapshot :: /- spider, aquarium /+ *strandio, ph-io, ph-test @@ -76,8 +79,7 @@ ^- test [(weld path /[name.test-arm]) strand.test-arm] :: +await-test-thread: run and return result of an aqua test thread -::TODO implement timeout, and if it is exceeded kill the thread and -:: report timeout failure. +:: ++ await-test-thread |= [file=path =test-strand] =/ m (strand ,thread-result) @@ -99,57 +101,22 @@ == ;< ~ bind:m (watch-our /awaiting/[tid] %spider /thread-result/[tid]) ;< ~ bind:m (poke-our %spider %spider-inline !>(inline-args)) + ::TODO implement timeout ;< =cage bind:m (take-fact /awaiting/[tid]) ;< ~ bind:m (take-kick /awaiting/[tid]) ?+ p.cage ~|([%strange-thread-result p.cage file tid] !!) %thread-done (pure:m %& q.cage) %thread-fail (pure:m %| !<([term tang] q.cage)) == -:: +sync-desk: sync aqua ship desk to host -:: host ship -> sync desk to virtual ship -:: %groups -> aqua %groups -++ sync-desk - |= [her=ship desk=@tas] - =/ m (strand ,~) - ^- form:m - ;< =bowl:strand bind:m get-bowl - =/ sab=path - /(scot %p our.bowl)/[desk]/(scot %da now.bowl) - =| =path - =| raw-files=(list [^path page:clay]) - =. raw-files - |- - =* loop $ - =+ .^(=arch %cy (weld sab path)) - =. raw-files - %+ roll ~(tap in ~(key by dir.arch)) - |= [dir=@ta =_raw-files] - (welp loop(path (snoc path dir)) raw-files) - ?~ fil.arch raw-files - =+ .^(=page:clay %cs (weld sab /blob/(scot %uv u.fil.arch))) - :_ raw-files - [path page] - =/ files - %+ turn raw-files - |= [=^path =page:clay] - =+ .^(=dais:clay %cb (snoc sab p.page)) - =+ .^(=tube:clay %cc (weld sab /[p.page]/mime)) - =+ !<(=mime (tube (vale:dais q.page))) - [path ~ mime] - =/ =beam [[her desk ud+1] /] - ;< ~ bind:m (send-events:ph-io [%event her /c/mount/0v1abc [%mont desk beam]]~) - =/ =task:clay - [%into desk & files] - ;< ~ bind:m (send-events:ph-io [%event her /c/sync/0v1abc task]~) - ;< ~ bind:m (sleep ~s0) - (pure:m ~) -- -:: ^- thread:spider |= args=vase =/ m (strand ,vase) ^- form:m -=+ !<(pax=(unit path) args) +=+ !<(args=(unit [pax=(unit path) snap=@t]) args) +=/ [pax=(unit path) snap=@] + ?~ args ~|(%no-args-found !!) + [pax.u.args snap.u.args] ;< =bowl:spider bind:m get-bowl =/ [byk=beak pax=path] ?~ pax @@ -162,9 +129,10 @@ [ship desk da+date] =? pax ?=(~ pax) /tests -=/ =arch .^(arch %cy (weld /(scot %p p.byk)/[q.byk]/(scot %da +.r.byk) pax)) :: if the path does not point into a directory, assume last component :: is a pattern. +:: +=/ =arch .^(arch %cy (weld /(scot %p p.byk)/[q.byk]/(scot %da +.r.byk) pax)) =/ [arm-pat=(unit @ta) pax=path] ?: ?=(^ dir.arch) [~ pax] @@ -195,44 +163,6 @@ ~> %slog.2^leaf+"No suitable aqua tests found" (pure:m !>(~)) ~> %slog.1^leaf+"{} test {?:((gth num 1) "threads" "thread")} built" -~> %slog.1^'Booting ships...' -;< vane-tids=(map term tid:spider) bind:m start-simple:ph-io -:: test ships -:: -;< ~ bind:m (init-ship:ph-io ~zod &) -;< ~ bind:m (init-ship:ph-io ~bud &) -;< ~ bind:m (init-ship:ph-io ~nec &) -:: provider ships -:: -:: bait provider -;< ~ bind:m (init-ship:ph-io ~fen &) -:: notify provider -;< ~ bind:m (init-ship:ph-io ~dem &) -:: -~> %slog.1^(crip "Syncing {} desk to ships...") -;< ~ bind:m (sync-desk ~zod %groups) -;< ~ bind:m (sync-desk ~bud %groups) -;< ~ bind:m (sync-desk ~nec %groups) -;< ~ bind:m (sync-desk ~fen %groups) -;< ~ bind:m (sync-desk ~dem %groups) -:: setup bait provider -:: -~> %slog.1^(crip "Setting ~fen as lure provider...") -;< ~ bind:m ph-test-init:ph-test -;< ~ bind:m (poke-app:ph-test [~zod %reel] reel-command+[%set-ship ~fen]) -;< ~ bind:m (poke-app:ph-test [~bud %reel] reel-command+[%set-ship ~fen]) -;< ~ bind:m (poke-app:ph-test [~nec %reel] reel-command+[%set-ship ~fen]) -;< ~ bind:m (poke-app:ph-test [~fen %reel] reel-command+[%set-ship ~fen]) -;< ~ bind:m (poke-app:ph-test [~dem %reel] reel-command+[%set-ship ~fen]) -;< ~ bind:m (sleep ~s0) -;< ~ bind:m ph-test-shut:ph-test -;< ~ bind:m (end:ph-io vane-tids) -:: TODO notify agent does not support swapping a provider -;< =bowl:spider bind:m get-bowl -=+ snap-id=(end 3^4 (sham eny.bowl)) -=+ snap=(cat 3 'aqua-tests-' (scot %uv snap-id)) -~> %slog.1^(crip "Taking snapshot...") -;< ~ bind:m (send-events:ph-io [%snap-ships snap ~[~zod ~bud ~nec ~fen ~dem]]~) ~> %slog.1^'Running tests...' =/ n (strand (list (pair path thread-result))) ;< results=(list (pair path thread-result)) bind:m @@ -256,8 +186,6 @@ [(div diff s) (div (mod diff s) ms)] ~> %slog.1^leaf+"{(trip name)} took {}.{((d-co:co 3) took-ms)}s" $(tests t.tests, results [[path.test thread-result] results]) -::TODO fix aqua to only clear a particular snap -;< ~ bind:m (poke-our %aqua noun+!>([%clear-snap snap])) =+ ok=& |- ?~ results (pure:m !>(ok)) diff --git a/desk/tests/ph/app/groups.hoon b/desk/tests/ph/app/groups.hoon index 7a2113060c..0861c639a0 100644 --- a/desk/tests/ph/app/groups.hoon +++ b/desk/tests/ph/app/groups.hoon @@ -35,6 +35,7 @@ :: that the subscription lifecycle follows through %watch, and then %done. :: finally, the group creation response is received. :: +:: ++ ph-test-group-join =/ m (strand ,~) ^- form:m diff --git a/desk/tests/ph/platform/lure.hoon b/desk/tests/ph/platform/lure.hoon index 3b091629fb..334e1d5b4a 100644 --- a/desk/tests/ph/platform/lure.hoon +++ b/desk/tests/ph/platform/lure.hoon @@ -11,7 +11,7 @@ :: ~bud is the invitee. he receives the invite :: and onboards to the network, joining the group. :: -:: ~fen is the bait provider +:: ~loshut-lonreg is the bait provider :: |% ++ my-test-flag ~zod^%test-group @@ -194,16 +194,16 @@ ipv4+.127.0.0.1 request == - :: ~fen the bait provider receives the onboarding request + :: ~loshut-lonreg the bait provider receives the onboarding request :: =/ =aqua-event - [%event ~fen /e/aqua/eyre/request task] + [%event ~loshut-lonreg /e/aqua/eyre/request task] (send-events ~[aqua-event]) ++ ph-test-lure-group =/ m (strand ,~) ^- form:m :: - ;< ~ bind:m (poke-app [~fen %bait] verb+[%volume %info]) + ;< ~ bind:m (poke-app [~loshut-lonreg %bait] verb+[%volume %info]) :: host a group on ~zod and enable lure links :: ;< ~ bind:m create-test-group @@ -211,7 +211,7 @@ ;< lure-invite=@t bind:m (generate-lure-invite lure-group-metadata) ;< ~ bind:m (watch-app /~bud/groups/v1/foreigns [~bud %groups] /v1/foreigns) ;< ~ bind:m (watch-app /~bud/chat/v4 [~bud %chat] /v4) - ;< cookie=(unit @t) bind:m (eyre-authenticate ~fen) + ;< cookie=(unit @t) bind:m (eyre-authenticate ~loshut-lonreg) ?> ?=(^ cookie) :: ~bud onboards from hosting through the lure invite. :: @@ -240,10 +240,10 @@ =/ m (strand ,~) ^- form:m :: - ;< ~ bind:m (poke-app [~fen %bait] verb+[%volume %info]) + ;< ~ bind:m (poke-app [~loshut-lonreg %bait] verb+[%volume %info]) ;< lure-invite=@t bind:m (generate-lure-invite lure-personal-metadata) ;< ~ bind:m (watch-app /~bud/chat/v4 [~bud %chat] /v4) - ;< cookie=(unit @t) bind:m (eyre-authenticate ~fen) + ;< cookie=(unit @t) bind:m (eyre-authenticate ~loshut-lonreg) ?> ?=(^ cookie) :: ~bud onboards from hosting through the lure invite. :: diff --git a/docs/aqua/introduction.md b/docs/aqua/introduction.md new file mode 100644 index 0000000000..c58eeb375d --- /dev/null +++ b/docs/aqua/introduction.md @@ -0,0 +1,118 @@ +# Aqua tests + +## Preparing aqua pill + +Before aqua can run virtual ships, the aqua pill must be prepared.This +pill is used to boot each virtual ship. Once aqua has been initialized with the pill +there is no need to reinitialize it unless: +1. Aqua state has been reset. +2. We want the pill to incorporate some changes, such as an updated desk or additional files. + +To initialize aqua with a fresh pill, we poke aqua with an assembled pill. This +can be done in one go by using the brass pill generator. +``` +> :aqua &pill +pill/brass %base desk1 desk2 +``` + +The brass pill takes a list of desks as an argument. The `%base` desk is required. + +Brass pill generation takes about 5 minutes or longer to complete. + +## Preparing aqua snapshot + +Each aqua test requires a reproducible environment consisting of a +fleet of virtual ships. It would be too expensive to boot the fleet +each time a test runs. Fortunately, aqua provides a way to snapshot a booted +set of ships. A snapshot can then be restored each time a test runs, providing +a fast way to initialize a test environment. + +To prepare a snapshot, decide upon a number of virtual ships required. +Since aqua tests use a hardcoded set of ships, by convention we use galaxies +in enumeration order as test ships. Thus, for a test involving 4 ships we would have +`~zod`, `~nec`, `~bud` and `~wes`. If more ships are required, discover the n-th galaxy id by using +``` +> `@p`n +``` + +Prepare the snapshot by running +``` +> -desk!ph-fleet fleet sync +``` +where `fleet` is the list of ships and `sync` is the desk sync flag. +Default the sync flag to `|`, unless there has been changes to the desk. +Currently, syncing desk to virtual ships takes quite some time. + +The thread will prepare an aqua snapshot, and return the snapshot id. +You can save it to a dojo variable with +``` +> =snap-id -desk!ph-fleet fleet sync +``` + +In addition to galaxy test ships, a test may require a number of _provider_ +ships to be running, to allow components of the system to properly integrate +with remote services. + +In the groups desk, we need `%bait` and `%notify` providers, running on `~loshut-lonreg` +and `~rivfur-livmet` respectively. In addition, virtual ames/mesa networking requires +the parent galaxies to be present. In this case, we must add `~fen` and `~dem` to the +fleet. This is handled by the `-ph-fleet` thread + +## Running tests + +To run aqua tests use the `-ph-test` test runner supplied with the desk. +``` +> -desk!ph-test `path snap +``` +The `path` is a directory containing aqua tests. Supply `~` to run from the default directory. +It will be scanned recursively, and any tests found will be build and scheduled to be run. + +`snap` is the aqua snapshot on which tests are going to run. + +The standard location for aqua tests is in the desk's `/tests/ph` directory. +The path can be extended to target a particular component. For instance, to +run only aqua tests for the group agent, use `/tests/ph/app/groups`. +You can also target a particular test by appending the test name, such as +`/tests/ph/app/groups/ph-test-group-join` to run only the groups group join test. + +The current virtual ames driver is inefficient. This make a single test run take +about `~20s`, or more, depending on the number of networked interactions between +virtual ships. + +### Understanding test output +While tests run, information will be displayed from a variety of sources. +The test runner will display test result and time taken after run of each test case. +The aqua virtual runtime is currently quite verbose, and will display a lot of noise, +primarily about unhandled effects. + +Running virtual ships will also display logging messages. +These messages are associated with a priority: +| prefix | priority | +|-----------|----------| +| no prefix | `%dbug` debug priority | +| `>` | `%info` information priority | +| `>>` | `%warn` warning priority | +| `>>>` | `%crit` critical priority | +Each logging message is prefix by the virtual source ship and agent names. + +An example error message +``` +>>> [~zod/groups] Critical failure in +se-u-groups +``` +means the `%groups` agent running on `~zod` is reporting an error generated +in `+se-u-groups` arm. + +Since the logging messages are not associated with a particular location +in the source code, to find the originating location we can either start with +the location indicated in the message. In the above example, that would +be the `+se-u-groups` arm somewhere in the agent's source code or its libraries. +If that does not yield results, searching for constant parts of the messsage in relevant files usually. + +## Listing tests +To find available aqua tests, use `-ph-test-ls path` command. +It will compile all files present at the path and assemble a list +of test arms that would be run by the test runner. + +## Cancelling a test run +The test runner is a thread, so in order to stop an ongoing test run you must +cancel the runner thread. In dojo, this is simply done by pressing a backspace. + From a12e6610b685c74881d30852936033e5f58f88fc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Tue, 5 May 2026 17:49:16 +0800 Subject: [PATCH 05/18] Aqua tests: add a first-ever agent-generated aqua test --- desk/lib/verb.hoon | 4 +- desk/tests/ph/app/groups.hoon | 75 +++++++++++++++++++++++++++++ docs/aqua/development.md | 88 +++++++++++++++++++++++++++++++++++ docs/aqua/introduction.md | 6 +++ 4 files changed, 171 insertions(+), 2 deletions(-) create mode 100644 docs/aqua/development.md diff --git a/desk/lib/verb.hoon b/desk/lib/verb.hoon index 1d0e046167..045fd13ed0 100644 --- a/desk/lib/verb.hoon +++ b/desk/lib/verb.hoon @@ -152,10 +152,10 @@ =/ =echo:logs ?- -.event %fail - [leaf+"[] fail {}" trace.event] + [leaf+"[/] fail {}" trace.event] :: %tell - [leaf+"[]" echo.event] + [leaf+"[/]" echo.event] == %- %- %*(. slog pri val) echo diff --git a/desk/tests/ph/app/groups.hoon b/desk/tests/ph/app/groups.hoon index 0861c639a0..bc2e8a1e04 100644 --- a/desk/tests/ph/app/groups.hoon +++ b/desk/tests/ph/app/groups.hoon @@ -27,6 +27,31 @@ ;< ~ bind:m (ex-equal !>(flag.rep) !>(flag)) ;< ~ bind:m (ex-equal !>(`@tas`-.r-group.rep) !>(tag)) (pure:m ~) +:: +ex-r-groups-meta: expect group metadata response fact +:: +++ ex-r-groups-meta + |= [=ship =flag:gv meta=data:meta:g] + =/ m (strand ,~) + ^- form:m + ;< kag=cage bind:m (wait-for-app-fact /(scot %p ship)/groups/v1/groups [ship %groups]) + ?> =(%group-response-1 p.kag) + =+ !<(rep=r-groups:v10:gv q.kag) + ;< ~ bind:m (ex-equal !>(flag.rep) !>(flag)) + ?> ?=(%meta -.r-group.rep) + ;< ~ bind:m (ex-equal !>(meta.r-group.rep) !>(meta)) + (pure:m ~) +:: +scry-test-group: scry group state from a virtual ship +:: +++ scry-test-group + |= =ship + =/ m (strand group:g) + ^- form:m + ;< =bowl:strand bind:m get-bowl + =/ aqua-pax + /gx/(scot %p ship)/groups/(scot %da now.bowl)/v2/groups/~zod/my-test-group/group-2 + ;< group=(unit group:v9:gv) bind:m + (scry-aqua (unit group:v9:gv) ship aqua-pax) + (pure:m `group:g`(need group)) :: +ph-test-group-join: test group joins :: :: scenario @@ -65,5 +90,55 @@ ;< ~ bind:m (ex-r-groups-fact ~bud ~zod^%my-test-group %create) ;< =bowl:strand bind:m get-bowl (pure:m ~) +:: +ph-test-admin-group-meta: test admin group metadata update +:: +:: scenario +:: +:: ~zod hosts a group and invites ~bud as an admin. ~bud joins the group. +:: after joining, ~bud issues a metadata update from their own ship. we +:: verify that the host applies the metadata update, and that the same +:: metadata update propagates back to ~bud. +:: +++ ph-test-admin-group-meta + =/ m (strand ,~) + ^- form:m + ;< ~ bind:m (watch-app /~zod/groups/v1/groups [~zod %groups] /v1/groups) + ;< ~ bind:m (watch-app /~bud/groups/v1/groups [~bud %groups] /v1/groups) + ;< ~ bind:m (watch-app /~bud/groups/v1/foreigns [~bud %groups] /v1/foreigns) + :: ~zod hosts a group and invites ~bud with the %admin role. + :: + =/ =create-group:g + :* %my-test-group + ['My Test Group' 'My testing group' '' ''] + %secret + [~ ~] + (my ~bud^(sy %admin ~) ~) + == + ;< ~ bind:m (poke-app [~zod %groups] group-command+[%create create-group]) + ;< ~ bind:m (ex-r-groups-fact ~zod my-test-flag %create) + :: ~bud joins the group using the received invite token. + :: + ;< kag=cage bind:m (wait-for-app-fact /~bud/groups/v1/foreigns [~bud %groups]) + ?> =(%foreigns-1 p.kag) + =+ !<(=foreigns:v8:gv q.kag) + =+ foreign=(~(got by foreigns) my-test-flag) + ?> ?=(^ invites.foreign) + =/ =a-foreigns:v8:gv + [%foreign my-test-flag %join token.i.invites.foreign] + ;< ~ bind:m (poke-app [~bud %groups] group-foreign-2+a-foreigns) + ;< ~ bind:m (ex-r-groups-fact ~bud my-test-flag %create) + :: ~bud updates group metadata as an admin. the host applies the + :: update and the response propagates back to ~bud. + :: + =/ meta=data:meta:g + ['New Test Group' 'New testing group description' '#112233' '#445566'] + =/ =a-groups:v8:gv + [%group my-test-flag [%meta meta]] + ;< ~ bind:m (poke-app [~bud %groups] group-action-4+a-groups) + ;< ~ bind:m (ex-r-groups-meta ~zod my-test-flag meta) + ;< group=group:g bind:m (scry-test-group ~zod) + ;< ~ bind:m (ex-equal !>(meta.group) !>(meta)) + ;< ~ bind:m (ex-r-groups-meta ~bud my-test-flag meta) + (pure:m ~) -- diff --git a/docs/aqua/development.md b/docs/aqua/development.md new file mode 100644 index 0000000000..acd0928c86 --- /dev/null +++ b/docs/aqua/development.md @@ -0,0 +1,88 @@ +# Aqua test development + +Aqua tests allow a test to execute in a reproducible, virtual urbit environment. +When a test runs, it is presented with: +1. A clean slate fleet of virtual ships, usually galaxies. +2. A connection to the aqua virtual runtime, which allows to + interface with running virtual ships. + +A single test file can define multiple tests. Between the run of each test, +the test runner thread `-ph-test` takes care of resetting the test environment. + +## The Structure of Aqua Tests + +Each aqua tests is a core with test arms, where each +test arm has the prefix `ph-test` and resolves to a strand with the signature `(strand ,~)`. + +The thread runner builds a test file, extracts and builds all matching test arms. +To run a test, the runner will restore the specified aqua snapshot, and run the defined +test thread. If the thread returns a null, it means the execution was successful. +A thread failure indicates test failure. The test runner then displays the error. + +Here is an example test file with a single test case `+ph-test-sleep`. +It waits for `~s5` and always return successfully. +```hoon +/- spider +/+ *ph-io, *ph-test +=, strand=strand:spider +:: +ph-test-sleep: sleep test +:: +++ ph-test-sleep + =/ m (strand ,~) + ^- form:m + ;< ~ bind:m (sleep ~s5) + (pure:m ~) +-- +``` + +If the test thread were to hang and never return, the test runner would eventually timeout +the test and report a failure. + +### Libraries + +Let's look at the imports in the above example. The `spider` sur file is the thread interface. +Since each test case is a thread, we generally use it, primarily to define the strand type. + +Next, we have `ph-io`. This is the library for interacting with the aqua runtime from threads, and defines +strands to send events to aqua and other utilities. + +Finally, `ph-test` is a dedicated library for writing aqua tests. It currently contains many useful utilities +missing from `ph-io`, such as strands interfacing with virtual ships: poking, watching, receiving facts and scrying. +It also contains test assertions, such as `+ex-equal`, `+ex-not-equal`. + +## Developing Aqua Tests + +When developing a new aqua tests, we start by defining the scope of the test. The scope could encompass +a single system component, such as a single gall agent, or target multiple system components involved +in the process under testing. + +Once we have identified the components, we then prepare appropriate snapshot with a fleet size big enough +to accommodate test scenarios. + +While we develop the test, we can use a snapshot targeted to our use case. +However, when the new test ships to production, we must make sure that the snapshot used in production +has a big enough fleet size. + +### Test scenarios + +For each test scenario of the system, we start by writing a conscise description +of the test at the top of an appropriately named test arm. It should contain +1. A clear one line description of the test. +2. The test scenario, written from a third-person perspective. It should + contain the most important expect assertions involved in the test. + +Here is an example test arm for the group join process +```hoon +:: +ph-test-group-join: test group joins +:: +:: scenario +:: +:: ~zod hosts a group. ~bud joins the group. we verify +:: that the subscription lifecycle follows through %watch, and then %done. +:: finally, the group creation response is received. +:: +:: +++ ph-test-group-join + =/ m (strand ,~) + ^- form:m +``` diff --git a/docs/aqua/introduction.md b/docs/aqua/introduction.md index c58eeb375d..30924e95d4 100644 --- a/docs/aqua/introduction.md +++ b/docs/aqua/introduction.md @@ -8,6 +8,12 @@ there is no need to reinitialize it unless: 1. Aqua state has been reset. 2. We want the pill to incorporate some changes, such as an updated desk or additional files. +In the latter case, we also have a choice to prepare an aqua snapshot with desk sync enabled, +that will contain the updated desk irrespective of the version frozen in the pill. +This is usually preferrable, unless we plan to generate multiple snapshots. In that case, +generating a new pill containing updated desk will usually be faster than the accumulated +cost of syncing virtual desks for each ship in a snapshot. + To initialize aqua with a fresh pill, we poke aqua with an assembled pill. This can be done in one go by using the brass pill generator. ``` From 11ce098c3161142aaa5f9eb46bab308275a49419 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Wed, 6 May 2026 14:55:33 +0800 Subject: [PATCH 06/18] Add -ph-test-ls thread --- desk/ted/ph/test-ls.hoon | 150 ++++++++++++++++++++++++++++++++++++++ desk/ted/ph/test.hoon | 4 +- docs/aqua/development.md | 22 ++++-- docs/aqua/introduction.md | 46 ++++++++++-- 4 files changed, 208 insertions(+), 14 deletions(-) create mode 100644 desk/ted/ph/test-ls.hoon diff --git a/desk/ted/ph/test-ls.hoon b/desk/ted/ph/test-ls.hoon new file mode 100644 index 0000000000..690eed78e7 --- /dev/null +++ b/desk/ted/ph/test-ls.hoon @@ -0,0 +1,150 @@ +:: list aqua tests +:: +:: pax=(unit path) optional test path +:: +:: returns & if all tests build successfully, | otherwise. +:: +/- spider, aquarium +/+ *strandio, ph-io, ph-test +=, strand=strand:spider +|% ++$ test [=path strand=test-strand] ++$ test-arm [name=term strand=test-strand] ++$ test-strand _*form:(strand ,~) +-- +:: +|% +++ find-test-files + |= [byk=beak pax=path] + =/ m (strand ,(list path)) + ^- form:m + =/ dir=path + ;: weld + /(scot %p p.byk)/[q.byk]/(scot %da +.r.byk) + pax + == + %- pure:m + %+ skim .^((list ^path) %ct dir) + |=(=path =(%hoon (rear path))) +:: +build-test-files: build supplied test files +:: +++ build-test-files + |= [byk=beak files=(list path)] + =| out=(list (pair path vase)) + =| fail=(list path) + =/ m (strand ,[_out _fail]) + ^- form:m + |- + ?~ files (pure:m [(flop out) (flop fail)]) + =* file i.files + ;< build=(unit vase) bind:m (build-file byk file) + =* filename (spud file) + ?~ build + ~> %slog.3^leaf+"FAILED BUILD {filename}" + $(files t.files, fail [file fail]) + $(files t.files, out [[file u.build] out]) +:: +get-test-arms: get test arms contained in a core +:: +++ get-test-arms + |= [=path [typ=type cor=*]] + ^- (list test-arm) + =/ arms=(list @tas) (sloe typ) + %+ roll (skim arms has-test-prefix) + |= [name=term test-arms=(list test-arm)] + =/ fire-arm=(unit (pair type nock)) + =; res + ?: ?=(%| -.res) + %- %- %*(. slog pri 3) p.res + ~>(%slog.3^leaf+"FAILED MINT {<(snoc (snip path) name)>}" ~) + `p.res + %- mule + ^- (trap (pair type nock)) + |.((~(mint ut typ) -:!>(*test-strand) [%limb name])) + ?~ fire-arm + test-arms + =+ !<(=test-strand [p.u.fire-arm .*(cor q.u.fire-arm)]) + :_ test-arms + [name test-strand] +:: +has-test-prefix: does the arm define a test we should run? +:: +++ has-test-prefix + |= a=term ^- ? + =((end [3 8] a) 'ph-test-') +:: +resolve-test-paths: add test names to paths to form full test identifiers +:: +++ resolve-test-paths + |= paths-to-test=(list [path (list test-arm)]) + ^- (list test) + %- sort :_ |=([a=test b=test] (aor path.a path.b)) + ^- (list test) + %- zing + %+ turn paths-to-test + |= [=path test-arms=(list test-arm)] + ^- (list test) + :: for each test, add the test's name to .path + :: + %+ turn test-arms + |= =test-arm + ^- test + [(weld path /[name.test-arm]) strand.test-arm] +-- +^- thread:spider +|= args=vase +=/ m (strand ,vase) +^- form:m +=+ !<(pax=(unit path) args) +;< =bowl:spider bind:m get-bowl +=/ [byk=beak pax=path] + ?~ pax + [byk.bowl ~] + ?> ?=([ship=@ desk=@ date=@ rest=*] u.pax) + =+ ship=(slav %p i.u.pax) + =+ desk=i.t.u.pax + =+ date=(slav %da i.t.t.u.pax) + :_ t.t.t.u.pax + [ship desk da+date] +=? pax ?=(~ pax) + /tests +:: if the path does not point into a directory/file, assume last component +:: is a pattern. +:: +=/ =arch .^(arch %cy (weld /(scot %p p.byk)/[q.byk]/(scot %da +.r.byk) pax)) +=/ [arm-pat=(unit @ta) pax=path] + ?: ?=(^ dir.arch) + [~ pax] + :_ (snip pax) + `(rear pax) +;< test-files=(list path) bind:m (find-test-files byk pax) +=. test-files + %+ sort test-files + |=([a=path b=path] (aor (spat a) (spat b))) +;< [test-cores=(list (pair path vase)) failed-builds=(list path)] + bind:m (build-test-files byk test-files) +=/ test-sets=(list [path (list test-arm)]) + %+ turn test-cores + ?~ arm-pat + |=([=path =vase] [path (get-test-arms path vase)]) + :: filter based on arm pattern + :: + =+ len=(met 3 u.arm-pat) + |= [=path =vase] + :- path + %+ skim (get-test-arms path vase) + |= =test-arm + =((cut 3 [0 len] name.test-arm) u.arm-pat) +|- +?~ test-sets + (pure:m !>(&)) +=* path -.i.test-sets +=* test-arms +.i.test-sets +=+ num-arms=(lent test-arms) +?: =(0 num-arms) $(test-sets t.test-sets) +=/ test-num + ?: (gth num-arms 1) + "{} tests" + "{} test" +~> %slog.0^leaf+"{} ({test-num})" +|- +?~ test-arms ^$(test-sets t.test-sets) +~> %slog.0^leaf+" - {}" +$(test-arms t.test-arms) diff --git a/desk/ted/ph/test.hoon b/desk/ted/ph/test.hoon index 2650490e12..be70ded948 100644 --- a/desk/ted/ph/test.hoon +++ b/desk/ted/ph/test.hoon @@ -3,6 +3,8 @@ :: pax=(unit path) optional test path :: snap=@t aqua fleet snapshot :: +:: returns & if all tests passed, | otherwise. +:: /- spider, aquarium /+ *strandio, ph-io, ph-test =, strand=strand:spider @@ -161,7 +163,7 @@ =+ num=(lent tests) ?: =(num 0) ~> %slog.2^leaf+"No suitable aqua tests found" - (pure:m !>(~)) + (pure:m !>(&)) ~> %slog.1^leaf+"{} test {?:((gth num 1) "threads" "thread")} built" ~> %slog.1^'Running tests...' =/ n (strand (list (pair path thread-result))) diff --git a/docs/aqua/development.md b/docs/aqua/development.md index acd0928c86..7a8367954f 100644 --- a/docs/aqua/development.md +++ b/docs/aqua/development.md @@ -1,6 +1,6 @@ # Aqua test development -Aqua tests allow a test to execute in a reproducible, virtual urbit environment. +Aqua tests allow a test to execute in a reproducible, virtual urbit environment. When a test runs, it is presented with: 1. A clean slate fleet of virtual ships, usually galaxies. 2. A connection to the aqua virtual runtime, which allows to @@ -9,18 +9,30 @@ When a test runs, it is presented with: A single test file can define multiple tests. Between the run of each test, the test runner thread `-ph-test` takes care of resetting the test environment. +## Updating the desk + +When changes have been made in the repository and are ready to be tested +on a running ship, use `rsync` or equivalent command to copy the `desk/` in the repository +to appropriate pier. This will put the files into the ship's desk unix mount point. +To make the changes visible on the ship, you must also issue `|commit` command, which +will detect any changes in the unix mount point directory. + +Important: do not delete files not present in the repository at the destination. +Running urbit desks are usually supplemented with other neccessary files not present +in the repository. + ## The Structure of Aqua Tests Each aqua tests is a core with test arms, where each test arm has the prefix `ph-test` and resolves to a strand with the signature `(strand ,~)`. The thread runner builds a test file, extracts and builds all matching test arms. -To run a test, the runner will restore the specified aqua snapshot, and run the defined +To run a test, the runner will restore the specified aqua snapshot, and run the defined test thread. If the thread returns a null, it means the execution was successful. A thread failure indicates test failure. The test runner then displays the error. Here is an example test file with a single test case `+ph-test-sleep`. -It waits for `~s5` and always return successfully. +It waits for `~s5` and always return successfully. ```hoon /- spider /+ *ph-io, *ph-test @@ -54,10 +66,10 @@ It also contains test assertions, such as `+ex-equal`, `+ex-not-equal`. When developing a new aqua tests, we start by defining the scope of the test. The scope could encompass a single system component, such as a single gall agent, or target multiple system components involved -in the process under testing. +in the process under testing. Once we have identified the components, we then prepare appropriate snapshot with a fleet size big enough -to accommodate test scenarios. +to accommodate test scenarios. While we develop the test, we can use a snapshot targeted to our use case. However, when the new test ships to production, we must make sure that the snapshot used in production diff --git a/docs/aqua/introduction.md b/docs/aqua/introduction.md index 30924e95d4..fba4260571 100644 --- a/docs/aqua/introduction.md +++ b/docs/aqua/introduction.md @@ -1,4 +1,21 @@ -# Aqua tests +# Aqua + +Aqua is a virtual urbit runtime implemented in Hoon. +It virtualizes arvo instances and allows a virtual urbit ship to be +hosted inside the host arvo at a fraction of the memory cost. + +## Aqua tests + +Aqua is an ideal urbit testing environment. +It allows us to run fleets of virtual ships in a reproducible manner. +There is no need to implement stubs for any of the kernel components, given +that virtual arvo is just a copy of the real arvo running on the host. + +The only deficiency is that aqua currently does not implement all of the +runtime effects. However, while this might pose some difficulties when running +a virtual ship indented to interface with the urbit livenet, +all the functionality needed to perform tests on a self-contained +fleet of virtual ships is there. ## Preparing aqua pill @@ -90,14 +107,15 @@ The test runner will display test result and time taken after run of each test c The aqua virtual runtime is currently quite verbose, and will display a lot of noise, primarily about unhandled effects. -Running virtual ships will also display logging messages. +Running virtual ships will also display logging messages. These messages are associated with a priority: -| prefix | priority | -|-----------|----------| +| prefix | priority | +|-----------|------------------------------| | no prefix | `%dbug` debug priority | | `>` | `%info` information priority | | `>>` | `%warn` warning priority | | `>>>` | `%crit` critical priority | + Each logging message is prefix by the virtual source ship and agent names. An example error message @@ -113,12 +131,24 @@ the location indicated in the message. In the above example, that would be the `+se-u-groups` arm somewhere in the agent's source code or its libraries. If that does not yield results, searching for constant parts of the messsage in relevant files usually. -## Listing tests -To find available aqua tests, use `-ph-test-ls path` command. -It will compile all files present at the path and assemble a list -of test arms that would be run by the test runner. +## Checking tests + +While developing tests, it is useful to have a way to verify any compilation errors +separately before triggering the actual test run. This can be done using +`-ph-test-ls path` command, which will find all aqua tests available at the +path and build them. + +There are two kind of errors that can occur at compilation time. +A `FAILED BUILD` error indicates the test file did not build. +The compiler error is shown directly above, because it is coming from clay. +A `FAILED MINT` error indicates that a test arm in a successfully compiled test +file does not resolve properly. Usually this indicates that the arm resolved +to a type that does not match the test signature, which should be a form of the strand `(strand ,~)`. +The possible compiler error is also displayed right the error line, for consistency +with file build errors. ## Cancelling a test run + The test runner is a thread, so in order to stop an ongoing test run you must cancel the runner thread. In dojo, this is simply done by pressing a backspace. From af07683d74124f84b2ac236e6ef6d8c2279bdfc5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Fri, 8 May 2026 13:29:58 +0800 Subject: [PATCH 07/18] Aqua tests: improve agent harness --- desk/app/contacts.hoon | 9 +++ desk/lib/ph/test.hoon | 17 +++++ desk/tests/ph/app/groups.hoon | 9 +-- docs/aqua/development.md | 137 ++++++++++++++++++++++++++++++---- docs/aqua/introduction.md | 27 +++++++ 5 files changed, 177 insertions(+), 22 deletions(-) diff --git a/desk/app/contacts.hoon b/desk/app/contacts.hoon index e6252cbd91..399c966fc7 100644 --- a/desk/app/contacts.hoon +++ b/desk/app/contacts.hoon @@ -1,3 +1,12 @@ +:: contacts: user profile and contact book +:: +:: the contacts agent manages the user's public profile, as well +:: as track profiles of network peers, creating an implicit social graph. +:: it also manages the contact book, which contains the profiles of +:: network peers marked as contacts, together with user-defined +:: metadata and profile data overlays. +:: +:: /- activity-ver /+ default-agent, dbug, verb, neg=negotiate /+ *contacts, kol diff --git a/desk/lib/ph/test.hoon b/desk/lib/ph/test.hoon index 4635679403..e22b914d2d 100644 --- a/desk/lib/ph/test.hoon +++ b/desk/lib/ph/test.hoon @@ -10,6 +10,7 @@ :: ++ ph-test-shut (leave-our /effect/unto %aqua) +:: +take-effect: receive aqua effect on a .wire :: ++ take-effect |= =wire @@ -19,6 +20,14 @@ ?> ?=(%aqua-effect p.res) =+ !<(=aqua-effect q.res) (pure:m aqua-effect) +:: +poke-app: poke a gall agent on a virtual ship +:: +:: .dock: target virtual ship and agent +:: .page: poke payload +:: +:: note that the poke is a $page, not a $vase. +:: this is because pokes to virtual ships are injected +:: into arvo, and thus need to pass through an untyped interface. :: ++ poke-app |= [=dock =page] @@ -38,6 +47,10 @@ ?^ p.sign (strand-fail %poke-ack u.p.sign) (pure:m ~) +:: +watch-app: watch a gall subscription to a virtual ship +:: +:: the resulting facts are received as aqua effects. +:: see +wait-for-app-fact. :: ++ watch-app |= [=wire =dock =path] @@ -60,6 +73,7 @@ ?^ p.sign (strand-fail %watch-ack u.p.sign) (pure:m ~) +:: +leave-app: leave a gall subscription to a virtual ship :: ++ leave-app |= [=wire =dock] @@ -71,6 +85,7 @@ [%event p.dock [%g wire] task] ;< ~ bind:m (send-events ~[aqua-event]) (pure:m ~) +:: +wait-for-app-fact: receive a gall fact from a virtual ship :: ++ wait-for-app-fact |= [=wire [our=ship dap=term]] @@ -89,6 +104,7 @@ =+ .^(=dais:clay %cb /(scot %p our.bowl)/groups/(scot %da now.bowl)/[mark]) =/ =vase (vale:dais noun.p.q.unix-effect) (pure:m [mark vase]) +:: +ex-equal: expect .actual to be equal to .expected :: ++ ex-equal |= [actual=vase expected=vase] @@ -99,6 +115,7 @@ ?~ tang `[%done ~] `[%fail %ex-equal tang] +:: +ex-not-equal: expect .actual not to be equal to .expected :: ++ ex-not-equal |= [actual=vase expected=vase] diff --git a/desk/tests/ph/app/groups.hoon b/desk/tests/ph/app/groups.hoon index bc2e8a1e04..5c09d4d49b 100644 --- a/desk/tests/ph/app/groups.hoon +++ b/desk/tests/ph/app/groups.hoon @@ -56,15 +56,13 @@ :: :: scenario :: -:: ~zod hosts a group. ~bud joins the group. we verify -:: that the subscription lifecycle follows through %watch, and then %done. -:: finally, the group creation response is received. -:: +:: ~zod hosts a group and sends an invitation to ~bud. +:: ~bud receives the invitation and joins the group successfully, +:: receiving the group creation fact. :: ++ ph-test-group-join =/ m (strand ,~) ^- form:m - ;< ~ bind:m (watch-app /~zod/groups/v1/groups [~zod %groups] /v1/groups) ;< ~ bind:m (watch-app /~bud/groups/v1/groups [~bud %groups] /v1/groups) ;< ~ bind:m (watch-app /~bud/groups/v1/foreigns [~bud %groups] /v1/foreigns) :: ~zod hosts a group and invites ~bud @@ -88,7 +86,6 @@ [%foreign my-test-flag %join token.i.invites.foreign] ;< ~ bind:m (poke-app [~bud %groups] group-foreign-2+a-foreigns) ;< ~ bind:m (ex-r-groups-fact ~bud ~zod^%my-test-group %create) - ;< =bowl:strand bind:m get-bowl (pure:m ~) :: +ph-test-admin-group-meta: test admin group metadata update :: diff --git a/docs/aqua/development.md b/docs/aqua/development.md index 7a8367954f..456f7b1cbc 100644 --- a/docs/aqua/development.md +++ b/docs/aqua/development.md @@ -3,10 +3,10 @@ Aqua tests allow a test to execute in a reproducible, virtual urbit environment. When a test runs, it is presented with: 1. A clean slate fleet of virtual ships, usually galaxies. -2. A connection to the aqua virtual runtime, which allows to +2. A connection to the aqua virtual runtime, which allows it to interface with running virtual ships. -A single test file can define multiple tests. Between the run of each test, +A single test file can define any number tests. Between the run of each test, the test runner thread `-ph-test` takes care of resetting the test environment. ## Updating the desk @@ -64,37 +64,142 @@ It also contains test assertions, such as `+ex-equal`, `+ex-not-equal`. ## Developing Aqua Tests -When developing a new aqua tests, we start by defining the scope of the test. The scope could encompass +When developing a new aqua test, we start by specifying the scope of the test. The scope could encompass a single system component, such as a single gall agent, or target multiple system components involved in the process under testing. -Once we have identified the components, we then prepare appropriate snapshot with a fleet size big enough +Once we have identified the components we then prepare appropriate snapshot with a fleet size big enough to accommodate test scenarios. While we develop the test, we can use a snapshot targeted to our use case. However, when the new test ships to production, we must make sure that the snapshot used in production -has a big enough fleet size. +has a big enough fleet size. If that is not the case, the production +test runner must be updated. ### Test scenarios -For each test scenario of the system, we start by writing a conscise description -of the test at the top of an appropriately named test arm. It should contain -1. A clear one line description of the test. -2. The test scenario, written from a third-person perspective. It should - contain the most important expect assertions involved in the test. - -Here is an example test arm for the group join process +Each test arm should correspond to a single test scenario. +When developing a new test, we start out by describing the scenario +in the comment directly above the test arm. The test scenario should be +be brief and written from third-person perspective. It should not +overwhelm the reader with too much detail, but should contain essential +information about assertion made during the test. +Here is an example test arm for the group join process: ```hoon :: +ph-test-group-join: test group joins :: :: scenario :: -:: ~zod hosts a group. ~bud joins the group. we verify -:: that the subscription lifecycle follows through %watch, and then %done. -:: finally, the group creation response is received. -:: +:: ~zod hosts a group and sends an invitation to ~bud. +:: ~bud receives the invitation and joins the group successfully, +:: receiving the group creation fact. :: ++ ph-test-group-join =/ m (strand ,~) ^- form:m +::... +``` + +### Test assertions + +Aqua tests do not have direct access to the underlying gall agent. The +test can only interface with the virtual ship through arvo tasks. +The perspective is that of a client integrating with a particular system component. + +When testing gall agents, we have essentially two ways to approach test +assertions. The first one is to establish a subscription and assert on +facts. The second is to directly query an agent state with a scry. + +Here is the full group join test +```hoon +++ ph-test-group-join + =/ m (strand ,~) + ^- form:m + ;< ~ bind:m (watch-app /~bud/groups/v1/groups [~bud %groups] /v1/groups) + ;< ~ bind:m (watch-app /~bud/groups/v1/foreigns [~bud %groups] /v1/foreigns) + :: ~zod hosts a group and invites ~bud + :: + =/ =create-group:g + :* %my-test-group + ['My Test Group' 'My testing group' '' ''] + %secret + [~ ~] + (my ~bud^~ ~) + == + ;< ~ bind:m (poke-app [~zod %groups] group-command+[%create create-group]) + :: ~bud joins the group using received invite token + :: + ;< kag=cage bind:m (wait-for-app-fact /~bud/groups/v1/foreigns [~bud %groups]) + ?> =(%foreigns-1 p.kag) + =+ !<(=foreigns:v8:gv q.kag) + =+ foreign=(~(got by foreigns) my-test-flag) + ?> ?=(^ invites.foreign) + =/ =a-foreigns:v8:gv + [%foreign my-test-flag %join token.i.invites.foreign] + ;< ~ bind:m (poke-app [~bud %groups] group-foreign-2+a-foreigns) + ;< ~ bind:m (ex-r-groups-fact ~bud ~zod^%my-test-group %create) + (pure:m ~) ``` +We first establish two app subscriptions to `%groups` on `~bud`. +After `~zod` created the group, we expect `~bud` to first receive the +group invitation on the foreigns subscription. After we have verified +that an invitation has indeed been received, we poke `~bud` to join the +group. We verify that the group has been successfully joined by +expecting a group creation response on the groups subscription. + +### Test assertions and Hoon assertions + +When asserting on a value expected by the test, we have two choices. +We can aqua assertion, such as `+ex-equal`. This will terminate the +thread with an appropriate error message, describing the discrepancy +between expected and actual values. Alternatively, we can use any of the +Hoon assertions such as `?>` or `?<`, which will simply crash the thread +without a specific error message. The obvious downside of Hoon +assertions is that they don't carry any information about the way +assertion has failed. + +We should generally use aqua assertions. However, when asserting on +things that are considered unchengable parts of an interface, and do not in +themselves implement any logic which could be broken, using +Hoon assertions is permissible. One example is asserting on marks of +received facts. Since these are generally considered fixed, crashing +with a Hoon assertions is more ergonomic. + +Sometimes we might be tempted to use Hoon assertions out of pure convenience, +such using crashing map getter `+get:by`, or unpacking a unit using `?>`. +However, taking such shortcuts will simply make for a later inconvenience when debugging a +broken test. + +When investigating a test failing on a Hoon assertion, which does not +display any values, using the debug print rune `~&` can be a good way to +investigate the problem. If the test failure does not stem from an +originally wrong implementation, it is likely the case that it could be +broken in the future and we should consider to convert it to an aqua +assertion. + +### Implementing a test + +We have so far talked about the structure of an aqua test. It is +composed of a prose description, followed by the test strand +implementation, which essentially is a sequence of events send to the aqua runtime +or assertions on various effects received. + +We are now going to focus on the test implementation. Once we have +specified the test scenario, how do we go about implementing it? +In some cases, it might be enough to look at relevant portions of the code to +be able to specify the sequence of events and assertions. However, +discerning the exact test process and data involved is not always easy just +by reading the code. Sometimes, the functionality might be spread across many libraries +or agents. + +Fortunately, we can observe the running system by enabling debugging mode. +Any agent that integrates the logging library, together with the verb wrapper, will +display debugging messages when enabled. To enable debugging mode, we adjust the logging +volume +``` +> :agent &verb [%volume volume] +``` +where volume can be any of `%dbug`, `%info`, `%warn` and `%crit`. +Adjusting the volume to `%dbug` will enable printing of all log messages at that priority or above. + + diff --git a/docs/aqua/introduction.md b/docs/aqua/introduction.md index fba4260571..ea5e2ad1fc 100644 --- a/docs/aqua/introduction.md +++ b/docs/aqua/introduction.md @@ -80,6 +80,28 @@ and `~rivfur-livmet` respectively. In addition, virtual ames/mesa networking req the parent galaxies to be present. In this case, we must add `~fen` and `~dem` to the fleet. This is handled by the `-ph-fleet` thread +When a host ship accumulated enough snapshots to be a memory burden, we +can delete all existing snapshots using an aqua poke: +``` +> :aqua &noun [%clear-snap snap-id] +``` + +**Important**: at present aqua does not handle the snap-id argument, and will delete +all snapshots upon receiving this poke. + +## Updating the desk + +When changes have been made in the repository and are ready to be tested +on a running ship, use `rsync` or equivalent command to copy the `desk/` in the repository +to an appropriate pier. This will put the files into the ship's desk unix mount point directory. +To make the changes visible on the ship, you must also issue `|commit` command, which +will detect any changes in the unix mount point directory. `|commit` can display `sync` spinner, +which indicates the commit is still ongoing. + +Important: do not delete files not present in the repository at the destination. +Running urbit desks are usually supplemented with other neccessary files not present +in the repository. + ## Running tests To run aqua tests use the `-ph-test` test runner supplied with the desk. @@ -109,6 +131,7 @@ primarily about unhandled effects. Running virtual ships will also display logging messages. These messages are associated with a priority: + | prefix | priority | |-----------|------------------------------| | no prefix | `%dbug` debug priority | @@ -152,3 +175,7 @@ with file build errors. The test runner is a thread, so in order to stop an ongoing test run you must cancel the runner thread. In dojo, this is simply done by pressing a backspace. +## Developing aqua tests + +To learn how to develop aqua tests, see ./docs/aqua/development.md + From 24e1dc4935798c9b2d89e4aa7821b57d42210603 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Fri, 8 May 2026 14:50:44 +0800 Subject: [PATCH 08/18] contacts: add aqua tests --- backend/run-aqua-tests.sh | 7 ++- desk/ted/ph/test-ls.hoon | 1 + desk/tests/ph/app/contacts.hoon | 92 +++++++++++++++++++++++++++++++++ desk/tests/ph/app/groups.hoon | 52 ------------------- docs/aqua/development.md | 41 ++++++++------- docs/aqua/introduction.md | 11 +++- 6 files changed, 131 insertions(+), 73 deletions(-) create mode 100644 desk/tests/ph/app/contacts.hoon diff --git a/backend/run-aqua-tests.sh b/backend/run-aqua-tests.sh index a0136d205c..fbe940901f 100755 --- a/backend/run-aqua-tests.sh +++ b/backend/run-aqua-tests.sh @@ -209,7 +209,7 @@ then exit 1 fi -echo "Starting %aqua..." +echo "Starting aqua..." ${run_click} $pier "/lib/pill/hoon"<(%ok)) EOF +echo "Preparing aqua snapshot" +# The snap thread needs to accept id argument. + # Run the unit tests +# +# Update to use the generated test snapshot echo "Running tests..." result=$( $run_click -t 1200 $pier < =(%contact-response-0 p.pay) + =+ !<(res=response:c q.pay) + ?> ?=(%peer -.res) + ;< ~ bind:m (ex-equal !>(who.res) !>(~zod)) + :: ~zod updates his public profile, then ~bud receives the broadcast. + :: + ;< ~ bind:m (poke-app [~zod %contacts] contact-action-1+[%self test-profile]) + ;< pay=cage bind:m (wait-for-app-fact /~bud/contacts/v1/news [~bud %contacts]) + ?> =(%contact-response-0 p.pay) + =+ !<(res=response:c q.pay) + ?> ?=(%peer -.res) + ;< ~ bind:m (ex-equal !>(who.res) !>(~zod)) + ;< ~ bind:m (ex-equal !>(con.res) !>(test-profile)) + (pure:m ~) +:: +ph-test-profile-field-types: test profile field value types +:: +:: scenario +:: +:: ~zod starts with an empty contact profile. +:: ~zod updates his profile with fields covering every contact value type, +:: then publishes the updated profile to local subscribers. +:: +++ ph-test-profile-field-types + =/ m (strand ,~) + ^- form:m + ;< ~ bind:m (watch-app /~zod/contacts/v1/contact [~zod %contacts] /v1/contact) + ;< pay=cage bind:m (wait-for-app-fact /~zod/contacts/v1/contact [~zod %contacts]) + ?> =(%contact-update-1 p.pay) + =+ !<(upd=update:c q.pay) + ?> ?=(%full -.upd) + ;< ~ bind:m (ex-equal !>(con.upd) !>(*contact:c)) + ;< ~ bind:m (watch-app /~zod/contacts/v1/news [~zod %contacts] /v1/news) + :: ~zod sets every contact value type and receives the local response. + :: + ;< ~ bind:m (poke-app [~zod %contacts] contact-action-1+[%self typed-profile]) + ;< pay=cage bind:m (wait-for-app-fact /~zod/contacts/v1/news [~zod %contacts]) + ?> =(%contact-response-0 p.pay) + =+ !<(res=response:c q.pay) + ?> ?=(%self -.res) + ;< ~ bind:m (ex-equal !>(con.res) !>(typed-profile)) + :: ~zod then publishes his full updated profile to contact subscribers. + :: + ;< pay=cage bind:m (wait-for-app-fact /~zod/contacts/v1/contact [~zod %contacts]) + ?> =(%contact-update-1 p.pay) + =+ !<(upd=update:c q.pay) + ?> ?=(%full -.upd) + ;< ~ bind:m (ex-equal !>(con.upd) !>(typed-profile)) + (pure:m ~) +-- diff --git a/desk/tests/ph/app/groups.hoon b/desk/tests/ph/app/groups.hoon index 5c09d4d49b..e79dfdd0c6 100644 --- a/desk/tests/ph/app/groups.hoon +++ b/desk/tests/ph/app/groups.hoon @@ -87,55 +87,3 @@ ;< ~ bind:m (poke-app [~bud %groups] group-foreign-2+a-foreigns) ;< ~ bind:m (ex-r-groups-fact ~bud ~zod^%my-test-group %create) (pure:m ~) -:: +ph-test-admin-group-meta: test admin group metadata update -:: -:: scenario -:: -:: ~zod hosts a group and invites ~bud as an admin. ~bud joins the group. -:: after joining, ~bud issues a metadata update from their own ship. we -:: verify that the host applies the metadata update, and that the same -:: metadata update propagates back to ~bud. -:: -++ ph-test-admin-group-meta - =/ m (strand ,~) - ^- form:m - ;< ~ bind:m (watch-app /~zod/groups/v1/groups [~zod %groups] /v1/groups) - ;< ~ bind:m (watch-app /~bud/groups/v1/groups [~bud %groups] /v1/groups) - ;< ~ bind:m (watch-app /~bud/groups/v1/foreigns [~bud %groups] /v1/foreigns) - :: ~zod hosts a group and invites ~bud with the %admin role. - :: - =/ =create-group:g - :* %my-test-group - ['My Test Group' 'My testing group' '' ''] - %secret - [~ ~] - (my ~bud^(sy %admin ~) ~) - == - ;< ~ bind:m (poke-app [~zod %groups] group-command+[%create create-group]) - ;< ~ bind:m (ex-r-groups-fact ~zod my-test-flag %create) - :: ~bud joins the group using the received invite token. - :: - ;< kag=cage bind:m (wait-for-app-fact /~bud/groups/v1/foreigns [~bud %groups]) - ?> =(%foreigns-1 p.kag) - =+ !<(=foreigns:v8:gv q.kag) - =+ foreign=(~(got by foreigns) my-test-flag) - ?> ?=(^ invites.foreign) - =/ =a-foreigns:v8:gv - [%foreign my-test-flag %join token.i.invites.foreign] - ;< ~ bind:m (poke-app [~bud %groups] group-foreign-2+a-foreigns) - ;< ~ bind:m (ex-r-groups-fact ~bud my-test-flag %create) - :: ~bud updates group metadata as an admin. the host applies the - :: update and the response propagates back to ~bud. - :: - =/ meta=data:meta:g - ['New Test Group' 'New testing group description' '#112233' '#445566'] - =/ =a-groups:v8:gv - [%group my-test-flag [%meta meta]] - ;< ~ bind:m (poke-app [~bud %groups] group-action-4+a-groups) - ;< ~ bind:m (ex-r-groups-meta ~zod my-test-flag meta) - ;< group=group:g bind:m (scry-test-group ~zod) - ;< ~ bind:m (ex-equal !>(meta.group) !>(meta)) - ;< ~ bind:m (ex-r-groups-meta ~bud my-test-flag meta) - (pure:m ~) --- - diff --git a/docs/aqua/development.md b/docs/aqua/development.md index 456f7b1cbc..aea6ce398a 100644 --- a/docs/aqua/development.md +++ b/docs/aqua/development.md @@ -9,18 +9,6 @@ When a test runs, it is presented with: A single test file can define any number tests. Between the run of each test, the test runner thread `-ph-test` takes care of resetting the test environment. -## Updating the desk - -When changes have been made in the repository and are ready to be tested -on a running ship, use `rsync` or equivalent command to copy the `desk/` in the repository -to appropriate pier. This will put the files into the ship's desk unix mount point. -To make the changes visible on the ship, you must also issue `|commit` command, which -will detect any changes in the unix mount point directory. - -Important: do not delete files not present in the repository at the destination. -Running urbit desks are usually supplemented with other neccessary files not present -in the repository. - ## The Structure of Aqua Tests Each aqua tests is a core with test arms, where each @@ -83,7 +71,7 @@ When developing a new test, we start out by describing the scenario in the comment directly above the test arm. The test scenario should be be brief and written from third-person perspective. It should not overwhelm the reader with too much detail, but should contain essential -information about assertion made during the test. +information about assertion made during the test. Here is an example test arm for the group join process: ```hoon :: +ph-test-group-join: test group joins @@ -100,6 +88,23 @@ Here is an example test arm for the group join process: ::... ``` +## Comment style guide + +Following a long maritime tradition, Urbit ships generally use feminine pronouns. +However for galaxies, which hold authority over the network, use masculine pronouns. + +In comments describing a sequence of assertions, try to use sequential +language. For instance, prefer "then" rather than "and" to describe a +sequence of events happening one after another. This is rule is not absolute, +sometimes, especially when talking about events concerning the same +ship, we might profitably employ "and". + +Prefer "and" to "then" especially if two events logically follow one another +in the context of the test. For example, we prefer to use "and" in "~zod hosts a +group and invites ~bud", since in the context of the test, there is no +decision on `~zod`'s part involved: the group was created _so that_ +`~bud` could join it. + ### Test assertions Aqua tests do not have direct access to the underlying gall agent. The @@ -163,12 +168,12 @@ things that are considered unchengable parts of an interface, and do not in themselves implement any logic which could be broken, using Hoon assertions is permissible. One example is asserting on marks of received facts. Since these are generally considered fixed, crashing -with a Hoon assertions is more ergonomic. +with a Hoon assertions is more ergonomic. -Sometimes we might be tempted to use Hoon assertions out of pure convenience, -such using crashing map getter `+get:by`, or unpacking a unit using `?>`. +Sometimes we might be tempted to use Hoon assertions out of pure convenience, +such using crashing map getter `+get:by`, or unpacking a unit using `?>`. However, taking such shortcuts will simply make for a later inconvenience when debugging a -broken test. +broken test. When investigating a test failing on a Hoon assertion, which does not display any values, using the debug print rune `~&` can be a good way to @@ -181,7 +186,7 @@ assertion. We have so far talked about the structure of an aqua test. It is composed of a prose description, followed by the test strand -implementation, which essentially is a sequence of events send to the aqua runtime +implementation, which essentially is a sequence of events send to the aqua runtime or assertions on various effects received. We are now going to focus on the test implementation. Once we have diff --git a/docs/aqua/introduction.md b/docs/aqua/introduction.md index ea5e2ad1fc..4b3aaac2f9 100644 --- a/docs/aqua/introduction.md +++ b/docs/aqua/introduction.md @@ -108,7 +108,7 @@ To run aqua tests use the `-ph-test` test runner supplied with the desk. ``` > -desk!ph-test `path snap ``` -The `path` is a directory containing aqua tests. Supply `~` to run from the default directory. +The `path` is a directory containing aqua tests (remember to include the backtick). Supply `~` to run from the default directory. It will be scanned recursively, and any tests found will be build and scheduled to be run. `snap` is the aqua snapshot on which tests are going to run. @@ -159,7 +159,14 @@ If that does not yield results, searching for constant parts of the messsage in While developing tests, it is useful to have a way to verify any compilation errors separately before triggering the actual test run. This can be done using `-ph-test-ls path` command, which will find all aqua tests available at the -path and build them. +path and build them. For example, to list all aqua files in the groups desk, we would run +``` +> -groups!ph-test-ls /=groups=/tests/ph +``` +, and to list tests for a particular agent we would use +``` +> -groups!ph-test-ls /=groups=/tests/ph/app/contacts +``` There are two kind of errors that can occur at compilation time. A `FAILED BUILD` error indicates the test file did not build. From b0b703875653c1272ee02c1b8a9920b574270ac6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Mon, 11 May 2026 13:01:30 +0800 Subject: [PATCH 09/18] Aqua tests: improve docs --- desk/tests/ph/app/groups.hoon | 25 ------------------------- docs/aqua/development.md | 12 +++++++++++- 2 files changed, 11 insertions(+), 26 deletions(-) diff --git a/desk/tests/ph/app/groups.hoon b/desk/tests/ph/app/groups.hoon index e79dfdd0c6..88b5f08302 100644 --- a/desk/tests/ph/app/groups.hoon +++ b/desk/tests/ph/app/groups.hoon @@ -27,31 +27,6 @@ ;< ~ bind:m (ex-equal !>(flag.rep) !>(flag)) ;< ~ bind:m (ex-equal !>(`@tas`-.r-group.rep) !>(tag)) (pure:m ~) -:: +ex-r-groups-meta: expect group metadata response fact -:: -++ ex-r-groups-meta - |= [=ship =flag:gv meta=data:meta:g] - =/ m (strand ,~) - ^- form:m - ;< kag=cage bind:m (wait-for-app-fact /(scot %p ship)/groups/v1/groups [ship %groups]) - ?> =(%group-response-1 p.kag) - =+ !<(rep=r-groups:v10:gv q.kag) - ;< ~ bind:m (ex-equal !>(flag.rep) !>(flag)) - ?> ?=(%meta -.r-group.rep) - ;< ~ bind:m (ex-equal !>(meta.r-group.rep) !>(meta)) - (pure:m ~) -:: +scry-test-group: scry group state from a virtual ship -:: -++ scry-test-group - |= =ship - =/ m (strand group:g) - ^- form:m - ;< =bowl:strand bind:m get-bowl - =/ aqua-pax - /gx/(scot %p ship)/groups/(scot %da now.bowl)/v2/groups/~zod/my-test-group/group-2 - ;< group=(unit group:v9:gv) bind:m - (scry-aqua (unit group:v9:gv) ship aqua-pax) - (pure:m `group:g`(need group)) :: +ph-test-group-join: test group joins :: :: scenario diff --git a/docs/aqua/development.md b/docs/aqua/development.md index aea6ce398a..43da9b7105 100644 --- a/docs/aqua/development.md +++ b/docs/aqua/development.md @@ -11,7 +11,7 @@ the test runner thread `-ph-test` takes care of resetting the test environment. ## The Structure of Aqua Tests -Each aqua tests is a core with test arms, where each +An aqua test is a core with test arms, where each test arm has the prefix `ph-test` and resolves to a strand with the signature `(strand ,~)`. The thread runner builds a test file, extracts and builds all matching test arms. @@ -182,6 +182,16 @@ originally wrong implementation, it is likely the case that it could be broken in the future and we should consider to convert it to an aqua assertion. +### Interfacing with gall agents + +The only interface exposed by aqua is that of an Urbit runtime, with its +4 standard arms. Aqua exposes an interface to poke and scry virtual +arvo, as well as receive effects. +It is therefore not possible to interface directly through gall API with apps running inside virtual ships. +Instead, we use arvo tasks to pass messages to vanes running on a +virtual ship. To receive effects, we can subscribe to a generic aqua +endpoint, and also target a specific type of arvo effects by using a specific subscription path. + ### Implementing a test We have so far talked about the structure of an aqua test. It is From 44f499c339b8b71f26bc038fe7c6d8a68b14aa08 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Mon, 11 May 2026 15:12:50 +0800 Subject: [PATCH 10/18] Aqua tests: misc fixes --- desk/ted/ph/fleet.hoon | 22 +++++++++++++--------- desk/tests/ph/app/contacts.hoon | 14 +++++++------- desk/tests/ph/app/groups.hoon | 1 + 3 files changed, 21 insertions(+), 16 deletions(-) diff --git a/desk/ted/ph/fleet.hoon b/desk/ted/ph/fleet.hoon index 9361f45163..b04974ed70 100644 --- a/desk/ted/ph/fleet.hoon +++ b/desk/ted/ph/fleet.hoon @@ -1,6 +1,8 @@ :: prepare aqua fleet snapshot :: +:: snap-id=(unit @t) :: fleet=(list ship) +:: sync=? :: /- spider, aquarium /+ *strandio, ph-io, ph-test @@ -48,10 +50,10 @@ |= args=vase =/ m (strand ,vase) ^- form:m -=+ !<(args=(unit [fleet=(list ship) sync=?]) args) -=/ [fleet=(list ship) sync=?] +=+ !<(args=(unit [snap-id=(unit @t) fleet=(list ship) sync=?]) args) +=/ [snap-id=(unit @t) fleet=(list ship) sync=?] ?~ args ~|(%no-args-found !!) - [fleet sync]:u.args + [snap-id fleet sync]:u.args =. fleet ^- (list ship) :* ~loshut-lonreg :: bait provider @@ -79,9 +81,11 @@ ;< ~ bind:n (sync-desk i.fleet %groups) $(fleet t.fleet) ;< =bowl:spider bind:m get-bowl -=+ snap-id=(end 3^4 (sham eny.bowl)) -=/ snap=@t - (cat 3 'aqua-tests-' (scot %uv snap-id)) -~> %slog.1^(crip "Taking snapshot...") -;< ~ bind:m (send-events:ph-io [%snap-ships snap fleet]~) -(pure:m !>(snap)) +=/ snap-id=@t + ?~ snap-id + =+ eny=(end 3^4 (sham eny.bowl)) + (cat 3 'aqua-tests-' (scot %uv eny)) + u.snap-id +~> %slog.1^(crip "Taking snapshot {}...") +;< ~ bind:m (send-events:ph-io [%snap-ships snap-id fleet]~) +(pure:m !>(snap-id)) diff --git a/desk/tests/ph/app/contacts.hoon b/desk/tests/ph/app/contacts.hoon index a29dcf2e49..7dc9b6a929 100644 --- a/desk/tests/ph/app/contacts.hoon +++ b/desk/tests/ph/app/contacts.hoon @@ -2,7 +2,7 @@ /+ *ph-io, *ph-test =, strand=strand:spider |% -++ test-profile +++ my-broadcast-profile ^- contact:c %- ~(gas by *(map @tas value:c)) :~ [%nickname text+'Zod Test'] @@ -10,7 +10,7 @@ [%status text+'testing contacts'] [%color tint+0xff.0000] == -++ typed-profile +++ my-typed-profile ^- contact:c %- ~(gas by *(map @tas value:c)) :~ [%nickname text+'Zod Typed'] @@ -47,13 +47,13 @@ ;< ~ bind:m (ex-equal !>(who.res) !>(~zod)) :: ~zod updates his public profile, then ~bud receives the broadcast. :: - ;< ~ bind:m (poke-app [~zod %contacts] contact-action-1+[%self test-profile]) + ;< ~ bind:m (poke-app [~zod %contacts] contact-action-1+[%self my-broadcast-profile]) ;< pay=cage bind:m (wait-for-app-fact /~bud/contacts/v1/news [~bud %contacts]) ?> =(%contact-response-0 p.pay) =+ !<(res=response:c q.pay) ?> ?=(%peer -.res) ;< ~ bind:m (ex-equal !>(who.res) !>(~zod)) - ;< ~ bind:m (ex-equal !>(con.res) !>(test-profile)) + ;< ~ bind:m (ex-equal !>(con.res) !>(my-broadcast-profile)) (pure:m ~) :: +ph-test-profile-field-types: test profile field value types :: @@ -75,18 +75,18 @@ ;< ~ bind:m (watch-app /~zod/contacts/v1/news [~zod %contacts] /v1/news) :: ~zod sets every contact value type and receives the local response. :: - ;< ~ bind:m (poke-app [~zod %contacts] contact-action-1+[%self typed-profile]) + ;< ~ bind:m (poke-app [~zod %contacts] contact-action-1+[%self my-typed-profile]) ;< pay=cage bind:m (wait-for-app-fact /~zod/contacts/v1/news [~zod %contacts]) ?> =(%contact-response-0 p.pay) =+ !<(res=response:c q.pay) ?> ?=(%self -.res) - ;< ~ bind:m (ex-equal !>(con.res) !>(typed-profile)) + ;< ~ bind:m (ex-equal !>(con.res) !>(my-typed-profile)) :: ~zod then publishes his full updated profile to contact subscribers. :: ;< pay=cage bind:m (wait-for-app-fact /~zod/contacts/v1/contact [~zod %contacts]) ?> =(%contact-update-1 p.pay) =+ !<(upd=update:c q.pay) ?> ?=(%full -.upd) - ;< ~ bind:m (ex-equal !>(con.upd) !>(typed-profile)) + ;< ~ bind:m (ex-equal !>(con.upd) !>(my-typed-profile)) (pure:m ~) -- diff --git a/desk/tests/ph/app/groups.hoon b/desk/tests/ph/app/groups.hoon index 88b5f08302..a49203c431 100644 --- a/desk/tests/ph/app/groups.hoon +++ b/desk/tests/ph/app/groups.hoon @@ -62,3 +62,4 @@ ;< ~ bind:m (poke-app [~bud %groups] group-foreign-2+a-foreigns) ;< ~ bind:m (ex-r-groups-fact ~bud ~zod^%my-test-group %create) (pure:m ~) +-- From f2b289b0e8db517bd25734098261a84fddcf1610 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Mon, 11 May 2026 18:34:59 +0800 Subject: [PATCH 11/18] Aqua tests: fix runner --- backend/run-aqua-tests.sh | 46 +++++++++++++++++++++++++++++++++------ docs/aqua/development.md | 5 +++++ docs/aqua/introduction.md | 3 +++ 3 files changed, 47 insertions(+), 7 deletions(-) diff --git a/backend/run-aqua-tests.sh b/backend/run-aqua-tests.sh index fbe940901f..251ae71cd3 100755 --- a/backend/run-aqua-tests.sh +++ b/backend/run-aqua-tests.sh @@ -39,7 +39,7 @@ esac echo $urbit_bin_url -echo "Running backend unit tests" +echo "Running aqua tests" #download_url=`jq -r ".[\"$ship\"][\"downloadUrl\"]" < $ship_manifest` download_url="https://bootstrap.urbit.org/zod-aqua-tests-409k.xst" @@ -205,8 +205,8 @@ desk_hash_b=`echo $result | sed 's/\[0 %avow 0 %noun \(.*\)\]/\1/'` if [ $desk_hash_a == $desk_hash_b ] then echo "Desk upgrade failed ❌" - kill -TERM $vere_pid - exit 1 + # kill -TERM $vere_pid + # exit 1 fi echo "Starting aqua..." @@ -224,10 +224,40 @@ ${run_click} $pier "/lib/pill/hoon"<(%ok)) EOF -echo "Preparing aqua snapshot" -# The snap thread needs to accept id argument. +echo "Preparing aqua snapshot..." +result=$( $run_click -t 1200 $pier <(\`start-args:spider\`[\`tid.bowl \`tid byk.bowl(q %groups) %ph-fleet !>(\`args)]) +;< ~ bind:m (watch-our /awaiting/[tid] %spider /thread-result/[tid]) +;< ~ bind:m (poke-our %spider %spider-start poke-vase) +;< =cage bind:m (take-fact /awaiting/[tid]) +;< ~ bind:m (take-kick /awaiting/[tid]) +=/ thread-result=(each vase [term tang]) + ?+ p.cage ~|([%strange-thread-result p.cage %ph-test tid] !!) + %thread-done [%& q.cage] + %thread-fail [%| !<([term tang] q.cage)] + == +?: ?=(%| -.thread-result) + %- (slog %thread-fail p.thread-result) + (pure:m !>(|)) +(pure:m !>(&)) +EOF +) + +result_code=`echo $result | sed 's/\[0 %avow 0 %noun \(.*\)\]/\1/'` + +if [[ $result_code != "0" ]] +then + echo "Failed to generate aqua snapshot ❌" + kill -TERM $vere_pid + exit 1 +fi -# Run the unit tests +# Run aqua tests # # Update to use the generated test snapshot echo "Running tests..." @@ -236,8 +266,10 @@ result=$( $run_click -t 1200 $pier <(\`start-args:spider\`[\`tid.bowl \`tid byk.bowl(q %groups) %ph-test !>(\`ph-tests)]) +=/ poke-vase !>(\`start-args:spider\`[\`tid.bowl \`tid byk.bowl(q %groups) %ph-test !>(\`args)]) ;< ~ bind:m (watch-our /awaiting/[tid] %spider /thread-result/[tid]) ;< ~ bind:m (poke-our %spider %spider-start poke-vase) ;< =cage bind:m (take-fact /awaiting/[tid]) diff --git a/docs/aqua/development.md b/docs/aqua/development.md index 43da9b7105..e28b97a316 100644 --- a/docs/aqua/development.md +++ b/docs/aqua/development.md @@ -88,6 +88,11 @@ Here is an example test arm for the group join process: ::... ``` +Important: when naming arms anywhere in the outer core, avoid names +starting with the `test` prefix. This would cause the `-test` unit test +runner to attempt to resolve the arm as a unit test, which would break +the unit test suite. + ## Comment style guide Following a long maritime tradition, Urbit ships generally use feminine pronouns. diff --git a/docs/aqua/introduction.md b/docs/aqua/introduction.md index 4b3aaac2f9..94057f3b7f 100644 --- a/docs/aqua/introduction.md +++ b/docs/aqua/introduction.md @@ -70,6 +70,7 @@ You can save it to a dojo variable with ``` > =snap-id -desk!ph-fleet fleet sync ``` +to avoid having to retype each time tests are run. In addition to galaxy test ships, a test may require a number of _provider_ ships to be running, to allow components of the system to properly integrate @@ -123,6 +124,8 @@ The current virtual ames driver is inefficient. This make a single test run take about `~20s`, or more, depending on the number of networked interactions between virtual ships. +As a general note, arguments for threads in dojo can be listed one after another. + ### Understanding test output While tests run, information will be displayed from a variety of sources. The test runner will display test result and time taken after run of each test case. From 2069efb69c5ff4f5ab3b08a9bcc4d49d3c022e0f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Mon, 11 May 2026 21:10:35 +0800 Subject: [PATCH 12/18] Aqua tests: minor fixes --- desk/lib/verb.hoon | 8 ++++---- desk/ted/ph/test.hoon | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/desk/lib/verb.hoon b/desk/lib/verb.hoon index 045fd13ed0..1b003de976 100644 --- a/desk/lib/verb.hoon +++ b/desk/lib/verb.hoon @@ -133,7 +133,7 @@ same :: ++ print-logs - |= =cage + |= [=bowl:gall =cage] |* etc=* ?. ?=(%log-action p.cage) etc =+ !<(=a-log:logs q.cage) @@ -152,10 +152,10 @@ =/ =echo:logs ?- -.event %fail - [leaf+"[/] fail {}" trace.event] + [leaf+"[{}/{}] fail {}" trace.event] :: %tell - [leaf+"[/]" echo.event] + [leaf+"[{}/{}]" echo.event] == %- %- %*(. slog pri val) echo @@ -183,7 +183,7 @@ :: intercept logging cards and print :: %- ?: &(=(%logs name) =(ship our.bowl)) - (print-logs cage) + (print-logs bowl cage) same [%poke p.card [ship name] p.cage `@`(mug q.q.cage)] :: diff --git a/desk/ted/ph/test.hoon b/desk/ted/ph/test.hoon index be70ded948..feeb0e834e 100644 --- a/desk/ted/ph/test.hoon +++ b/desk/ted/ph/test.hoon @@ -163,7 +163,7 @@ =+ num=(lent tests) ?: =(num 0) ~> %slog.2^leaf+"No suitable aqua tests found" - (pure:m !>(&)) + (pure:m !>(|)) ~> %slog.1^leaf+"{} test {?:((gth num 1) "threads" "thread")} built" ~> %slog.1^'Running tests...' =/ n (strand (list (pair path thread-result))) From eedc54c2fa44409b240b9663244db0fe213ee230 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Tue, 12 May 2026 18:31:29 +0800 Subject: [PATCH 13/18] Aqua tests: allow agents to cool down It takes about a minutes to establish an ames flow between ships who had no previous contact. When creating a snapshot, we allow agents both to establish ames flows as well as to process events generated during desk reloading. --- desk/lib/ph/test.hoon | 1 + desk/ted/ph/fleet.hoon | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/desk/lib/ph/test.hoon b/desk/lib/ph/test.hoon index e22b914d2d..cf7d829892 100644 --- a/desk/lib/ph/test.hoon +++ b/desk/lib/ph/test.hoon @@ -1,6 +1,7 @@ /- spider /+ *strandio, *ph-io, test =, strand=strand:spider +=+ timeout=~s15 |% :: +ph-test-init: setup test strand environment :: diff --git a/desk/ted/ph/fleet.hoon b/desk/ted/ph/fleet.hoon index b04974ed70..129463e49e 100644 --- a/desk/ted/ph/fleet.hoon +++ b/desk/ted/ph/fleet.hoon @@ -80,6 +80,11 @@ ?~ fleet (pure:n ~) ;< ~ bind:n (sync-desk i.fleet %groups) $(fleet t.fleet) +:: allow agents time to cool down. it takes about a minute +:: for ames connections to be established. +:: +~> %slog.1^(crip "Cooling down %groups agents...") +;< ~ bind:m (sleep ~m1) ;< =bowl:spider bind:m get-bowl =/ snap-id=@t ?~ snap-id From aeaa1a2a536beefb68cbc01b4bf3e708deb65c1c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Wed, 13 May 2026 14:12:53 +0800 Subject: [PATCH 14/18] expose: don't inject empty citations The expose agent updates user's citations in the contacts profile on each reload. This causes non-deterministic behavior, where the profile after a restart depends on whether expose was updated. This is especially a problem in aqua tests. We fix this by preventing expose from generating updates when there are no citations. --- desk/app/expose.hoon | 2 ++ 1 file changed, 2 insertions(+) diff --git a/desk/app/expose.hoon b/desk/app/expose.hoon index 3c55f6b7e8..870b834ecb 100644 --- a/desk/app/expose.hoon +++ b/desk/app/expose.hoon @@ -12,6 +12,7 @@ /= page /app/expose/page /= widget /app/expose/widget :: +:: |% +$ state-2 $: %2 @@ -104,6 +105,7 @@ ^- (list [term value:co]) :: then look at our state and inject as appropriate :: + ?: =(~ open) ~ :_ ~ :- %expose-cites :- %set From 80786ec83589312e6beae1c8a084adfa8b28c987 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Wed, 13 May 2026 14:59:37 +0800 Subject: [PATCH 15/18] Aqua tests: various fixes --- backend/run-aqua-tests.sh | 4 ++-- desk/ted/ph/fleet.hoon | 1 + desk/ted/ph/test-ls.hoon | 2 ++ docs/aqua/introduction.md | 47 +++++++++++++++++++-------------------- 4 files changed, 28 insertions(+), 26 deletions(-) diff --git a/backend/run-aqua-tests.sh b/backend/run-aqua-tests.sh index 251ae71cd3..dc17a4d86d 100755 --- a/backend/run-aqua-tests.sh +++ b/backend/run-aqua-tests.sh @@ -205,8 +205,8 @@ desk_hash_b=`echo $result | sed 's/\[0 %avow 0 %noun \(.*\)\]/\1/'` if [ $desk_hash_a == $desk_hash_b ] then echo "Desk upgrade failed ❌" - # kill -TERM $vere_pid - # exit 1 + kill -TERM $vere_pid + exit 1 fi echo "Starting aqua..." diff --git a/desk/ted/ph/fleet.hoon b/desk/ted/ph/fleet.hoon index 129463e49e..5594b7a653 100644 --- a/desk/ted/ph/fleet.hoon +++ b/desk/ted/ph/fleet.hoon @@ -85,6 +85,7 @@ :: ~> %slog.1^(crip "Cooling down %groups agents...") ;< ~ bind:m (sleep ~m1) +;< ~ bind:m (end-test:ph-io vane-tids) ;< =bowl:spider bind:m get-bowl =/ snap-id=@t ?~ snap-id diff --git a/desk/ted/ph/test-ls.hoon b/desk/ted/ph/test-ls.hoon index 27e838bdf4..206dc29b01 100644 --- a/desk/ted/ph/test-ls.hoon +++ b/desk/ted/ph/test-ls.hoon @@ -135,6 +135,8 @@ =((cut 3 [0 len] name.test-arm) u.arm-pat) |- ?~ test-sets + ?. ?=(~ failed-builds) + (pure:m !>(|)) (pure:m !>(&)) =* path -.i.test-sets =* test-arms +.i.test-sets diff --git a/docs/aqua/introduction.md b/docs/aqua/introduction.md index 94057f3b7f..91d61ae44b 100644 --- a/docs/aqua/introduction.md +++ b/docs/aqua/introduction.md @@ -103,6 +103,29 @@ Important: do not delete files not present in the repository at the destination. Running urbit desks are usually supplemented with other neccessary files not present in the repository. +## Checking tests + +While developing tests, it is useful to have a way to verify any compilation errors +separately before triggering the actual test run. This can be done using +`-ph-test-ls path` command, which will find all aqua tests available at the +path and build them. For example, to list all aqua files in the groups desk, we would run +``` +> -groups!ph-test-ls /=groups=/tests/ph +``` +, and to list tests for a particular agent we would use +``` +> -groups!ph-test-ls /=groups=/tests/ph/app/contacts +``` + +There are two kind of errors that can occur at compilation time. +A `FAILED BUILD` error indicates the test file did not build. +The compiler error is shown directly above, because it is coming from clay. +A `FAILED MINT` error indicates that a test arm in a successfully compiled test +file does not resolve properly. Usually this indicates that the arm resolved +to a type that does not match the test signature, which should be a form of the strand `(strand ,~)`. +The possible compiler error is also displayed right the error line, for consistency +with file build errors. + ## Running tests To run aqua tests use the `-ph-test` test runner supplied with the desk. @@ -111,7 +134,6 @@ To run aqua tests use the `-ph-test` test runner supplied with the desk. ``` The `path` is a directory containing aqua tests (remember to include the backtick). Supply `~` to run from the default directory. It will be scanned recursively, and any tests found will be build and scheduled to be run. - `snap` is the aqua snapshot on which tests are going to run. The standard location for aqua tests is in the desk's `/tests/ph` directory. @@ -157,29 +179,6 @@ the location indicated in the message. In the above example, that would be the `+se-u-groups` arm somewhere in the agent's source code or its libraries. If that does not yield results, searching for constant parts of the messsage in relevant files usually. -## Checking tests - -While developing tests, it is useful to have a way to verify any compilation errors -separately before triggering the actual test run. This can be done using -`-ph-test-ls path` command, which will find all aqua tests available at the -path and build them. For example, to list all aqua files in the groups desk, we would run -``` -> -groups!ph-test-ls /=groups=/tests/ph -``` -, and to list tests for a particular agent we would use -``` -> -groups!ph-test-ls /=groups=/tests/ph/app/contacts -``` - -There are two kind of errors that can occur at compilation time. -A `FAILED BUILD` error indicates the test file did not build. -The compiler error is shown directly above, because it is coming from clay. -A `FAILED MINT` error indicates that a test arm in a successfully compiled test -file does not resolve properly. Usually this indicates that the arm resolved -to a type that does not match the test signature, which should be a form of the strand `(strand ,~)`. -The possible compiler error is also displayed right the error line, for consistency -with file build errors. - ## Cancelling a test run The test runner is a thread, so in order to stop an ongoing test run you must From d6b5bc0f8389b396c5ab4fead932db9e362ab3ae Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Fri, 15 May 2026 13:40:00 +0800 Subject: [PATCH 16/18] Aqua tests: misc - Do not hardcode thread ids in the aqua test runner - Do not hardcode any ships in the `ph-fleet` thread - Minor refactoring --- backend/run-aqua-tests.sh | 8 ++++---- desk/lib/ph/test.hoon | 1 - desk/ted/ph/fleet.hoon | 10 ++-------- desk/ted/ph/test.hoon | 6 ++---- 4 files changed, 8 insertions(+), 17 deletions(-) diff --git a/backend/run-aqua-tests.sh b/backend/run-aqua-tests.sh index dc17a4d86d..844e5accbc 100755 --- a/backend/run-aqua-tests.sh +++ b/backend/run-aqua-tests.sh @@ -228,9 +228,9 @@ echo "Preparing aqua snapshot..." result=$( $run_click -t 1200 $pier <(\`start-args:spider\`[\`tid.bowl \`tid byk.bowl(q %groups) %ph-fleet !>(\`args)]) ;< ~ bind:m (watch-our /awaiting/[tid] %spider /thread-result/[tid]) ;< ~ bind:m (poke-our %spider %spider-start poke-vase) @@ -266,9 +266,9 @@ result=$( $run_click -t 1200 $pier <(\`start-args:spider\`[\`tid.bowl \`tid byk.bowl(q %groups) %ph-test !>(\`args)]) ;< ~ bind:m (watch-our /awaiting/[tid] %spider /thread-result/[tid]) ;< ~ bind:m (poke-our %spider %spider-start poke-vase) diff --git a/desk/lib/ph/test.hoon b/desk/lib/ph/test.hoon index cf7d829892..e22b914d2d 100644 --- a/desk/lib/ph/test.hoon +++ b/desk/lib/ph/test.hoon @@ -1,7 +1,6 @@ /- spider /+ *strandio, *ph-io, test =, strand=strand:spider -=+ timeout=~s15 |% :: +ph-test-init: setup test strand environment :: diff --git a/desk/ted/ph/fleet.hoon b/desk/ted/ph/fleet.hoon index 5594b7a653..661bf5d624 100644 --- a/desk/ted/ph/fleet.hoon +++ b/desk/ted/ph/fleet.hoon @@ -54,15 +54,9 @@ =/ [snap-id=(unit @t) fleet=(list ship) sync=?] ?~ args ~|(%no-args-found !!) [snap-id fleet sync]:u.args -=. fleet - ^- (list ship) - :* ~loshut-lonreg :: bait provider - ~rivfur-livmet :: notify provider - ~dem :: bait provider galaxy - ~fen :: notify provider galaxy - fleet - == ;< =bowl:spider bind:m get-bowl +:: remove duplicates +=. fleet ~(tap in (silt fleet)) ~> %slog.1^(crip "Booting fleet {}") ;< vane-tids=(map term tid:spider) bind:m start-simple:ph-io ;< ~ bind:m diff --git a/desk/ted/ph/test.hoon b/desk/ted/ph/test.hoon index feeb0e834e..6e4dd308a8 100644 --- a/desk/ted/ph/test.hoon +++ b/desk/ted/ph/test.hoon @@ -115,10 +115,8 @@ |= args=vase =/ m (strand ,vase) ^- form:m -=+ !<(args=(unit [pax=(unit path) snap=@t]) args) -=/ [pax=(unit path) snap=@] - ?~ args ~|(%no-args-found !!) - [pax.u.args snap.u.args] +=+ ~| %no-args-found + (need !<((unit [pax=(unit path) snap=@t]) args)) ;< =bowl:spider bind:m get-bowl =/ [byk=beak pax=path] ?~ pax From d4f8c73e0dabd9d928cbd1ec849963d371725aa1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Fri, 15 May 2026 14:37:32 +0800 Subject: [PATCH 17/18] Aqua tests: fix thread ids in the runner --- backend/run-aqua-tests.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/backend/run-aqua-tests.sh b/backend/run-aqua-tests.sh index 844e5accbc..2ed1a5fa33 100755 --- a/backend/run-aqua-tests.sh +++ b/backend/run-aqua-tests.sh @@ -228,7 +228,7 @@ echo "Preparing aqua snapshot..." result=$( $run_click -t 1200 $pier <(\`start-args:spider\`[\`tid.bowl \`tid byk.bowl(q %groups) %ph-fleet !>(\`args)]) @@ -268,7 +268,7 @@ result=$( $run_click -t 1200 $pier <(\`start-args:spider\`[\`tid.bowl \`tid byk.bowl(q %groups) %ph-test !>(\`args)]) ;< ~ bind:m (watch-our /awaiting/[tid] %spider /thread-result/[tid]) ;< ~ bind:m (poke-our %spider %spider-start poke-vase) From 31eb6d434aa8fac85d712334c1c8145955a28d3d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Miko=C5=82aj=20Paraniak?= Date: Fri, 15 May 2026 21:29:55 +0800 Subject: [PATCH 18/18] Aqua tests: fix typo in test runner --- backend/run-aqua-tests.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/backend/run-aqua-tests.sh b/backend/run-aqua-tests.sh index 2ed1a5fa33..9712230a9c 100755 --- a/backend/run-aqua-tests.sh +++ b/backend/run-aqua-tests.sh @@ -268,7 +268,7 @@ result=$( $run_click -t 1200 $pier <(\`start-args:spider\`[\`tid.bowl \`tid byk.bowl(q %groups) %ph-test !>(\`args)]) ;< ~ bind:m (watch-our /awaiting/[tid] %spider /thread-result/[tid]) ;< ~ bind:m (poke-our %spider %spider-start poke-vase)