Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Plan for adding more generators to tests of initOf #2345

Open
1 of 16 tasks
jeshecdom opened this issue Mar 10, 2025 · 0 comments
Open
1 of 16 tasks

Plan for adding more generators to tests of initOf #2345

jeshecdom opened this issue Mar 10, 2025 · 0 comments
Labels
kind: testing Tests (*.spec.ts)

Comments

@jeshecdom
Copy link
Contributor

jeshecdom commented Mar 10, 2025

This is a follow-up to PR #2083. The following features are required in such PR:

Generators for:

  • Contract method calls

  • Getter method calls

  • Extension global function method calls

  • Mutating global function method calls

  • Calls to methods in traits

  • Contract fields with default values

  • Contract constants

  • Global constants

  • Structs with default values

  • Arbitrary well-typed expressions with function calls that carry out send

  • Foreach statement

  • Internal receivers that carry out initOf and send

  • Bounced receivers that carry out initOf and send

  • External receivers that carry out initOf and send

  • Repeat all the above but for codeof

Currently, the PR explores all possible combinations of blocks of code where initOf could be placed up to a depth of two global function calls. In order words, it carries out an exhaustive search. The exploration space is already huge, which means that adding more generators would make it prohibitely impossible to explore, because the exploration space grows exponentially for each added generator (additionally, we compile each test and run it through Sandbox).

So, we need to add an strategy similar to what swarm testing does: before starting the test generation, create a random collection of "feature sets" and for each such feature set, generate tests with only features in that set. In our case, a feature is simply a generator, so we would do the following:

  • Randomly choose a number n to be the number of feature sets to create, probably we want this number to be at least 10.
  • for i in [1, n]:
    • Go through the list of all possible generators and for each generator randomly include or exclude the generator by throwing a coin.
      The set of chosen generators will be the feature set for i.

After we have our collection of feature sets, carry out the tests:

  • For each feature set F_i (this step could be done in parallel):
    • Carry out the exhaustive search using only generators in F_i.

Probably I will need to adapt the test so that instead of an exhaustive search on each F_i, it carries out a random search with deeper levels of function calls. This is not hard to do, because instead of chaining the generators as the PR currently does, we just need to use the oneof combinator.

  • Adapt the script to use the above idea from swarm testing
@jeshecdom jeshecdom added the kind: testing Tests (*.spec.ts) label Mar 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: testing Tests (*.spec.ts)
Projects
None yet
Development

No branches or pull requests

1 participant