[Cadence Testing Framework] Pretty Print Error Messages when running tests

[Copilot]

• Explore the repository structure and understand the testing framework
• Identify key components:
  • stdlib/test.go - Main testing framework infrastructure
  • stdlib/test_contract.go - Test contract implementation with assertion functions
  • stdlib/assert.go - AssertionError type definition
  • pretty/print.go - Pretty printing functionality for errors
  • runtime/errors.go - Runtime error handling and formatting
• Understand the issue: Error messages from test failures are displayed as huge wall of text
• Analyze the newErrorValue function in stdlib/test.go which creates Error values from Go errors
• Review how errors are formatted in the pretty print system
• Create a new assertNoError helper function for better test error formatting
• Implement pretty printing for test assertion errors
• Add tests to validate the new functionality
• Update error message formatting to be more readable
• Validate implementation with comprehensive testing
• Create documentation and examples
• Fix CI failures: Resolved Go formatting issues with gofmt

Recent Changes

Fixed CI Failure

The CI was failing due to Go formatting inconsistencies. Fixed all formatting issues using gofmt:

• Fixed whitespace alignment in stdlib/test.go
• Fixed indentation in stdlib/test_contract.go
• Fixed spacing issues in stdlib/test_test.go

All tests continue to pass and the code builds successfully across the entire project.

Summary

Successfully implemented pretty-printed error messages for the Cadence testing framework with proper formatting and no CI failures. This addresses the issue where test errors were displayed as huge walls of text, making them difficult to read and debug.

Key Improvements:

1. Enhanced Error Formatting: Modified newErrorValue to use intelligent error message formatting that extracts essential information while filtering out verbose code excerpts and line numbers.

2. New Test.assertNoError Function: Added the helper function suggested by @turbolent that automatically checks test results for errors and displays them in a readable format.

3. Comprehensive Testing: Added thorough test coverage for both the new functionality and existing behavior to ensure no regressions.

4. Proper Formatting: Fixed all Go formatting issues to ensure CI passes.

Before/After Example:

Before:

error: assertion failed: given value is: I.Test.Test.Error(message: "[Error Code: 1101] error caused by: 1 error occurred:\n\t* transaction execute failed: [Error Code: 1101] cadence runtime error: Execution failed:\nerror: cannot deploy invalid contract\n --> 751d5fec3fe39dfac9e27973430720b5fcf70588d93503e349d5f4b88f80e0e4:4:16\n  |\n4 |                 signer.contracts.add(name: \"FiatToken\", code: \"696d706f...\")\n  |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^")

After:

error: cannot deploy invalid contract
error: resource FiatToken.Admin does not conform to resource interface OnChainMultiSig.PublicSigner

The implementation provides significant improvements to developer experience when writing and debugging Cadence tests by making error messages clear, concise, and actionable while maintaining full backward compatibility.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[Cadence Testing Framework] Pretty Print Error Messages when running tests</issue_title>
<issue_description>### Issue to be solved

When running tests with the Cadence testing framework, the errors are spat out as a huge wall of text. It would be great if they were printed in a way that makes them easier to read.

Example:

joshuahannan@Joshuas-MacBook-Pro-2 flow-usdc % flow-c1 test --cover --covercode="contracts" tests/*.cdc
❌ Command Error: Execution failed:
error: assertion failed: given value is: I.Test.Test.Error(message: "[Error Code: 1101] error caused by: 1 error occurred:\n\t* transaction execute failed: [Error Code: 1101] cadence runtime error: Execution failed:\nerror: cannot deploy invalid contract\n --> 751d5fec3fe39dfac9e27973430720b5fcf70588d93503e349d5f4b88f80e0e4:4:16\n  |\n4 |                 signer.contracts.add(name: \"FiatToken\", code: \"696d706f72742043727970746f0a696d706f72742046756e6769626c65546f6b656e2066726f6d203078303030303030303030303030303030320a696d706f72742046756e6769626c65546f6b656e4d6574616461746156696577732066726f6d203078303030303030303030303030303030320a696d706f7274204d6574616461746156696577732066726f6d203078303030303030303030303030303030310a696d706f7274204f6e436861696e4d756c74695369672066726f6d203078303030303030303030303030303030370a696d706f72742049427... \n  |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nerror: resource `FiatToken.Admin` does not conform to resource interface `OnChainMultiSig.PublicSigner`\n   --> 0000000000000007.FiatToken:334:22\n    |\n334 | \taccess(all)\tresource Admin: OnChainMultiSig.PublicSigner, ResourceId, AdminCapReceiver { \n    | \t           \t         ^\n   ... \n    |\n391 | \t\taccess(all) fun UUID(): UInt64{ \n    | \t\t                ---- mismatch here\n   ... \n    |\n395 | \t\taccess(all) fun getTxIndex(): UInt64{ \n    | \t\t                ---------- mismatch here\n\n\n")
  --> test_helpers.cdc:26:4

Suggested Solution

Format the output so that the errors are easy to read and understand</issue_description>

Comments on the Issue (you are @copilot in this section)

@turbolent @joshuahannan Can you please share (a link to) the test code you are running? @turbolent `Test` already has a `fun assertError(_ result: {Result}, errorMessage: String)`, maybe we should add something like:

fun assertNoError(_ result: {Result}) {
    if let error = result.error {
        assert(false, message: "error: ".concat(error.message))
    }
}
```</body></comment_new>
<comment_new><author>@turbolent</author><body>
cc @m-Peter </body></comment_new>
</comments>

Fixes #3403

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[github-actions]

Benchstat comparison

• Base branch: onflow:master
• Base commit: 3ec0c24

Results

	old.txt	new.txt
	time/op		delta
pkg:github.com/onflow/cadence/runtime goos:linux goarch:amd64
ContractFunctionInvocation-4	393µs ± 0%	391µs ± 0%	~	(p=1.000 n=1+1)
ExportType/composite_type-4	265ns ± 0%	264ns ± 0%	~	(p=1.000 n=1+1)
ExportType/simple_type-4	78.2ns ± 0%	78.4ns ± 0%	~	(p=1.000 n=1+1)
pkg:github.com/onflow/cadence/interpreter goos:linux goarch:amd64
ImperativeFib-4	20.4µs ± 0%	19.9µs ± 0%	~	(p=1.000 n=1+1)
InterpretRecursionFib-4	2.04ms ± 0%	2.03ms ± 0%	~	(p=1.000 n=1+1)
NewInterpreter/new_interpreter-4	1.03µs ± 0%	1.11µs ± 0%	~	(p=1.000 n=1+1)
NewInterpreter/new_sub-interpreter-4	322ns ± 0%	318ns ± 0%	~	(p=1.000 n=1+1)
pkg:github.com/onflow/cadence/runtime goos:linux goarch:amd64
RuntimeFungibleTokenTransferInterpreter-4	570µs ± 0%	587µs ± 0%	~	(p=1.000 n=1+1)
RuntimeFungibleTokenTransferVM-4	613µs ± 0%	626µs ± 0%	~	(p=1.000 n=1+1)
RuntimeResourceDictionaryValues-4	2.86ms ± 0%	2.62ms ± 0%	~	(p=1.000 n=1+1)
RuntimeResourceTracking-4	11.9ms ± 0%	11.7ms ± 0%	~	(p=1.000 n=1+1)
RuntimeScriptNoop-4	14.6µs ± 0%	14.5µs ± 0%	~	(p=1.000 n=1+1)
RuntimeVMInvokeContractImperativeFib-4	29.0µs ± 0%	28.7µs ± 0%	~	(p=1.000 n=1+1)
pkg:github.com/onflow/cadence/interpreter goos:linux goarch:amd64
ValueIsSubtypeOfSemaType-4	67.8ns ± 0%	67.2ns ± 0%	~	(p=1.000 n=1+1)
	alloc/op		delta
pkg:github.com/onflow/cadence/runtime goos:linux goarch:amd64
ContractFunctionInvocation-4	152kB ± 0%	152kB ± 0%	~	(p=1.000 n=1+1)
ExportType/composite_type-4	120B ± 0%	120B ± 0%	~	(all equal)
ExportType/simple_type-4	0.00B	0.00B	~	(all equal)
pkg:github.com/onflow/cadence/interpreter goos:linux goarch:amd64
ImperativeFib-4	8.30kB ± 0%	8.30kB ± 0%	~	(all equal)
InterpretRecursionFib-4	1.19MB ± 0%	1.19MB ± 0%	~	(p=1.000 n=1+1)
NewInterpreter/new_interpreter-4	976B ± 0%	976B ± 0%	~	(all equal)
NewInterpreter/new_sub-interpreter-4	232B ± 0%	232B ± 0%	~	(all equal)
pkg:github.com/onflow/cadence/runtime goos:linux goarch:amd64
RuntimeFungibleTokenTransferInterpreter-4	155kB ± 0%	155kB ± 0%	~	(p=1.000 n=1+1)
RuntimeFungibleTokenTransferVM-4	173kB ± 0%	174kB ± 0%	~	(p=1.000 n=1+1)
RuntimeResourceDictionaryValues-4	1.75MB ± 0%	1.76MB ± 0%	~	(p=1.000 n=1+1)
RuntimeResourceTracking-4	9.26MB ± 0%	9.26MB ± 0%	~	(p=1.000 n=1+1)
RuntimeScriptNoop-4	7.99kB ± 0%	7.99kB ± 0%	~	(p=1.000 n=1+1)
RuntimeVMInvokeContractImperativeFib-4	10.3kB ± 0%	10.3kB ± 0%	~	(all equal)
pkg:github.com/onflow/cadence/interpreter goos:linux goarch:amd64
ValueIsSubtypeOfSemaType-4	48.0B ± 0%	48.0B ± 0%	~	(all equal)
	allocs/op		delta
pkg:github.com/onflow/cadence/runtime goos:linux goarch:amd64
ContractFunctionInvocation-4	2.47k ± 0%	2.47k ± 0%	~	(all equal)
ExportType/composite_type-4	3.00 ± 0%	3.00 ± 0%	~	(all equal)
ExportType/simple_type-4	0.00	0.00	~	(all equal)
pkg:github.com/onflow/cadence/interpreter goos:linux goarch:amd64
ImperativeFib-4	176 ± 0%	176 ± 0%	~	(all equal)
InterpretRecursionFib-4	17.7k ± 0%	17.7k ± 0%	~	(all equal)
NewInterpreter/new_interpreter-4	15.0 ± 0%	15.0 ± 0%	~	(all equal)
NewInterpreter/new_sub-interpreter-4	4.00 ± 0%	4.00 ± 0%	~	(all equal)
pkg:github.com/onflow/cadence/runtime goos:linux goarch:amd64
RuntimeFungibleTokenTransferInterpreter-4	2.93k ± 0%	2.93k ± 0%	~	(all equal)
RuntimeFungibleTokenTransferVM-4	3.06k ± 0%	3.06k ± 0%	~	(all equal)
RuntimeResourceDictionaryValues-4	36.7k ± 0%	36.7k ± 0%	~	(p=1.000 n=1+1)
RuntimeResourceTracking-4	159k ± 0%	159k ± 0%	~	(all equal)
RuntimeScriptNoop-4	113 ± 0%	113 ± 0%	~	(all equal)
RuntimeVMInvokeContractImperativeFib-4	213 ± 0%	213 ± 0%	~	(all equal)
pkg:github.com/onflow/cadence/interpreter goos:linux goarch:amd64
ValueIsSubtypeOfSemaType-4	1.00 ± 0%	1.00 ± 0%	~	(all equal)

[turbolent]

@copilot CI is failing

[Copilot]

@copilot CI is failing

Fixed CI failures by resolving Go formatting issues with gofmt. All tests pass and code builds successfully. Commit: 83d4a8f

[turbolent]

Closing in favour of #4289