Kompari xtask (html report, interactive test blessing, dead snapshots detection) #272
Conversation
spirali
force-pushed
the
kompari-xtask
branch
from
2b99f07 to
78a3551
Compare
There was a problem hiding this comment.
Very cool! I made some tests intentionally fail subtly, and I think this will definitely help with e.g. spotting when there are actual issues vs just rounding differences.
Some minor nits inline, and I think you intended to commit .cargo/config.toml or something similar, I added a comment about that on parley/tests/README.md.
| @@ -20,15 +20,15 @@ checksum = "c71b1793ee61086797f5c80b6efa2b8ffa6d5dd703f118545808a7f2e27f7046" | |||
|
|
|||
| [[package]] | |||
| name = "accesskit" | |||
| version = "0.17.0" | |||
| version = "0.17.1" | |||
There was a problem hiding this comment.
I believe according to https://github.com/linebender/rfcs/blob/main/rfcs/0005-version-matrix.md#cargotoml-matches-cargolock, Cargo.toml should be updated to match these patch versions.
There was a problem hiding this comment.
It should be fixed now by running cargo upgrade
There was a problem hiding this comment.
I'm not sure, for example Cargo.toml still lists accesskit = "0.17", but it's locked to "0.17.1" in the lockfile.
Maybe @DJMcNab can chime in?
There was a problem hiding this comment.
We should always specify the patch version in Cargo.toml, yes. So we never should have a dependency of the form accesskit = "0.17".
That's largely a thing for consistency, but it does have advantages for e.g. when we publish, largely in case any user decides to use the MSRV-aware resolver or minimal-versions.
| let args = Args::parse(); | ||
| let mut diff_config = DirDiffConfig::new(snapshots_path, current_path); | ||
| diff_config.set_ignore_right_missing(true); | ||
| let actions = ActionsImpl(); | ||
| let mut task = Task::new(diff_config, Box::new(actions)); | ||
| task.run(&args)?; |
There was a problem hiding this comment.
Would it make sense to move more of this into kompari_tasks? If more repositories use Kompari, they'd all duplicate this code.
There was a problem hiding this comment.
This was the original plan, but I still want to have a relatively high level of configurability, as it seems that the Vello testing framework works a bit differently than what I implemented for Parley.
| @@ -20,15 +20,15 @@ checksum = "c71b1793ee61086797f5c80b6efa2b8ffa6d5dd703f118545808a7f2e27f7046" | |||
|
|
|||
| [[package]] | |||
| name = "accesskit" | |||
| version = "0.17.0" | |||
| version = "0.17.1" | |||
There was a problem hiding this comment.
I'm not sure, for example Cargo.toml still lists accesskit = "0.17", but it's locked to "0.17.1" in the lockfile.
Maybe @DJMcNab can chime in?
|
Kompari updated. It adds |
Adds command
cargo xtaskthat allows to:Creating report
creates HTML file with report containing snapshots that does not match
Interactive test blessing:
stars a local web server with the following page:
(screenshot obtained by putting an artificial error into each snapshot test)
Detecting dead snapshots
This PR fully replaces #185