-
Notifications
You must be signed in to change notification settings - Fork 5
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
22 changed files
with
736 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,122 @@ | ||
# yunit | ||
|
||
A data driven testing tool for .NET using markdown and YAML | ||
`yunit` is a data driven testing tool for .NET using markdown and YAML. | ||
|
||
Data driven tests separate your test data from test logic, making it easy to add, modify and review test cases. | ||
|
||
Markdown and YAML are popular formats designed to be writable and readable by humans, using these formats makes testing more enjoyable. | ||
|
||
 | ||
|
||
## Getting Started | ||
|
||
We have some [samples](samples) to demonstrate the usage of `yunit`, or you can follow these steps to scaffold a new test project: | ||
|
||
1. Create a new class library | ||
|
||
```cmd | ||
dotnet new console | ||
``` | ||
|
||
2. Add NuGet reference to `yunit` and `Microsoft.NET.Test.Sdk` | ||
|
||
```cmd | ||
dotnet add package yunit | ||
dotnet add package Microsoft.NET.Test.Sdk | ||
``` | ||
|
||
3. Create a serializable data contract to describe your test | ||
|
||
```csharp | ||
public class HelloTestSpec | ||
{ | ||
public string Input; | ||
public string Output; | ||
} | ||
``` | ||
|
||
4. Create a public method that takes the data contract with the corresponding verification code: | ||
|
||
- Add `[YamlTest]` attribute to include test cases written in [YAML format](#writing-yaml-tests) | ||
- Add `[MarkdownTest]` attribute to include test cases written in [markdown format](#writing-markdown-tests) | ||
|
||
> 💡 The glob pattern is relative to the current working directory, which could very based on current build configuration and target platform. Use `~/` to denote path relative to the current git repository. | ||
```csharp | ||
public class HelloTest | ||
{ | ||
[YamlTest("~/*.yml")] | ||
[MarkdownTest("~/README.md")] | ||
public void Hello(HelloTestSpec spec) | ||
{ | ||
Assert.Equal(spec.Output, $"Hello {spec.Input}"); | ||
} | ||
} | ||
|
||
``` | ||
|
||
5. Run your tests using `dotnet test` or Visual Studio Test Explorer. All test cases run in parallel. | ||
|
||
## Writing YAML Tests | ||
|
||
A YAML file can contain multiple YAML documents separated by `---`. Each YAML document is a test case. | ||
|
||
The first line of comment appears as test description in Visual Studio test explorer. | ||
|
||
```yml | ||
# Hello YAML test! | ||
input: YAML test! | ||
output: Hello YAML test! | ||
--- | ||
# Hello another YAML test | ||
input: Another YAML test | ||
output: Hello Another YAML test | ||
``` | ||
## Writing Markdown Tests | ||
A markdown test is a specialized YAML code block in an ordinary markdown file. A normal code block starts with 3 backticks (` ``` `). A test code block starts with _at least 6_ backticks (` `````` `). | ||
|
||
``````````markdown | ||
# This is an ordinary markdown file | ||
|
||
Using markdown tests, you can enhance your docs with real examples that are correct and up to date. | ||
|
||
``````yml | ||
input: Markdown Test! | ||
output: Hello Markdown Test! | ||
`````` | ||
`````````` | ||
|
||
You can further customize how tests are extracted from markdown files in the `[MarkdownTest]` attributes. | ||
|
||
## Use `json-diff` for data driven assertion | ||
|
||
You can use _any_ assertion library to verify your test expectations. | ||
|
||
`json-diff` is a built-in tool to write assertions in YAML that fits the `yunit` testing style. | ||
|
||
```csharp | ||
var jsonDiff = new JsonDiffBuilder().Build(); | ||
jsonDiff.Verify(expected, actual, "Test failed"); | ||
``` | ||
|
||
The idea of `json-dff` is to serialize your expected object and actual object into JSON and compare them using a diff algorithm. | ||
|
||
|
||
 | ||
|
||
The bare metal comparison expect the actual object to match exactly the same as the expected object, but you can custimize the `json-diff` pipeline to transform the expected and actual object before doing the text comparison. The built in `json-diff` pipeline comes with several handy extensions: | ||
|
||
|
||
method | description | example | ||
-------|-----------------------------------------------|------------- | ||
`UseIgnoreNull` | Ignore the actual result of a property if the expected value is null | `{ "a": null }` matches `{”a": xxx}` where `xxx` could be anything | ||
`UseNegate` | Assert the actual result must not be the expected result if the expected value starts with `!` | Given an expectation of `{ "a": "!value"}` , `{"a": "value"} should fail | ||
`UseRegex` | The value must match a regex if the expectation looks like `/{regex}/` | `{"a": "/^.*$/" }` uses `^.*$` to match the actual value | ||
`UseWildcard` | Use wildcard match if the expectation contains `*` | `{"a": "a*" }` uses wildcard `a*` to match against actual value | ||
`UseJson` | The value must be a JSON string and its value is verified using the current pipeline, all the above rules will apply if they are configured in the pipeline | | ||
`UseHtml` | The value must be an HTML string and its value is normalized before verification | | ||
|
||
`json-diff` allows you to write stateful data driven tests. The [Shopping Cart Web API sample](./samples) demonstrates a custom `json-diff` extension that preserves states between multiple web API calls for a typical web API testing scenario | ||
. |
This file was deleted.
Oops, something went wrong.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
<Project Sdk="Microsoft.NET.Sdk"> | ||
|
||
<PropertyGroup> | ||
<TargetFramework>netcoreapp3.0</TargetFramework> | ||
</PropertyGroup> | ||
|
||
<ItemGroup> | ||
<PackageReference Include="yunit" Version="1.0.0-*" /> | ||
<PackageReference Include="xunit" Version="2.4.1" /> | ||
</ItemGroup> | ||
|
||
</Project> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
using Microsoft.DocAsTest; | ||
using Xunit; | ||
|
||
public class HelloTestSpec | ||
{ | ||
public string Input; | ||
public string Output; | ||
} | ||
|
||
public class HelloTest | ||
{ | ||
[YamlTest("~/1-hello/**/*.yml")] | ||
[MarkdownTest("~/1-hello/**/*.md", FenceTip="test")] | ||
public void Hello(HelloTestSpec spec) | ||
{ | ||
Assert.Equal(spec.Output, $"Hello {spec.Input}"); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
|
||
1. Create a new class library | ||
|
||
```cmd | ||
dotnet new console | ||
``` | ||
|
||
2. Add a pre-release NuGet feed to `NuGet.config` | ||
|
||
```xml | ||
<?xml version="1.0" encoding="utf-8"?> | ||
<configuration> | ||
<packageSources> | ||
<add key="docfx-v3" value="https://www.myget.org/F/docfx-v3/api/v3/index.json" /> | ||
</packageSources> | ||
</configuration> | ||
``` | ||
|
||
3. Add NuGet reference to `Microsoft.DocAsTest` | ||
|
||
```cmd | ||
dotnet add package Microsoft.DocAsTest --version 3.0.0-* | ||
``` | ||
|
||
4. Create test logic | ||
|
||
```csharp | ||
public class HelloTestSpec | ||
{ | ||
public string Input; | ||
public string Output; | ||
} | ||
|
||
public class HelloTest | ||
{ | ||
[YamlTest("~/*.yml")] | ||
[MarkdownTest("~/README.md")] | ||
public void Hello(HelloTestSpec spec) | ||
{ | ||
Assert.Equal(spec.Output, $"Hello {spec.Input}"); | ||
} | ||
} | ||
|
||
``` | ||
|
||
5. Create YAML test data | ||
|
||
```yml | ||
# Hello YAML Test! <-- hello.yml | ||
input: YAML Test! | ||
output: Hello YAML Test! | ||
``` | ||
6. Create Markdown test data | ||
``````````markdown | ||
You can _also_ write test data in markdown files using YAML code blocks that starts with 6 or more backticks. | ||
``````test | ||
input: Markdown Test! | ||
output: Hello Markdown Test! | ||
`````` | ||
`````````` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
# Hello YAML Test! | ||
input: YAML Test! | ||
output: Hello YAML Test! | ||
--- | ||
# This is a failed test | ||
input: YAML Test | ||
output: | ||
--- | ||
# [skip] This is a skipped test | ||
input: Skipped test | ||
output: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
<Project Sdk="Microsoft.NET.Sdk"> | ||
|
||
<PropertyGroup> | ||
<TargetFramework>netcoreapp2.2</TargetFramework> | ||
</PropertyGroup> | ||
|
||
<ItemGroup> | ||
<PackageReference Include="yunit" Version="1.0.0-*" /> | ||
<PackageReference Include="xunit" Version="2.4.1" /> | ||
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.1" /> | ||
</ItemGroup> | ||
|
||
</Project> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,99 @@ | ||
using Microsoft.DocAsTest; | ||
using Xunit; | ||
|
||
public class ShoppingCart | ||
{ | ||
public ShoppingCartLine[] Lines { get; set; } | ||
} | ||
|
||
public class ShoppingCartLine | ||
{ | ||
public string Name { get; set; } | ||
|
||
public string[] Tags { get; set; } | ||
|
||
public int Quatity { get; set; } | ||
} | ||
|
||
public class JsonDiffTest | ||
{ | ||
private readonly ShoppingCart _expectedCart = new ShoppingCart | ||
{ | ||
Lines = new ShoppingCartLine[] | ||
{ | ||
new ShoppingCartLine | ||
{ | ||
Name = "Transformers: The Last Knight Bumblebee Voice Changer Mask", | ||
Tags = new[] | ||
{ | ||
"4.6 x 8.5 x 11 inches", | ||
"1.1 pounds", | ||
"B01N1834JJ", | ||
"C1324", | ||
"5 years and up", | ||
"3 AA batteries required. (included)", | ||
}, | ||
Quatity = 1, | ||
} | ||
} | ||
}; | ||
|
||
private readonly ShoppingCart _actualCart = new ShoppingCart | ||
{ | ||
Lines = new ShoppingCartLine[] | ||
{ | ||
new ShoppingCartLine | ||
{ | ||
Name = "Transformers: The Last Knight Bumblebee Voice Changer Mask", | ||
Tags = new[] | ||
{ | ||
"4.6 x 8.5 x 11 inches", | ||
"1.1 pounds", | ||
"B01N1834JJ", | ||
"C1324", | ||
"5 years and up", | ||
"2 AA batteries required. (included)", | ||
}, | ||
Quatity = 1, | ||
} | ||
}, | ||
}; | ||
|
||
[Fact] | ||
public void XunitEquals() | ||
{ | ||
Assert.Equal(_expectedCart.Lines.Length, _actualCart.Lines.Length); | ||
|
||
for (var i = 0; i < _expectedCart.Lines.Length; i++) | ||
{ | ||
Assert.Equal(_expectedCart.Lines[i].Name, _actualCart.Lines[i].Name); | ||
Assert.Equal(_expectedCart.Lines[i].Tags, _actualCart.Lines[i].Tags); | ||
Assert.Equal(_expectedCart.Lines[i].Quatity, _actualCart.Lines[i].Quatity); | ||
} | ||
} | ||
|
||
[Fact] | ||
public void JsonDiffEquals() | ||
{ | ||
new JsonDiff().Verify(_expectedCart, _actualCart); | ||
} | ||
|
||
[Fact] | ||
public void JsonDiffPipelineEquals() | ||
{ | ||
var jsonDiff = new JsonDiffBuilder() | ||
.UseAdditionalProperties() | ||
.UseWildcard() | ||
.Build(); | ||
|
||
// Allow additional properties | ||
jsonDiff.Verify( | ||
expected: new { foo = "bar" }, | ||
actual: new { foo = "bar", additionalProperty = "baz" }); | ||
|
||
// Allow wildcard match | ||
jsonDiff.Verify( | ||
expected: new { foo = "this * sentence" }, | ||
actual: new { foo = "this is a whole sentence" }); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
<Project Sdk="Microsoft.NET.Sdk"> | ||
|
||
<PropertyGroup> | ||
<TargetFramework>netcoreapp3.0</TargetFramework> | ||
</PropertyGroup> | ||
|
||
<ItemGroup> | ||
<PackageReference Include="Microsoft.AspNetCore.App" /> | ||
<PackageReference Include="Microsoft.AspNetCore.TestHost" Version="2.2.0" /> | ||
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.3.0" /> | ||
<PackageReference Include="yunit" Version="1.0.0-*" /> | ||
</ItemGroup> | ||
|
||
<ItemGroup> | ||
<ProjectReference Include="..\3-aspnetcore\3-aspnetcore.csproj" /> | ||
</ItemGroup> | ||
|
||
</Project> |
Oops, something went wrong.