Skip to content

Commit

Permalink
Update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
yufeih committed Nov 25, 2019
1 parent c95782f commit 8391102
Show file tree
Hide file tree
Showing 22 changed files with 736 additions and 2 deletions.
121 changes: 120 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,122 @@
# yunit

A data driven testing tool for .NET using markdown and YAML
`yunit` is a data driven testing tool for .NET using markdown and YAML.

Data driven tests separate your test data from test logic, making it easy to add, modify and review test cases.

Markdown and YAML are popular formats designed to be writable and readable by humans, using these formats makes testing more enjoyable.

![yunit in Visual Studio](./media/yunit-in-visual-studio.png)

## Getting Started

We have some [samples](samples) to demonstrate the usage of `yunit`, or you can follow these steps to scaffold a new test project:

1. Create a new class library

```cmd
dotnet new console
```

2. Add NuGet reference to `yunit` and `Microsoft.NET.Test.Sdk`

```cmd
dotnet add package yunit
dotnet add package Microsoft.NET.Test.Sdk
```

3. Create a serializable data contract to describe your test

```csharp
public class HelloTestSpec
{
public string Input;
public string Output;
}
```

4. Create a public method that takes the data contract with the corresponding verification code:

- Add `[YamlTest]` attribute to include test cases written in [YAML format](#writing-yaml-tests)
- Add `[MarkdownTest]` attribute to include test cases written in [markdown format](#writing-markdown-tests)

> 💡 The glob pattern is relative to the current working directory, which could very based on current build configuration and target platform. Use `~/` to denote path relative to the current git repository.
```csharp
public class HelloTest
{
[YamlTest("~/*.yml")]
[MarkdownTest("~/README.md")]
public void Hello(HelloTestSpec spec)
{
Assert.Equal(spec.Output, $"Hello {spec.Input}");
}
}

```

5. Run your tests using `dotnet test` or Visual Studio Test Explorer. All test cases run in parallel.

## Writing YAML Tests

A YAML file can contain multiple YAML documents separated by `---`. Each YAML document is a test case.

The first line of comment appears as test description in Visual Studio test explorer.

```yml
# Hello YAML test!
input: YAML test!
output: Hello YAML test!
---
# Hello another YAML test
input: Another YAML test
output: Hello Another YAML test
```
## Writing Markdown Tests
A markdown test is a specialized YAML code block in an ordinary markdown file. A normal code block starts with 3 backticks (` ``` `). A test code block starts with _at least 6_ backticks (` `````` `).

``````````markdown
# This is an ordinary markdown file

Using markdown tests, you can enhance your docs with real examples that are correct and up to date.

``````yml
input: Markdown Test!
output: Hello Markdown Test!
``````
``````````

You can further customize how tests are extracted from markdown files in the `[MarkdownTest]` attributes.

## Use `json-diff` for data driven assertion

You can use _any_ assertion library to verify your test expectations.

`json-diff` is a built-in tool to write assertions in YAML that fits the `yunit` testing style.

```csharp
var jsonDiff = new JsonDiffBuilder().Build();
jsonDiff.Verify(expected, actual, "Test failed");
```

The idea of `json-dff` is to serialize your expected object and actual object into JSON and compare them using a diff algorithm.


![Json Diff in Visual Studio](./media/json-diff-in-visual-studio.png)

The bare metal comparison expect the actual object to match exactly the same as the expected object, but you can custimize the `json-diff` pipeline to transform the expected and actual object before doing the text comparison. The built in `json-diff` pipeline comes with several handy extensions:


method | description | example
-------|-----------------------------------------------|-------------
`UseIgnoreNull` | Ignore the actual result of a property if the expected value is null | `{ "a": null }` matches `{”a": xxx}` where `xxx` could be anything
`UseNegate` | Assert the actual result must not be the expected result if the expected value starts with `!` | Given an expectation of `{ "a": "!value"}` , `{"a": "value"} should fail
`UseRegex` | The value must match a regex if the expectation looks like `/{regex}/` | `{"a": "/^.*$/" }` uses `^.*$` to match the actual value
`UseWildcard` | Use wildcard match if the expectation contains `*` | `{"a": "a*" }` uses wildcard `a*` to match against actual value
`UseJson` | The value must be a JSON string and its value is verified using the current pipeline, all the above rules will apply if they are configured in the pipeline |
`UseHtml` | The value must be an HTML string and its value is normalized before verification |

`json-diff` allows you to write stateful data driven tests. The [Shopping Cart Web API sample](./samples) demonstrates a custom `json-diff` extension that preserves states between multiple web API calls for a typical web API testing scenario
.
1 change: 0 additions & 1 deletion docs/_config.yml

This file was deleted.

Binary file added media/json-diff-in-visual-studio.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
12 changes: 12 additions & 0 deletions samples/1-hello/1-hello.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="yunit" Version="1.0.0-*" />
<PackageReference Include="xunit" Version="2.4.1" />
</ItemGroup>

</Project>
18 changes: 18 additions & 0 deletions samples/1-hello/HelloTest.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
using Microsoft.DocAsTest;
using Xunit;

public class HelloTestSpec
{
public string Input;
public string Output;
}

public class HelloTest
{
[YamlTest("~/1-hello/**/*.yml")]
[MarkdownTest("~/1-hello/**/*.md", FenceTip="test")]
public void Hello(HelloTestSpec spec)
{
Assert.Equal(spec.Output, $"Hello {spec.Input}");
}
}
63 changes: 63 additions & 0 deletions samples/1-hello/hello.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@

1. Create a new class library

```cmd
dotnet new console
```

2. Add a pre-release NuGet feed to `NuGet.config`

```xml
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="docfx-v3" value="https://www.myget.org/F/docfx-v3/api/v3/index.json" />
</packageSources>
</configuration>
```

3. Add NuGet reference to `Microsoft.DocAsTest`

```cmd
dotnet add package Microsoft.DocAsTest --version 3.0.0-*
```

4. Create test logic

```csharp
public class HelloTestSpec
{
public string Input;
public string Output;
}

public class HelloTest
{
[YamlTest("~/*.yml")]
[MarkdownTest("~/README.md")]
public void Hello(HelloTestSpec spec)
{
Assert.Equal(spec.Output, $"Hello {spec.Input}");
}
}

```

5. Create YAML test data

```yml
# Hello YAML Test! <-- hello.yml
input: YAML Test!
output: Hello YAML Test!
```
6. Create Markdown test data
``````````markdown
You can _also_ write test data in markdown files using YAML code blocks that starts with 6 or more backticks.
``````test
input: Markdown Test!
output: Hello Markdown Test!
``````
``````````
11 changes: 11 additions & 0 deletions samples/1-hello/hello.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Hello YAML Test!
input: YAML Test!
output: Hello YAML Test!
---
# This is a failed test
input: YAML Test
output:
---
# [skip] This is a skipped test
input: Skipped test
output:
13 changes: 13 additions & 0 deletions samples/2-json-diff/2-json-diff.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>netcoreapp2.2</TargetFramework>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="yunit" Version="1.0.0-*" />
<PackageReference Include="xunit" Version="2.4.1" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.1" />
</ItemGroup>

</Project>
99 changes: 99 additions & 0 deletions samples/2-json-diff/JsonDiffTest.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
using Microsoft.DocAsTest;
using Xunit;

public class ShoppingCart
{
public ShoppingCartLine[] Lines { get; set; }
}

public class ShoppingCartLine
{
public string Name { get; set; }

public string[] Tags { get; set; }

public int Quatity { get; set; }
}

public class JsonDiffTest
{
private readonly ShoppingCart _expectedCart = new ShoppingCart
{
Lines = new ShoppingCartLine[]
{
new ShoppingCartLine
{
Name = "Transformers: The Last Knight Bumblebee Voice Changer Mask",
Tags = new[]
{
"4.6 x 8.5 x 11 inches",
"1.1 pounds",
"B01N1834JJ",
"C1324",
"5 years and up",
"3 AA batteries required. (included)",
},
Quatity = 1,
}
}
};

private readonly ShoppingCart _actualCart = new ShoppingCart
{
Lines = new ShoppingCartLine[]
{
new ShoppingCartLine
{
Name = "Transformers: The Last Knight Bumblebee Voice Changer Mask",
Tags = new[]
{
"4.6 x 8.5 x 11 inches",
"1.1 pounds",
"B01N1834JJ",
"C1324",
"5 years and up",
"2 AA batteries required. (included)",
},
Quatity = 1,
}
},
};

[Fact]
public void XunitEquals()
{
Assert.Equal(_expectedCart.Lines.Length, _actualCart.Lines.Length);

for (var i = 0; i < _expectedCart.Lines.Length; i++)
{
Assert.Equal(_expectedCart.Lines[i].Name, _actualCart.Lines[i].Name);
Assert.Equal(_expectedCart.Lines[i].Tags, _actualCart.Lines[i].Tags);
Assert.Equal(_expectedCart.Lines[i].Quatity, _actualCart.Lines[i].Quatity);
}
}

[Fact]
public void JsonDiffEquals()
{
new JsonDiff().Verify(_expectedCart, _actualCart);
}

[Fact]
public void JsonDiffPipelineEquals()
{
var jsonDiff = new JsonDiffBuilder()
.UseAdditionalProperties()
.UseWildcard()
.Build();

// Allow additional properties
jsonDiff.Verify(
expected: new { foo = "bar" },
actual: new { foo = "bar", additionalProperty = "baz" });

// Allow wildcard match
jsonDiff.Verify(
expected: new { foo = "this * sentence" },
actual: new { foo = "this is a whole sentence" });
}
}
18 changes: 18 additions & 0 deletions samples/3-aspnetcore-test/3-aspnetcore-test.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>netcoreapp3.0</TargetFramework>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.AspNetCore.App" />
<PackageReference Include="Microsoft.AspNetCore.TestHost" Version="2.2.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.3.0" />
<PackageReference Include="yunit" Version="1.0.0-*" />
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\3-aspnetcore\3-aspnetcore.csproj" />
</ItemGroup>

</Project>
Loading

0 comments on commit 8391102

Please sign in to comment.