Skip to content

Commit f0592dd

Browse files
EvangelinkCopilot
Evangelink
and
Copilot
authored
Improve analyzers docs (#51209)
* Improve analyzers docs * Address code review comments * Fix toc issue * Add missing periods to tables and lists in MSTest analyzer documentation (#51301) * Initial plan * Add missing periods to tables and lists in analyzer rules documentation Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com> --------- Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
1 parent 2389c9e commit f0592dd

6 files changed

Lines changed: 500 additions & 81 deletions

File tree

‎docs/core/testing/mstest-analyzers/design-rules.md‎

Lines changed: 62 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -8,20 +8,65 @@ ms.date: 01/03/2024
88

99
# MSTest design rules
1010

11-
Design rules will help you create and maintain test suites that adhere to proper design and good practices.
12-
13-
Identifier | Name | Description
14-
-----------|------|------------
15-
[MSTEST0004](mstest0004.md) | PublicTypeShouldBeTestClassAnalyzer | It's considered a good practice to have only test classes marked public in a test project.
16-
[MSTEST0006](mstest0006.md) | AvoidExpectedExceptionAttributeAnalyzer | Prefer `Assert.ThrowsExactly` or `Assert.ThrowsExactlyAsync` over `[ExpectedException]` as it ensures that only the expected call throws the expected exception. The assert APIs also provide more flexibility and allow you to assert extra properties of the exception.
17-
[MSTEST0015](mstest0015.md) | TestMethodShouldNotBeIgnored | Test methods should not be ignored (marked with `[Ignore]`).
18-
[MSTEST0016](mstest0016.md) | TestClassShouldHaveTestMethod | Test class should have at least one test method or be 'static' with method(s) marked by `[AssemblyInitialization]` and/or `[AssemblyCleanup]`.
19-
[MSTEST0019](mstest0019.md) | PreferTestInitializeOverConstructorAnalyzer | Prefer TestInitialize methods over constructors
20-
[MSTEST0020](mstest0020.md) | PreferConstructorOverTestInitializeAnalyzer | Prefer constructors over TestInitialize methods
21-
[MSTEST0021](mstest0021.md) | PreferDisposeOverTestCleanupAnalyzer | Prefer Dispose over TestCleanup methods
22-
[MSTEST0022](mstest0022.md) | PreferTestCleanupOverDisposeAnalyzer | Prefer TestCleanup over Dispose methods
23-
[MSTEST0025](mstest0025.md) | PreferAssertFailOverAlwaysFalseConditionsAnalyzer | Use 'Assert.Fail' instead of an always-failing assert
24-
[MSTEST0029](mstest0029.md) | PublicMethodShouldBeTestMethod | A `public` method of a class marked with `[TestClass]` should be a test method (marked with `[TestMethod]`). The rule ignores methods that are marked with `[TestInitialize]`, or `[TestCleanup]` attributes.
25-
[MSTEST0036](mstest0036.md) | DoNotUseShadowingAnalyzer | Shadowing test members could cause testing issues (such as NRE).
26-
[MSTEST0044](mstest0044.md) | PreferTestMethodOverDataTestMethodAnalyzer | A method or type uses <xref:Microsoft.VisualStudio.TestTools.UnitTesting.DataTestMethodAttribute> or inherits from it.
27-
[MSTEST0045](mstest0045.md) | UseCooperativeCancellationForTimeoutAnalyzer | A test method uses <xref:Microsoft.VisualStudio.TestTools.UnitTesting.TimeoutAttribute> without setting the `CooperativeCancellation` property to `true`.
11+
Design rules help you create and maintain test suites that adhere to proper design and good practices. These rules focus on test structure, best practices, and common patterns that lead to maintainable test code.
12+
13+
## Rules in this category
14+
15+
| Rule ID | Title | Severity | Fix Available |
16+
|---------|-------|----------|---------------|
17+
| [MSTEST0004](mstest0004.md) | Public types should be test classes. | Info | Yes |
18+
| [MSTEST0006](mstest0006.md) | Avoid ExpectedException attribute. | Info | Yes |
19+
| [MSTEST0015](mstest0015.md) | Test method should not be ignored. | None (opt-in) | No |
20+
| [MSTEST0016](mstest0016.md) | Test class should have test method. | Info | No |
21+
| [MSTEST0019](mstest0019.md) | Prefer TestInitialize over constructors. | None (opt-in) | Yes |
22+
| [MSTEST0020](mstest0020.md) | Prefer constructors over TestInitialize. | None (opt-in) | Yes |
23+
| [MSTEST0021](mstest0021.md) | Prefer Dispose over TestCleanup. | None (opt-in) | Yes |
24+
| [MSTEST0022](mstest0022.md) | Prefer TestCleanup over Dispose. | None (opt-in) | Yes |
25+
| [MSTEST0025](mstest0025.md) | Prefer Assert.Fail over always-false conditions. | Info | Yes |
26+
| [MSTEST0029](mstest0029.md) | Public method should be test method. | Info | Yes |
27+
| [MSTEST0036](mstest0036.md) | Do not use shadowing. | Warning | No |
28+
| [MSTEST0044](mstest0044.md) | Prefer TestMethod over DataTestMethod. | Info | Yes |
29+
| [MSTEST0045](mstest0045.md) | Use cooperative cancellation for timeout. | Info | Yes |
30+
31+
## Common scenarios
32+
33+
### Test class structure
34+
35+
When creating test classes, these rules help ensure proper design:
36+
37+
- **[MSTEST0004](mstest0004.md)**: Keep helper classes internal, only test classes should be public.
38+
- **[MSTEST0016](mstest0016.md)**: Ensure test classes contain at least one test method.
39+
- **[MSTEST0029](mstest0029.md)**: Public methods in test classes should be test methods.
40+
41+
### Initialization patterns
42+
43+
MSTest supports both constructors and TestInitialize methods. These mutually exclusive rules let you enforce a consistent pattern:
44+
45+
- **[MSTEST0019](mstest0019.md)**: Enforce TestInitialize for initialization (useful for async scenarios).
46+
- **[MSTEST0020](mstest0020.md)**: Enforce constructors for initialization (better for readonly fields).
47+
48+
### Cleanup patterns
49+
50+
Similarly, choose between Dispose and TestCleanup:
51+
52+
- **[MSTEST0021](mstest0021.md)**: Enforce Dispose pattern for cleanup.
53+
- **[MSTEST0022](mstest0022.md)**: Enforce TestCleanup for cleanup.
54+
55+
### Better assertions
56+
57+
- **[MSTEST0006](mstest0006.md)**: Use Assert.ThrowsExactly instead of [ExpectedException] for better precision.
58+
- **[MSTEST0025](mstest0025.md)**: Use Assert.Fail instead of Assert.IsTrue(false).
59+
60+
### Test quality
61+
62+
- **[MSTEST0015](mstest0015.md)**: Flag ignored tests (opt-in rule).
63+
- **[MSTEST0036](mstest0036.md)**: Avoid shadowing base class members.
64+
- **[MSTEST0044](mstest0044.md)**: Use TestMethod unless data-driven testing is needed.
65+
- **[MSTEST0045](mstest0045.md)**: Enable cancellation tokens for timeout handling.
66+
67+
## Related documentation
68+
69+
- [Write tests with MSTest](../unit-testing-mstest-writing-tests.md)
70+
- [Lifecycle](../unit-testing-mstest-writing-tests-lifecycle.md)
71+
- [Attributes](../unit-testing-mstest-writing-tests-attributes.md)
72+
- [Assertions](../unit-testing-mstest-writing-tests-assertions.md)

‎docs/core/testing/mstest-analyzers/overview.md‎

Lines changed: 193 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,9 @@ This mode is more aggressive than `Recommended`. All rules are enabled as warnin
7070
> - [MSTEST0021: Prefer Dispose over TestCleanup methods](mstest0021.md)
7171
> - [MSTEST0022: Prefer TestCleanup over Dispose methods](mstest0022.md)
7272
73-
## Categories
73+
## Rules by category
74+
75+
The analyzer rules are organized into the following categories:
7476

7577
### Design rules
7678

@@ -88,7 +90,196 @@ This mode is more aggressive than `Recommended`. All rules are enabled as warnin
8890

8991
[Usage rules](usage-rules.md) support proper usage of MSTest.
9092

91-
### MSTESTEXP
93+
## Rules by concept
94+
95+
Find rules organized by common testing scenarios and concepts:
96+
97+
### Test structure & attributes
98+
99+
Rules that help ensure your test classes and methods are properly structured and decorated:
100+
101+
- [MSTEST0002](mstest0002.md) - Test class should be valid
102+
- [MSTEST0003](mstest0003.md) - Test method should be valid
103+
- [MSTEST0004](mstest0004.md) - Public types should be test classes
104+
- [MSTEST0007](mstest0007.md) - Use attribute on test method
105+
- [MSTEST0016](mstest0016.md) - Test class should have test method
106+
- [MSTEST0029](mstest0029.md) - Public method should be test method
107+
- [MSTEST0030](mstest0030.md) - Type containing test method should be a test class
108+
- [MSTEST0036](mstest0036.md) - Do not use shadowing
109+
- [MSTEST0041](mstest0041.md) - Use condition-based attributes with test class
110+
- [MSTEST0044](mstest0044.md) - Prefer TestMethod over DataTestMethod
111+
- [MSTEST0056](mstest0056.md) - TestMethodAttribute should set DisplayName correctly
112+
- [MSTEST0057](mstest0057.md) - TestMethodAttribute should propagate source information
113+
- [MSTEST0060](mstest0060.md) - Duplicate TestMethodAttribute
114+
115+
Related documentation: [Write tests with MSTest](../unit-testing-mstest-writing-tests.md), [Attributes](../unit-testing-mstest-writing-tests-attributes.md)
116+
117+
### Async/await patterns
118+
119+
Rules for writing asynchronous test code correctly:
120+
121+
- [MSTEST0013](mstest0013.md) - AssemblyCleanup should be valid (includes async rules)
122+
- [MSTEST0027](mstest0027.md) - Suppress async suffix for test methods
123+
- [MSTEST0028](mstest0028.md) - Suppress async suffix for test fixture methods
124+
- [MSTEST0039](mstest0039.md) - Use newer Assert.Throws methods (async variants)
125+
- [MSTEST0040](mstest0040.md) - Avoid using asserts in async void context
126+
- [MSTEST0045](mstest0045.md) - Use cooperative cancellation for timeout
127+
- [MSTEST0049](mstest0049.md) - Flow TestContext CancellationToken
128+
- [MSTEST0054](mstest0054.md) - Use CancellationToken property
129+
130+
Related documentation: [TestContext](../unit-testing-mstest-writing-tests-testcontext.md)
131+
132+
### Data-driven testing
133+
134+
Rules for working with data-driven test scenarios:
135+
136+
- [MSTEST0007](mstest0007.md) - Use attribute on test method
137+
- [MSTEST0014](mstest0014.md) - DataRow should be valid
138+
- [MSTEST0018](mstest0018.md) - DynamicData should be valid
139+
- [MSTEST0042](mstest0042.md) - Duplicate DataRow
140+
- [MSTEST0052](mstest0052.md) - Avoid explicit DynamicDataSourceType
141+
- [MSTEST0062](mstest0062.md) - Avoid out/ref test method parameters
142+
143+
Related documentation: [Attributes used for data-driven testing](../unit-testing-mstest-writing-tests-attributes.md#attributes-used-for-data-driven-testing)
144+
145+
### Lifecycle & initialization
146+
147+
Rules for test initialization, cleanup, and lifecycle management:
148+
149+
- [MSTEST0008](mstest0008.md) - TestInitialize should be valid
150+
- [MSTEST0009](mstest0009.md) - TestCleanup should be valid
151+
- [MSTEST0010](mstest0010.md) - ClassInitialize should be valid
152+
- [MSTEST0011](mstest0011.md) - ClassCleanup should be valid
153+
- [MSTEST0012](mstest0012.md) - AssemblyInitialize should be valid
154+
- [MSTEST0013](mstest0013.md) - AssemblyCleanup should be valid
155+
- [MSTEST0019](mstest0019.md) - Prefer TestInitialize over constructors
156+
- [MSTEST0020](mstest0020.md) - Prefer constructors over TestInitialize
157+
- [MSTEST0021](mstest0021.md) - Prefer Dispose over TestCleanup
158+
- [MSTEST0022](mstest0022.md) - Prefer TestCleanup over Dispose
159+
- [MSTEST0034](mstest0034.md) - Use ClassCleanupBehavior.EndOfClass
160+
- [MSTEST0050](mstest0050.md) - Global test fixture should be valid
161+
162+
Related documentation: [Lifecycle](../unit-testing-mstest-writing-tests-lifecycle.md)
163+
164+
### Assertions
165+
166+
Rules for using assertion methods correctly and effectively:
167+
168+
- [MSTEST0006](mstest0006.md) - Avoid ExpectedException attribute
169+
- [MSTEST0014](mstest0014.md) - DataRow should be valid
170+
- [MSTEST0017](mstest0017.md) - Assertion args should be passed in correct order
171+
- [MSTEST0023](mstest0023.md) - Do not negate boolean assertion
172+
- [MSTEST0025](mstest0025.md) - Prefer Assert.Fail over always-false conditions
173+
- [MSTEST0026](mstest0026.md) - Assertion args should avoid conditional access
174+
- [MSTEST0032](mstest0032.md) - Review always-true assert condition
175+
- [MSTEST0037](mstest0037.md) - Use proper assert methods
176+
- [MSTEST0038](mstest0038.md) - Avoid Assert.AreSame with value types
177+
- [MSTEST0039](mstest0039.md) - Use newer Assert.Throws methods
178+
- [MSTEST0046](mstest0046.md) - Use Assert instead of StringAssert
179+
- [MSTEST0051](mstest0051.md) - Assert.Throws should contain single statement
180+
- [MSTEST0053](mstest0053.md) - Avoid Assert format parameters
181+
- [MSTEST0058](mstest0058.md) - Avoid asserts in catch blocks
182+
183+
Related documentation: [Assertions](../unit-testing-mstest-writing-tests-assertions.md)
184+
185+
### TestContext
186+
187+
Rules for properly using the TestContext object:
188+
189+
- [MSTEST0005](mstest0005.md) - TestContext should be valid
190+
- [MSTEST0024](mstest0024.md) - Do not store static TestContext
191+
- [MSTEST0033](mstest0033.md) - Suppress non-nullable reference not initialized warning
192+
- [MSTEST0048](mstest0048.md) - TestContext property usage
193+
- [MSTEST0049](mstest0049.md) - Flow TestContext CancellationToken
194+
- [MSTEST0054](mstest0054.md) - Use CancellationToken property
195+
196+
Related documentation: [TestContext](../unit-testing-mstest-writing-tests-testcontext.md)
197+
198+
### Test configuration
199+
200+
Rules for configuring test execution, parallelization, and other test settings:
201+
202+
- [MSTEST0001](mstest0001.md) - Use Parallelize attribute
203+
- [MSTEST0015](mstest0015.md) - Test method should not be ignored
204+
- [MSTEST0031](mstest0031.md) - Do not use System.ComponentModel.DescriptionAttribute
205+
- [MSTEST0035](mstest0035.md) - Use DeploymentItem with test method or test class
206+
- [MSTEST0043](mstest0043.md) - Use retry attribute on test method
207+
- [MSTEST0045](mstest0045.md) - Use cooperative cancellation for timeout
208+
- [MSTEST0055](mstest0055.md) - Do not ignore string method return value
209+
- [MSTEST0059](mstest0059.md) - Use Parallelize attribute correctly
210+
- [MSTEST0061](mstest0061.md) - Use OSCondition attribute instead of runtime check
211+
212+
Related documentation: [Configure MSTest](../unit-testing-mstest-configure.md), [Running tests](../unit-testing-mstest-running-tests.md)
213+
214+
## All rules (quick reference)
215+
216+
| Rule ID | Category | Title | Default Severity |
217+
|---------|----------|-------|------------------|
218+
| [MSTEST0001](mstest0001.md) | Performance | Use Parallelize attribute | Info |
219+
| [MSTEST0002](mstest0002.md) | Usage | Test class should be valid | Warning |
220+
| [MSTEST0003](mstest0003.md) | Usage | Test method should be valid | Warning → Error* |
221+
| [MSTEST0004](mstest0004.md) | Design | Public types should be test classes | Info |
222+
| [MSTEST0005](mstest0005.md) | Usage | TestContext should be valid | Warning |
223+
| [MSTEST0006](mstest0006.md) | Design | Avoid ExpectedException attribute | Info |
224+
| [MSTEST0007](mstest0007.md) | Usage | Use attribute on test method | Warning |
225+
| [MSTEST0008](mstest0008.md) | Usage | TestInitialize should be valid | Warning |
226+
| [MSTEST0009](mstest0009.md) | Usage | TestCleanup should be valid | Warning |
227+
| [MSTEST0010](mstest0010.md) | Usage | ClassInitialize should be valid | Warning |
228+
| [MSTEST0011](mstest0011.md) | Usage | ClassCleanup should be valid | Warning |
229+
| [MSTEST0012](mstest0012.md) | Usage | AssemblyInitialize should be valid | Warning |
230+
| [MSTEST0013](mstest0013.md) | Usage | AssemblyCleanup should be valid | Warning |
231+
| [MSTEST0014](mstest0014.md) | Usage | DataRow should be valid | Warning |
232+
| [MSTEST0015](mstest0015.md) | Design | Test method should not be ignored | None (opt-in) |
233+
| [MSTEST0016](mstest0016.md) | Design | Test class should have test method | Info |
234+
| [MSTEST0017](mstest0017.md) | Usage | Assertion args should be passed in correct order | Info |
235+
| [MSTEST0018](mstest0018.md) | Usage | DynamicData should be valid | Warning |
236+
| [MSTEST0019](mstest0019.md) | Design | Prefer TestInitialize over constructors | None (opt-in) |
237+
| [MSTEST0020](mstest0020.md) | Design | Prefer constructors over TestInitialize | None (opt-in) |
238+
| [MSTEST0021](mstest0021.md) | Design | Prefer Dispose over TestCleanup | None (opt-in) |
239+
| [MSTEST0022](mstest0022.md) | Design | Prefer TestCleanup over Dispose | None (opt-in) |
240+
| [MSTEST0023](mstest0023.md) | Usage | Do not negate boolean assertion | Info |
241+
| [MSTEST0024](mstest0024.md) | Usage | Do not store static TestContext | Warning |
242+
| [MSTEST0025](mstest0025.md) | Design | Prefer Assert.Fail over always-false conditions | Info |
243+
| [MSTEST0026](mstest0026.md) | Usage | Assertion args should avoid conditional access | Info |
244+
| [MSTEST0027](mstest0027.md) | Suppression | Suppress async suffix for test methods | N/A |
245+
| [MSTEST0028](mstest0028.md) | Suppression | Suppress async suffix for test fixture methods | N/A |
246+
| [MSTEST0029](mstest0029.md) | Design | Public method should be test method | Info |
247+
| [MSTEST0030](mstest0030.md) | Usage | Type containing test method should be a test class | Warning |
248+
| [MSTEST0031](mstest0031.md) | Usage | Do not use System.ComponentModel.DescriptionAttribute | Info |
249+
| [MSTEST0032](mstest0032.md) | Usage | Review always-true assert condition | Info |
250+
| [MSTEST0033](mstest0033.md) | Suppression | Suppress non-nullable reference not initialized | N/A |
251+
| [MSTEST0034](mstest0034.md) | Usage | Use ClassCleanupBehavior.EndOfClass | Info |
252+
| [MSTEST0035](mstest0035.md) | Usage | Use DeploymentItem with test method or test class | Info |
253+
| [MSTEST0036](mstest0036.md) | Design | Do not use shadowing | Warning |
254+
| [MSTEST0037](mstest0037.md) | Usage | Use proper assert methods | Info |
255+
| [MSTEST0038](mstest0038.md) | Usage | Avoid Assert.AreSame with value types | Info |
256+
| [MSTEST0039](mstest0039.md) | Usage | Use newer Assert.Throws methods | Info |
257+
| [MSTEST0040](mstest0040.md) | Usage | Avoid using asserts in async void context | Warning |
258+
| [MSTEST0041](mstest0041.md) | Usage | Use condition-based attributes with test class | Warning |
259+
| [MSTEST0042](mstest0042.md) | Usage | Duplicate DataRow | Warning |
260+
| [MSTEST0043](mstest0043.md) | Usage | Use retry attribute on test method | Warning → Error* |
261+
| [MSTEST0044](mstest0044.md) | Design | Prefer TestMethod over DataTestMethod | Info |
262+
| [MSTEST0045](mstest0045.md) | Design | Use cooperative cancellation for timeout | Info |
263+
| [MSTEST0046](mstest0046.md) | Usage | Use Assert instead of StringAssert | Info |
264+
| [MSTEST0048](mstest0048.md) | Usage | TestContext property usage | Warning |
265+
| [MSTEST0049](mstest0049.md) | Usage | Flow TestContext CancellationToken | Info |
266+
| [MSTEST0050](mstest0050.md) | Usage | Global test fixture should be valid | Warning |
267+
| [MSTEST0051](mstest0051.md) | Usage | Assert.Throws should contain single statement | Info |
268+
| [MSTEST0052](mstest0052.md) | Usage | Avoid explicit DynamicDataSourceType | Info |
269+
| [MSTEST0053](mstest0053.md) | Usage | Avoid Assert format parameters | Info |
270+
| [MSTEST0054](mstest0054.md) | Usage | Use CancellationToken property | Info |
271+
| [MSTEST0055](mstest0055.md) | Usage | Do not ignore string method return value | Warning |
272+
| [MSTEST0056](mstest0056.md) | Usage | TestMethodAttribute should set DisplayName correctly | Info |
273+
| [MSTEST0057](mstest0057.md) | Usage | TestMethodAttribute should propagate source information | Warning |
274+
| [MSTEST0058](mstest0058.md) | Usage | Avoid asserts in catch blocks | Info |
275+
| [MSTEST0059](mstest0059.md) | Usage | Use Parallelize attribute correctly | Warning |
276+
| [MSTEST0060](mstest0060.md) | Usage | Duplicate TestMethodAttribute | Warning |
277+
| [MSTEST0061](mstest0061.md) | Usage | Use OSCondition attribute instead of runtime check | Info |
278+
| [MSTEST0062](mstest0062.md) | Usage | Avoid out/ref test method parameters | Warning |
279+
280+
\* Escalated to Error in `Recommended` and `All` modes.
281+
282+
## MSTESTEXP
92283

93284
Several APIs of MSTest are decorated with the <xref:System.Diagnostics.CodeAnalysis.ExperimentalAttribute>. This attribute indicates that the API is experimental and may be removed or changed in future versions of MSTest. The attribute is used to identify APIs that aren't yet stable and may not be suitable for production use.
94285

0 commit comments

Comments
 (0)