-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathhotspot-unit-tests.html
500 lines (500 loc) · 25.1 KB
/
hotspot-unit-tests.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang="">
<head>
<meta charset="utf-8" />
<meta name="generator" content="pandoc" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
<title>Native/Unit Test Development Guidelines</title>
<style>
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
div.columns{display: flex; gap: min(4vw, 1.5em);}
div.column{flex: auto; overflow-x: auto;}
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
ul.task-list{list-style: none;}
ul.task-list li input[type="checkbox"] {
width: 0.8em;
margin: 0 0.8em 0.2em -1.6em;
vertical-align: middle;
}
.display.math{display: block; text-align: center; margin: 0.5rem auto;}
</style>
<link rel="stylesheet" href="../make/data/docs-resources/resources/jdk-default.css" />
<!--[if lt IE 9]>
<script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv-printshiv.min.js"></script>
<![endif]-->
</head>
<body>
<header id="title-block-header">
<h1 class="title">Native/Unit Test Development Guidelines</h1>
</header>
<nav id="TOC" role="doc-toc">
<ul>
<li><a href="#good-test-properties" id="toc-good-test-properties">Good
test properties</a>
<ul>
<li><a href="#lightness" id="toc-lightness">Lightness</a></li>
<li><a href="#isolation" id="toc-isolation">Isolation</a></li>
<li><a href="#atomicity-and-self-containment"
id="toc-atomicity-and-self-containment">Atomicity and
self-containment</a></li>
<li><a href="#repeatability"
id="toc-repeatability">Repeatability</a></li>
<li><a href="#informativeness"
id="toc-informativeness">Informativeness</a></li>
<li><a href="#testing-instead-of-visiting"
id="toc-testing-instead-of-visiting">Testing instead of
visiting</a></li>
<li><a href="#nearness" id="toc-nearness">Nearness</a></li>
</ul></li>
<li><a href="#asserts" id="toc-asserts">Asserts</a>
<ul>
<li><a href="#several-checks" id="toc-several-checks">Several
checks</a></li>
<li><a href="#first-parameter-is-expected-value"
id="toc-first-parameter-is-expected-value">First parameter is expected
value</a></li>
<li><a href="#floating-point-comparison"
id="toc-floating-point-comparison">Floating-point comparison</a></li>
<li><a href="#c-string-comparison" id="toc-c-string-comparison">C string
comparison</a></li>
<li><a href="#error-messages" id="toc-error-messages">Error
messages</a></li>
<li><a href="#uncluttered-output"
id="toc-uncluttered-output">Uncluttered output</a></li>
<li><a href="#failures-propagation"
id="toc-failures-propagation">Failures propagation</a></li>
</ul></li>
<li><a href="#naming-and-grouping" id="toc-naming-and-grouping">Naming
and Grouping</a>
<ul>
<li><a href="#test-group-names" id="toc-test-group-names">Test group
names</a></li>
<li><a href="#filename" id="toc-filename">Filename</a></li>
<li><a href="#file-location" id="toc-file-location">File
location</a></li>
<li><a href="#test-names" id="toc-test-names">Test names</a></li>
<li><a href="#fixture-classes" id="toc-fixture-classes">Fixture
classes</a></li>
<li><a href="#friend-classes" id="toc-friend-classes">Friend
classes</a></li>
<li><a href="#oscpu-specific-tests" id="toc-oscpu-specific-tests">OS/CPU
specific tests</a></li>
</ul></li>
<li><a href="#miscellaneous" id="toc-miscellaneous">Miscellaneous</a>
<ul>
<li><a href="#hotspot-style" id="toc-hotspot-style">Hotspot
style</a></li>
<li><a href="#codetest-metrics" id="toc-codetest-metrics">Code/test
metrics</a></li>
<li><a href="#access-to-non-public-members"
id="toc-access-to-non-public-members">Access to non-public
members</a></li>
<li><a href="#death-tests" id="toc-death-tests">Death tests</a></li>
<li><a href="#external-flags" id="toc-external-flags">External
flags</a></li>
<li><a href="#test-specific-flags"
id="toc-test-specific-flags">Test-specific flags</a></li>
<li><a href="#flag-restoring" id="toc-flag-restoring">Flag
restoring</a></li>
<li><a href="#googletest-documentation"
id="toc-googletest-documentation">GoogleTest documentation</a></li>
</ul></li>
<li><a href="#todo" id="toc-todo">TODO</a></li>
</ul>
</nav>
<p>The purpose of these guidelines is to establish a shared vision on
what kind of native tests and how we want to develop them for Hotspot
using GoogleTest. Hence these guidelines include style items as well as
test approach items.</p>
<p>First section of this document describes properties of good tests
which are common for almost all types of test regardless of language,
framework, etc. Further sections provide recommendations to achieve
those properties and other HotSpot and/or GoogleTest specific
guidelines.</p>
<h2 id="good-test-properties">Good test properties</h2>
<h3 id="lightness">Lightness</h3>
<p>Use the most lightweight type of tests.</p>
<p>In Hotspot, there are 3 different types of tests regarding their
dependency on a JVM, each next level is slower than previous</p>
<ul>
<li><p><code>TEST</code> : a test does not depend on a JVM</p></li>
<li><p><code>TEST_VM</code> : a test does depend on an initialized JVM,
but are supposed not to break a JVM, i.e. leave it in a workable
state.</p></li>
<li><p><code>TEST_OTHER_VM</code> : a test depends on a JVM and requires
a freshly initialized JVM or leaves a JVM in non-workable state</p></li>
</ul>
<h3 id="isolation">Isolation</h3>
<p>Tests have to be isolated: not to have visible side-effects,
influences on other tests results.</p>
<p>Results of one test should not depend on test execution order, other
tests, otherwise it is becoming almost impossible to find out why a test
failed. Due to hotspot-specific, it is not so easy to get a full
isolation, e.g. we share an initialized JVM between all
<code>TEST_VM</code> tests, so if your test changes JVM's state too
drastically and does not change it back, you had better consider
<code>TEST_OTHER_VM</code>.</p>
<h3 id="atomicity-and-self-containment">Atomicity and
self-containment</h3>
<p>Tests should be <em>atomic</em> and <em>self-contained</em> at the
same time.</p>
<p>One test should check a particular part of a class, subsystem,
functionality, etc. Then it is quite easy to determine what parts of a
product are broken basing on test failures. On the other hand, a test
should test that part more-or-less entirely, because when one sees a
test <code>FooTest::bar</code>, they assume all aspects of bar from
<code>Foo</code> are tested.</p>
<p>However, it is impossible to cover all aspects even of a method, not
to mention a subsystem. In such cases, it is recommended to have several
tests, one for each aspect of a thing under test. For example one test
to tests how <code>Foo::bar</code> works if an argument is
<code>null</code>, another test to test how it works if an argument is
acceptable but <code>Foo</code> is not in the right state to accept it
and so on. This helps not only to make tests atomic, self-contained but
also makes test name self-descriptive (discussed in more details in <a
href="#test-names">Test names</a>).</p>
<h3 id="repeatability">Repeatability</h3>
<p>Tests have to be repeatable.</p>
<p>Reproducibility is very crucial for a test. No one likes sporadic
test failures, they are hard to investigate, fix and verify a fix.</p>
<p>In some cases, it is quite hard to write a 100% repeatable test,
since besides a test there can be other moving parts, e.g. in case of
<code>TEST_VM</code> there are several concurrently running threads.
Despite this, we should try to make a test as reproducible as
possible.</p>
<h3 id="informativeness">Informativeness</h3>
<p>In case of a failure, a test should be as <em>informative</em> as
possible.</p>
<p>Having more information about a test failure than just compared
values can be very useful for failure troubleshooting, it can reduce or
even completely eliminate debugging hours. This is even more important
in case of not 100% reproducible failures.</p>
<p>Achieving this property, one can easily make a test too verbose, so
it will be really hard to find useful information in the ocean of
useless information. Hence they should not only think about how to
provide <a href="#error-messages">good information</a>, but also <a
href="#uncluttered-output">when to do it</a>.</p>
<h3 id="testing-instead-of-visiting">Testing instead of visiting</h3>
<p>Tests should <em>test</em>.</p>
<p>It is not enough just to "visit" some code, a test should check that
code does that it has to do, compare return values with expected values,
check that desired side effects are done, and undesired are not, and so
on. In other words, a test should contain at least one GoogleTest
assertion and do not rely on JVM asserts.</p>
<p>Generally speaking to write a good test, one should create a model of
the system under tests, a model of possible bugs (or bugs which one
wants to find) and design tests using those models.</p>
<h3 id="nearness">Nearness</h3>
<p>Prefer having checks inside test code.</p>
<p>Not only does having test logic outside, e.g. verification method,
depending on asserts in product code contradict with several items above
but also decreases test’s readability and stability. It is much easier
to understand that a test is testing when all testing logic is located
inside a test or nearby in shared test libraries. As a rule of thumb,
the closer a check to a test, the better.</p>
<h2 id="asserts">Asserts</h2>
<h3 id="several-checks">Several checks</h3>
<p>Prefer <code>EXPECT</code> over <code>ASSERT</code> if possible.</p>
<p>This is related to the <a href="#informativeness">informativeness</a>
property of tests, information for other checks can help to better
localize a defect’s root-cause. One should use <code>ASSERT</code> if it
is impossible to continue test execution or if it does not make much
sense. Later in the text, <code>EXPECT</code> forms will be used to
refer to both <code>ASSERT/EXPECT</code>.</p>
<p>When it is possible to make several different checks, but impossible
to continue test execution if at least one check fails, you can use
<code>::testing::Test::HasNonfatalFailure()</code> function. The
recommended way to express that is
<code>ASSERT_FALSE(::testing::Test::HasNonfatalFailure())</code>.
Besides making it clear why a test is aborted, it also allows you to
provide more information about a failure.</p>
<h3 id="first-parameter-is-expected-value">First parameter is expected
value</h3>
<p>In all equality assertions, expected values should be passed as the
first parameter.</p>
<p>This convention is adopted by GoogleTest, and there is a slight
difference in how GoogleTest treats parameters, the most important one
is <code>null</code> detection. Due to different reasons,
<code>null</code> detection is enabled only for the first parameter,
that is to said <code>EXPECT_EQ(NULL, object)</code> checks that object
is <code>null</code>, while <code>EXPECT_EQ(object, NULL)</code> checks
that object equals to <code>NULL</code>, GoogleTest is very strict
regarding types of compared values so the latter will generates a
compile-time error.</p>
<h3 id="floating-point-comparison">Floating-point comparison</h3>
<p>Use floating-point special macros to compare
<code>float/double</code> values.</p>
<p>Because of floating-point number representations and round-off
errors, regular equality comparison will not return true in most cases.
There are special <code>EXPECT_FLOAT_EQ/EXPECT_DOUBLE_EQ</code>
assertions which check that the distance between compared values is not
more than 4 ULPs, there is also <code>EXPECT_NEAR(v1, v2, eps)</code>
which checks that the absolute value of the difference between
<code>v1</code> and <code>v2</code> is not greater than
<code>eps</code>.</p>
<h3 id="c-string-comparison">C string comparison</h3>
<p>Use string special macros for C strings comparisons.</p>
<p><code>EXPECT_EQ</code> just compares pointers’ values, which is
hardly what one wants comparing C strings. GoogleTest provides
<code>EXPECT_STREQ</code> and <code>EXPECT_STRNE</code> macros to
compare C string contents. There are also case-insensitive versions
<code>EXPECT_STRCASEEQ</code>, <code>EXPECT_STRCASENE</code>.</p>
<h3 id="error-messages">Error messages</h3>
<p>Provide informative, but not too verbose error messages.</p>
<p>All GoogleTest asserts print compared expressions and their values,
so there is no need to have them in error messages. Asserts print only
compared values, they do not print any of interim variables, e.g.
<code>ASSERT_TRUE((val1 == val2 && isFail(foo(8)) || i == 18)</code>
prints only one value. If you use some complex predicates, please
consider <code>EXPECT_PRED*</code> or <code>EXPECT_FORMAT_PRED</code>
assertions family, they check that a predicate returns true/success and
print out all parameters values.</p>
<p>However in some cases, default information is not enough, a commonly
used example is an assert inside a loop, GoogleTest will not print
iteration values (unless it is an assert's parameter). Other
demonstrative examples are printing error code and a corresponding error
message; printing internal states which might have an impact on results.
One should add this information to assert message using
<code><<</code> operator.</p>
<h3 id="uncluttered-output">Uncluttered output</h3>
<p>Print information only if it is needed.</p>
<p>Too verbose tests which print all information even if they pass are
very bad practice. They just pollute output, so it becomes harder to
find useful information. In order not print information till it is
really needed, one should consider saving it to a temporary buffer and
pass to an assert. <a
href="https://git.openjdk.org/jdk/blob/master/test/hotspot/gtest/gc/shared/test_memset_with_concurrent_readers.cpp"
class="uri">https://git.openjdk.org/jdk/blob/master/test/hotspot/gtest/gc/shared/test_memset_with_concurrent_readers.cpp</a>
has a good example how to do that.</p>
<h3 id="failures-propagation">Failures propagation</h3>
<p>Wrap a subroutine call into <code>EXPECT_NO_FATAL_FAILURE</code>
macro to propagate failures.</p>
<p><code>ASSERT</code> and <code>FAIL</code> abort only the current
function, so if you have them in a subroutine, a test will not be
aborted after the subroutine even if <code>ASSERT</code> or
<code>FAIL</code> fails. You should call such subroutines in
<code>ASSERT_NO_FATAL_FAILURE</code> macro to propagate fatal failures
and abort a test. <code>(EXPECT|ASSERT)_NO_FATAL_FAILURE</code> can also
be used to provide more information.</p>
<p>Due to obvious reasons, there are no
<code>(EXPECT|ASSERT)_NO_NONFATAL_FAILURE</code> macros. However, if you
need to check if a subroutine generated a nonfatal failure (failed an
<code>EXPECT</code>), you can use
<code>::testing::Test::HasNonfatalFailure</code> function, or
<code>::testing::Test::HasFailure</code> function to check if a
subroutine generated any failures, see <a href="#several-checks">Several
checks</a>.</p>
<h2 id="naming-and-grouping">Naming and Grouping</h2>
<h3 id="test-group-names">Test group names</h3>
<p>Test group names should be in CamelCase, start and end with a letter.
A test group should be named after tested class, functionality,
subsystem, etc.</p>
<p>This naming scheme helps to find tests, filter them and simplifies
test failure analysis. For example, class <code>Foo</code> - test group
<code>Foo</code>, compiler logging subsystem - test group
<code>CompilerLogging</code>, G1 GC — test group <code>G1GC</code>, and
so forth.</p>
<h3 id="filename">Filename</h3>
<p>A test file must have <code>test_</code> prefix and <code>.cpp</code>
suffix.</p>
<p>Both are actually requirements from the current build system to
recognize your tests.</p>
<h3 id="file-location">File location</h3>
<p>Test file location should reflect a location of the tested part of
the product.</p>
<ul>
<li><p>All unit tests for a class from <code>foo/bar/baz.cpp</code>
should be placed <code>foo/bar/test_baz.cpp</code> in
<code>hotspot/test/native/</code> directory. Having all tests for a
class in one file is a common practice for unit tests, it helps to see
all existing tests at once, share functions and/or resources without
losing encapsulation.</p></li>
<li><p>For tests which test more than one class, directory hierarchy
should be the same as product hierarchy, and file name should reflect
the name of the tested subsystem/functionality. For example, if a
sub-system under tests belongs to <code>gc/g1</code>, tests should be
placed in <code>gc/g1</code> directory.</p></li>
</ul>
<p>Please note that framework prepends directory name to a test group
name. For example, if <code>TEST(foo, check_this)</code> and
<code>TEST(bar, check_that)</code> are defined in
<code>hotspot/test/native/gc/shared/test_foo.cpp</code> file, they will
be reported as <code>gc/shared/foo::check_this</code> and
<code>gc/shared/bar::check_that</code>.</p>
<h3 id="test-names">Test names</h3>
<p>Test names should be in small_snake_case, start and end with a
letter. A test name should reflect that a test checks.</p>
<p>Such naming makes tests self-descriptive and helps a lot during the
whole test life cycle. It is easy to do test planning, test inventory,
to see what things are not tested, to review tests, to analyze test
failures, to evolve a test, etc. For example
<code>foo_return_0_if_name_is_null</code> is better than
<code>foo_sanity</code> or <code>foo_basic</code> or just
<code>foo</code>,
<code>humongous_objects_can_not_be_moved_by_young_gc</code> is better
than <code>ho_young_gc</code>.</p>
<p>Actually using underscore is against GoogleTest project convention,
because it can lead to illegal identifiers, however, this is too strict.
Restricting usage of underscore for test names only and prohibiting test
name starts or ends with an underscore are enough to be safe.</p>
<h3 id="fixture-classes">Fixture classes</h3>
<p>Fixture classes should be named after tested classes, subsystems, etc
(follow <a href="#test-group-names">Test group names rule</a>) and have
<code>Test</code> suffix to prevent class name conflicts.</p>
<h3 id="friend-classes">Friend classes</h3>
<p>All test purpose friends should have either <code>Test</code> or
<code>Testable</code> suffix.</p>
<p>It greatly simplifies understanding of friendship’s purpose and
allows statically check that private members are not exposed
unexpectedly. Having <code>FooTest</code> as a friend of
<code>Foo</code> without any comments will be understood as a necessary
evil to get testability.</p>
<h3 id="oscpu-specific-tests">OS/CPU specific tests</h3>
<p>Guard OS/CPU specific tests by <code>#ifdef</code> and have OS/CPU
name in filename.</p>
<p>For the time being, we do not support separate directories for OS,
CPU, OS-CPU specific tests, in case we will have lots of such tests, we
will change directory layout and build system to support that in the
same way it is done in hotspot.</p>
<h2 id="miscellaneous">Miscellaneous</h2>
<h3 id="hotspot-style">Hotspot style</h3>
<p>Abide the norms and rules accepted in Hotspot style guide.</p>
<p>Tests are a part of Hotspot, so everything (if applicable) we use for
Hotspot, should be used for tests as well. Those guidelines cover
test-specific things.</p>
<h3 id="codetest-metrics">Code/test metrics</h3>
<p>Coverage information and other code/test metrics are quite useful to
decide what tests should be written, what tests should be improved and
what can be removed.</p>
<p>For unit tests, widely used and well-known coverage metric is branch
coverage, which provides good quality of tests with relatively easy test
development process. For other levels of testing, branch coverage is not
as good, and one should consider others metrics, e.g. transaction flow
coverage, data flow coverage.</p>
<h3 id="access-to-non-public-members">Access to non-public members</h3>
<p>Use explicit friend class to get access to non-public members.</p>
<p>We do not use GoogleTest macro to declare friendship relation,
because, from our point of view, it is less clear than an explicit
declaration.</p>
<p>Declaring a test fixture class as a friend class of a tested test is
the easiest and the clearest way to get access. However, it has some
disadvantages, here is some of them:</p>
<ul>
<li>Each test has to be declared as a friend</li>
<li>Subclasses do not inheritance friendship relation</li>
</ul>
<p>In other words, it is harder to share code between tests. Hence if
you want to share code or expect it to be useful in other tests, you
should consider making members in a tested class protected and introduce
a shared test-only class which expose those members via public
functions, or even making members publicly accessible right away in a
product class. If it is not an option to change members visibility, one
can create a friend class which exposes members.</p>
<h3 id="death-tests">Death tests</h3>
<p>You can not use death tests inside <code>TEST_OTHER_VM</code> and
<code>TEST_VM_ASSERT*</code>.</p>
<p>We tried to make Hotspot-GoogleTest integration as transparent as
possible, however, due to the current implementation of
<code>TEST_OTHER_VM</code> and <code>TEST_VM_ASSERT*</code> tests, you
cannot use death test functionality in them. These tests are implemented
as GoogleTest death tests, and GoogleTest does not allow to have a death
test inside another death test.</p>
<h3 id="external-flags">External flags</h3>
<p>Passing external flags to a tested JVM is not supported.</p>
<p>The rationality of such design decision is to simplify both tests and
a test framework and to avoid failures related to incompatible flags
combination till there is a good solution for that. However there are
cases when one wants to test a JVM with specific flags combination,
<code>_JAVA_OPTIONS</code> environment variable can be used to do that.
Flags from <code>_JAVA_OPTIONS</code> will be used in
<code>TEST_VM</code>, <code>TEST_OTHER_VM</code> and
<code>TEST_VM_ASSERT*</code> tests.</p>
<h3 id="test-specific-flags">Test-specific flags</h3>
<p>Passing flags to a tested JVM in <code>TEST_OTHER_VM</code> and
<code>TEST_VM_ASSERT*</code> should be possible, but is not implemented
yet.</p>
<p>Facility to pass test-specific flags is needed for system, regression
or other types of tests which require a fully initialized JVM in some
particular configuration, e.g. with Serial GC selected. There is no
support for such tests now, however, there is a plan to add that in
upcoming releases.</p>
<p>For now, if a test depends on flags values, it should have
<code>if (!<flag>) { return }</code> guards in the very beginning
and <code>@requires</code> comment similar to jtreg
<code>@requires</code> directive right before test macros. <a
href="https://git.openjdk.org/jdk/blob/master/test/hotspot/gtest/gc/g1/test_g1IHOPControl.cpp"
class="uri">https://git.openjdk.org/jdk/blob/master/test/hotspot/gtest/gc/g1/test_g1IHOPControl.cpp</a>
ha an example of this temporary workaround. It is important to follow
that pattern as it allows us to easily find all such tests and update
them as soon as there is an implementation of flag passing facility.</p>
<p>In long-term, we expect jtreg to support GoogleTest tests as first
class citizens, that is to say, jtreg will parse <span class="citation"
data-cites="requires">@requires</span> comments and filter out
inapplicable tests.</p>
<h3 id="flag-restoring">Flag restoring</h3>
<p>Restore changed flags.</p>
<p>It is quite common for tests to configure JVM in a certain way
changing flags’ values. GoogleTest provides two ways to set up
environment before a test and restore it afterward: using either
constructor and destructor or <code>SetUp</code> and
<code>TearDown</code> functions. Both ways require to use a test fixture
class, which sometimes is too wordy. The simpler facilities like
<code>FLAG_GUARD</code> macro or <code>*FlagSetting</code> classes could
be used in such cases to restore/set values.</p>
<p>Caveats:</p>
<ul>
<li><p>Changing a flag’s value could break the invariants between flags'
values and hence could lead to unexpected/unsupported JVM
state.</p></li>
<li><p><code>FLAG_SET_*</code> macros can change more than one flag (in
order to maintain invariants) so it is hard to predict what flags will
be changed and it makes restoring all changed flags a nontrivial task.
Thus in case one uses <code>FLAG_SET_*</code> macros, they should use
<code>TEST_OTHER_VM</code> test type.</p></li>
</ul>
<h3 id="googletest-documentation">GoogleTest documentation</h3>
<p>In case you have any questions regarding GoogleTest itself, its
asserts, test declaration macros, other macros, etc, please consult its
documentation.</p>
<h2 id="todo">TODO</h2>
<p>Although this document provides guidelines on the most important
parts of test development using GTest, it still misses a few items:</p>
<ul>
<li><p>Examples, esp for <a href="#access-to-non-public-members">access
to non-public members</a></p></li>
<li><p>test types: purpose, drawbacks, limitation</p>
<ul>
<li><code>TEST_VM</code></li>
<li><code>TEST_VM_F</code></li>
<li><code>TEST_OTHER_VM</code></li>
<li><code>TEST_VM_ASSERT</code></li>
<li><code>TEST_VM_ASSERT_MSG</code></li>
</ul></li>
<li><p>Miscellaneous</p>
<ul>
<li>Test libraries
<ul>
<li>where to place</li>
<li>how to write</li>
<li>how to use</li>
</ul></li>
<li>test your tests
<ul>
<li>how to run tests in random order</li>
<li>how to run only specific tests</li>
<li>how to run each test separately</li>
<li>check that a test can find bugs it is supposed to by introducing
them</li>
</ul></li>
<li>mocks/stubs/dependency injection</li>
<li>setUp/tearDown
<ul>
<li>vs c-tor/d-tor</li>
<li>empty test to test them</li>
</ul></li>
<li>internal (declared in .cpp) struct/classes</li>
</ul></li>
</ul>
</body>
</html>