-
Notifications
You must be signed in to change notification settings - Fork 17
/
Copy pathindex.html
6251 lines (6077 loc) · 269 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="Asciidoctor 2.0.18">
<meta name="description" content="TestNG user documentation">
<meta name="keywords" content="TestNG">
<title>TestNG Documentation</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700">
<link rel="stylesheet" href="./asciidoctor.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/prettify/r298/prettify.min.css">
<style>
/* Tocbot dynamic TOC, works with tocbot 3.0.2 */
/* Source: https://github.com/asciidoctor/asciidoctor/issues/699#issuecomment-321066006 */
#tocbot a.toc-link.node-name--H1{ font-style: italic }
@media screen{
#tocbot > ul.toc-list{ margin-bottom: 0.5em; margin-left: 0.125em }
#tocbot ul.sectlevel0, #tocbot a.toc-link.node-name--H1 + ul{
padding-left: 0 }
#tocbot a.toc-link{ height:100% }
.is-collapsible{ max-height:3000px; overflow:hidden; }
.is-collapsed{ max-height:0 }
.is-active-link{ font-weight:700 }
}
@media print{
#tocbot a.toc-link.node-name--H4{ display:none }
}
</style>
</head>
<body class="article toc2 toc-left">
<div id="header">
<h1>TestNG Documentation</h1>
<div class="details">
<span id="revnumber">7.9.0 </span>
</div>
<div id="toc" class="toc2">
<div id="toctitle">Table Of Contents</div>
<ul class="sectlevel1">
<li><a href="#_welcome_to_testng">1. Welcome to TestNG</a>
<ul class="sectlevel2">
<li><a href="#_requirements">1.1. Requirements</a></li>
<li><a href="#_mailing_lists">1.2. Mailing-lists</a></li>
<li><a href="#_locations_of_the_projects">1.3. Locations of the projects</a></li>
<li><a href="#_bug_reports">1.4. Bug reports</a></li>
<li><a href="#_license">1.5. License</a></li>
</ul>
</li>
<li><a href="#_download">2. Download</a>
<ul class="sectlevel2">
<li><a href="#_current_release_version">2.1. Current Release Version</a>
<ul class="sectlevel3">
<li><a href="#_maven">2.1.1. Maven</a></li>
<li><a href="#_gradle">2.1.2. Gradle</a></li>
<li><a href="#_snapshots">2.1.3. Snapshots</a></li>
<li><a href="#_eclipse_plug_in">2.1.4. Eclipse plug-in</a></li>
<li><a href="#_build_testng_from_source_code">2.1.5. Build TestNG from source code</a></li>
<li><a href="#_build_the_testng_eclipse_plugin_from_source_code">2.1.6. Build the TestNG Eclipse Plugin from source code</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_testng_documentation">3. TestNG Documentation</a>
<ul class="sectlevel2">
<li><a href="#_introduction">3.1. Introduction</a></li>
<li><a href="#_annotations">3.2. Annotations</a></li>
<li><a href="#_testng_xml">3.3. <code>testng.xml</code></a></li>
<li><a href="#_running_testng">3.4. Running TestNG</a>
<ul class="sectlevel3">
<li><a href="#_command_line_parameters">3.4.1. Command Line Parameters</a></li>
<li><a href="#_system_properties">3.4.2. System properties</a></li>
<li><a href="#_running_tests_from_within_a_test_jar">3.4.3. Running tests from within a test jar</a></li>
</ul>
</li>
<li><a href="#_test_methods_test_classes_and_test_groups">3.5. Test methods, Test classes and Test groups</a>
<ul class="sectlevel3">
<li><a href="#_test_methods">3.5.1. Test methods</a></li>
<li><a href="#_test_groups">3.5.2. Test groups</a>
<ul class="sectlevel4">
<li><a href="#_method_groups">Method groups</a></li>
</ul>
</li>
<li><a href="#_groups_of_groups">3.5.3. Groups of groups</a></li>
<li><a href="#_exclusion_groups">3.5.4. Exclusion groups</a></li>
<li><a href="#_partial_groups">3.5.5. Partial groups</a></li>
</ul>
</li>
<li><a href="#_parameters">3.6. Parameters</a>
<ul class="sectlevel3">
<li><a href="#_parameters_from_testng_xml">3.6.1. Parameters from testng.xml</a></li>
<li><a href="#_parameters_with_dataproviders">3.6.2. Parameters with DataProviders</a></li>
<li><a href="#_retries_and_data_providers">3.6.3. Retries and data providers</a></li>
<li><a href="#_controlling_threadpool_usage">3.6.4. Controlling ThreadPool Usage</a></li>
<li><a href="#_parameters_from_system_properties">3.6.5. Parameters from System Properties</a>
<ul class="sectlevel4">
<li><a href="#_parameters_in_reports">Parameters in reports</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#_dependencies">3.7. Dependencies</a>
<ul class="sectlevel3">
<li><a href="#_dependencies_with_annotations">3.7.1. Dependencies with annotations</a></li>
<li><a href="#_dependencies_in_xml">3.7.2. Dependencies in XML</a></li>
</ul>
</li>
<li><a href="#_factories">3.8. Factories</a></li>
<li><a href="#_class_level_annotations">3.9. Class level annotations</a></li>
<li><a href="#_ignoring_tests">3.10. Ignoring tests</a></li>
<li><a href="#_parallelism_and_time_outs">3.11. Parallelism and time-outs</a>
<ul class="sectlevel3">
<li><a href="#_parallel_suites">3.11.1. Parallel suites</a></li>
<li><a href="#_parallel_tests_classes_and_methods">3.11.2. Parallel tests, classes and methods</a></li>
</ul>
</li>
<li><a href="#_rerunning_failed_tests">3.12. Rerunning failed tests</a></li>
<li><a href="#_junit_tests">3.13. JUnit tests</a></li>
<li><a href="#_beanshell_and_advanced_group_selection">3.14. BeanShell and advanced group selection</a>
<ul class="sectlevel3">
<li><a href="#_annotation_transformers">3.14.1. Annotation Transformers</a></li>
<li><a href="#_via_xml_suite_file">3.14.2. Via xml suite file</a></li>
<li><a href="#_via_command_line_arguments">3.14.3. Via command line arguments</a></li>
<li><a href="#_via_your_code">3.14.4. Via your code</a></li>
</ul>
</li>
<li><a href="#_method_interceptors">3.15. Method Interceptors</a></li>
<li><a href="#_interceptors_for_data_providers">3.16. Interceptors for Data Providers</a></li>
<li><a href="#_testng_listeners">3.17. TestNG Listeners</a>
<ul class="sectlevel3">
<li><a href="#_listener_list">3.17.1. Listener list</a></li>
<li><a href="#_specifying_listeners_with_testng_xml_or_in_java">3.17.2. Specifying listeners with testng.xml or in Java</a></li>
<li><a href="#_specifying_listeners_with_serviceloader">3.17.3. Specifying listeners with ServiceLoader</a></li>
<li><a href="#_ordering_listeners_in_testng">3.17.4. Ordering listeners in TestNG</a></li>
</ul>
</li>
<li><a href="#_dependency_injection">3.18. Dependency injection</a>
<ul class="sectlevel3">
<li><a href="#_native_dependency_injection">3.18.1. Native dependency injection</a></li>
<li><a href="#_guice_dependency_injection">3.18.2. Guice dependency injection</a></li>
</ul>
</li>
<li><a href="#_listening_to_testng_lifecycle_events">3.19. Listening to TestNG lifecycle events</a></li>
<li><a href="#_listening_to_method_invocations">3.20. Listening to method invocations</a></li>
<li><a href="#_listening_to_configuration_invocations">3.21. Listening to configuration invocations</a></li>
<li><a href="#_listening_to_class_level_invocations">3.22. Listening to class level invocations</a></li>
<li><a href="#_listening_to_data_provider_invocations">3.23. Listening to data provider invocations</a></li>
<li><a href="#_listening_to_suite_level_invocations">3.24. Listening to Suite level invocations</a></li>
<li><a href="#_overriding_test_methods">3.25. Overriding test methods</a></li>
<li><a href="#_overriding_configuration_methods">3.26. Overriding configuration methods</a></li>
<li><a href="#_altering_suites_or_tests">3.27. Altering suites (or) tests</a></li>
</ul>
</li>
<li><a href="#_test_results">4. Test results</a>
<ul class="sectlevel2">
<li><a href="#_success_failure_and_assert">4.1. Success, failure and assert</a></li>
<li><a href="#_logging_and_results">4.2. Logging and results</a></li>
<li><a href="#_logging_listeners">4.3. Logging Listeners</a></li>
<li><a href="#_logging_reporters">4.4. Logging Reporters</a></li>
<li><a href="#_junit_reports">4.5. JUnit Reports</a></li>
<li><a href="#_reporter_api">4.6. Reporter API</a></li>
<li><a href="#_xml_reports">4.7. XML Reports</a></li>
<li><a href="#_testng_exit_codes">4.8. TestNG Exit Codes</a></li>
</ul>
</li>
<li><a href="#_yaml">5. YAML</a></li>
<li><a href="#_dry_run_for_your_tests">6. Dry Run for your tests</a></li>
<li><a href="#_jvm_arguments_in_testng">7. JVM Arguments in TestNG</a></li>
<li><a href="#_logging_framework_integration_in_testng">8. Logging framework integration in TestNG</a></li>
<li><a href="#_migrating_from_junit">9. Migrating from JUnit</a>
<ul class="sectlevel2">
<li><a href="#_using_eclipse">9.1. Using Eclipse</a></li>
<li><a href="#_asserts">9.2. Asserts</a></li>
<li><a href="#_running_junit_tests">9.3. Running JUnit Tests</a></li>
<li><a href="#_related_reading">9.4. Related reading</a></li>
</ul>
</li>
<li><a href="#_javadocs">10. Javadocs</a></li>
<li><a href="#_running_testng_2">11. Running TestNG</a>
<ul class="sectlevel2">
<li><a href="#_eclipse_plug_in_2">11.1. Eclipse plug-in</a></li>
<li><a href="#_testng_idea_plug_in">11.2. TestNG IDEA Plug-in</a></li>
<li><a href="#_using_testng_with_gradle">11.3. Using TestNG with Gradle</a>
<ul class="sectlevel3">
<li><a href="#_specifying_your_gradle_build_file">11.3.1. Specifying your Gradle build file</a></li>
</ul>
</li>
<li><a href="#_using_testng_with_maven">11.4. Using TestNG with Maven</a>
<ul class="sectlevel3">
<li><a href="#_specifying_your_pom_xml">11.4.1. Specifying your pom.xml</a></li>
<li><a href="#_sample_reports">11.4.2. Sample reports</a></li>
</ul>
</li>
<li><a href="#_testng_ant_task">11.5. TestNG Ant Task</a></li>
<li><a href="#_running_testng_programmatically">11.6. Running TestNG programmatically</a></li>
</ul>
</li>
<li><a href="#_more_testng_reading">12. MORE TESTNG READING</a>
<ul class="sectlevel2">
<li><a href="#_articles_and_presentations">12.1. Articles and presentations</a>
<ul class="sectlevel3">
<li><a href="#_english">12.1.1. English</a></li>
<li><a href="#_french">12.1.2. French</a></li>
<li><a href="#_german">12.1.3. German</a></li>
</ul>
</li>
<li><a href="#_contributors">12.2. CONTRIBUTORS</a></li>
<li><a href="#_special_thanks">12.3. SPECIAL THANKS</a></li>
<li><a href="#_testimonies">12.4. TESTIMONIES</a></li>
</ul>
</li>
<li><a href="#_the_testng_textbook_next_generation_java_testing">13. The TestNG TextBook: Next Generation Java Testing</a>
<ul class="sectlevel2">
<li><a href="#_book_description">13.1. Book Description</a></li>
<li><a href="#_about_the_author">13.2. About the Author</a></li>
</ul>
</li>
<li><a href="#_reportng_a_reporting_tool">14. ReportNG - A reporting tool</a></li>
<li><a href="#_github_repository">15. GitHub Repository</a>
<ul class="sectlevel2">
<li><a href="#_testng_core">15.1. TestNG Core</a></li>
<li><a href="#_testng_documentation_2">15.2. TestNG Documentation</a></li>
<li><a href="#_testng_eclipse_plugin">15.3. TestNG Eclipse plugin</a></li>
<li><a href="#_reportng_a_reporting_tool_2">15.4. ReportNG - A reporting tool</a></li>
</ul>
</li>
</ul>
</div>
</div>
<div id="content">
<div class="sect1">
<h2 id="_welcome_to_testng">1. Welcome to TestNG</h2>
<div class="sectionbody">
<div class="paragraph">
<p>TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use, such as:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Annotations.</p>
</li>
<li>
<p>Run your tests in arbitrarily big thread pools with various policies available (all methods in their own thread, one thread per test class, etc…​).</p>
</li>
<li>
<p>Test that your code is multithread safe.</p>
</li>
<li>
<p>Flexible test configuration.</p>
</li>
<li>
<p>Support for data-driven testing (with @DataProvider).</p>
</li>
<li>
<p>Support for parameters.</p>
</li>
<li>
<p>Powerful execution model (no more TestSuite).</p>
</li>
<li>
<p>Supported by a variety of tools and plug-ins (Eclipse, IDEA, Maven, etc…​).</p>
</li>
<li>
<p>Embeds BeanShell for further flexibility.</p>
</li>
<li>
<p>Default JDK functions for runtime and logging (no dependencies).</p>
</li>
<li>
<p>Dependent methods for application server testing.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>TestNG is designed to cover all categories of tests: unit, functional, end-to-end, integration, etc…​</p>
</div>
<div class="paragraph">
<p>I started TestNG out of frustration for some JUnit deficiencies which I have documented on my weblog here and here Reading these entries might give you a better idea of the goal I am trying to achieve with TestNG. You can also check out a quick <a href="https://www.beust.com/weblog/announcing-testng-1-0/">overview of the main features</a> and an <a href="https://www.beust.com/weblog/using-annotation-inheritance-for-testing/">article</a> describing a very concrete example where the combined use of several TestNG’s features provides for a very intuitive and maintainable testing design.</p>
</div>
<div class="paragraph">
<p>Here is a very simple test:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="java">package example1;
import org.testng.annotations.*;
public class SimpleTest {
@BeforeClass
public void setUp() {
// code that will be invoked when this test is instantiated
}
@Test(groups = {"fast"})
public void aFastTest() {
System.out.println("Fast test");
}
@Test(groups = {"slow"})
public void aSlowTest() {
System.out.println("Slow test");
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>The method <code>setUp()</code> will be invoked after the test class has been built and before any test method is run. In this example, we will be running the group fast, so <code>aFastTest()</code> will be invoked while <code>aSlowTest()</code> will be skipped.</p>
</div>
<div class="paragraph">
<p>Things to note:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>No need to extend a class or implement an interface.</p>
</li>
<li>
<p>Even though the example above uses the JUnit conventions, our methods can be called any name you like, it’s the annotations that tell TestNG what they are.</p>
</li>
<li>
<p>A test method can belong to one or several groups.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Once you have compiled your test class into the build directory, you can invoke your test with the command line, an ant task (shown below) or an XML file:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><project default="test">
<path id="cp">
<pathelement location="lib/testng-testng-5.13.1.jar"/>
<pathelement location="build"/>
</path>
<taskdef name="testng" classpathref="cp"
classname="org.testng.TestNGAntTask"/>
<target name="test">
<testng classpathref="cp" groups="fast">
<classfileset dir="build" includes="example1/*.class"/>
</testng>
</target>
</project></code></pre>
</div>
</div>
<div class="paragraph">
<p>Use ant to invoke it:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="shell">c:> ant
Buildfile: build.xml
test:
[testng] Fast test
[testng] ===============================================
[testng] Suite for Command line test
[testng] Total tests run: 1, Failures: 0, Skips: 0
[testng] ===============================================
BUILD SUCCESSFUL
Total time: 4 seconds</code></pre>
</div>
</div>
<div class="paragraph">
<p>Then you can browse the result of your tests:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="shell">start test-output\index.html (on Windows)</code></pre>
</div>
</div>
<div class="sect2">
<h3 id="_requirements">1.1. Requirements</h3>
<div class="ulist">
<ul>
<li>
<p>TestNG Upto v7.5: JDK 8.</p>
</li>
<li>
<p>TestNG v7.6.0 and above: JDK 11 or higher.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_mailing_lists">1.2. Mailing-lists</h3>
<div class="ulist">
<ul>
<li>
<p>The users mailing-list can be found on <a href="https://groups.google.com/group/testng-users">Google Groups</a>.</p>
</li>
<li>
<p>If you are interested in working on TestNG itself, join the <a href="https://groups.google.com/group/testng-dev">developer mailing-list</a>.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_locations_of_the_projects">1.3. Locations of the projects</h3>
<div class="paragraph">
<p>If you are interested in contributing to TestNG or one of the IDE plug-ins, you will find them in the following locations:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/testng-team/testng">TestNG</a></p>
</li>
<li>
<p><a href="https://github.com/testng-team/testng-eclipse/">Eclipse plug-in</a></p>
</li>
<li>
<p><a href="https://github.com/JetBrains/intellij-community/tree/master/plugins/testng">IDEA IntelliJ plug-in</a></p>
</li>
<li>
<p><a href="https://marketplace.visualstudio.com/items?itemName=vscjava.vscode-java-test">Visual Studio Code plugin-in</a></p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_bug_reports">1.4. Bug reports</h3>
<div class="paragraph">
<p>If you think you found a bug, here is how to report it:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Create a small project that will allow us to reproduce this bug. In most cases, one or two Java source files and a testng.xml file should be sufficient. Then you can either zip it and email it to the <a href="https://groups.google.com/group/testng-dev">testng-dev mailing-list</a> or make it available on an open source hosting site, such as <a href="https://github.com/">github</a> and email testng-dev so we know about it. Please make sure that this project is self contained so that we can build it right away (remove the dependencies on external or proprietary frameworks, etc…​).</p>
</li>
<li>
<p>If the bug you observed is on the Eclipse plug-in, make sure your sample project contains the <code>.project</code> and <code>.classpath</code> files.</p>
</li>
<li>
<p>File a bug.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>For more information, you can either <a href="https://testng.org/doc/download.html">download TestNG</a>, read the manual or browse the links at the top.</p>
</div>
</div>
<div class="sect2">
<h3 id="_license">1.5. License</h3>
<div class="paragraph">
<p><a href="https://testng.org/license">Apache 2.0</a></p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_download">2. Download</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_current_release_version">2.1. Current Release Version</h3>
<div class="sect3">
<h4 id="_maven">2.1.1. Maven</h4>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>7.9.0</version>
<scope>test</scope>
</dependency></code></pre>
</div>
</div>
</div>
<div class="sect3">
<h4 id="_gradle">2.1.2. Gradle</h4>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="groovy">dependencies {
testCompile 'org.testng:testng:7.9.0'
}</code></pre>
</div>
</div>
</div>
<div class="sect3">
<h4 id="_snapshots">2.1.3. Snapshots</h4>
<div class="paragraph">
<p>TestNG automatically uploads <a href="https://oss.sonatype.org/content/repositories/snapshots/org/testng/testng/">snapshots to Sonatype</a> which you can access by adding the following repository:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="groovy">repositories {
maven {
url 'https://oss.sonatype.org/content/repositories/snapshots'
}
}</code></pre>
</div>
</div>
</div>
<div class="sect3">
<h4 id="_eclipse_plug_in">2.1.4. Eclipse plug-in</h4>
<div class="paragraph">
<p>See <a href="https://testng.org/testng-eclipse/download">the related documentation</a>.</p>
</div>
</div>
<div class="sect3">
<h4 id="_build_testng_from_source_code">2.1.5. Build TestNG from source code</h4>
<div class="paragraph">
<p>TestNG is also hosted on GitHub, where you can download the source and build the distribution yourself:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="shell">$ git clone git://github.com/testng-team/testng.git
$ cd testng
$ ./gradlew build</code></pre>
</div>
</div>
<div class="paragraph">
<p>You will then find the jar file in the <code>testng/build/libs</code> directory</p>
</div>
<div class="paragraph">
<p>Some useful tasks:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>./gradlew tasks</code> - to see what all tasks are available</p>
</li>
<li>
<p><code>./gradlew test</code> - to kick off the tests (incremental build)</p>
</li>
<li>
<p><code>./gradlew clean test</code> - if you want to have a clean build.</p>
</li>
<li>
<p><code>./gradlew autostyleApply</code> - Applies code formatting steps to sourcecode in-place.</p>
</li>
<li>
<p><code>./gradlew autostyleCheck</code> - Checks that sourcecode satisfies formatting steps.</p>
</li>
<li>
<p><code>./gradlew check</code> - Runs the below checks:</p>
<div class="ulist">
<ul>
<li>
<p>"Applies code formatting steps to sourcecode in-place." and</p>
</li>
<li>
<p>"Checks that sourcecode satisfies formatting steps.".</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="_build_the_testng_eclipse_plugin_from_source_code">2.1.6. Build the TestNG Eclipse Plugin from source code</h4>
<div class="paragraph">
<p>TestNG Eclipse Plugin is <a href="https://github.com/testng-team/testng-eclipse">hosted on GitHub</a>, you can download the source code and <a href="https://github.com/testng-team/testng-eclipse/blob/master/README.md#building">build by ourselves</a>.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="_testng_documentation">3. TestNG Documentation</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_introduction">3.1. Introduction</h3>
<div class="paragraph">
<p>TestNG is a testing framework designed to simplify a broad range of testing needs, from unit testing (testing a class in isolation of the others) to integration testing (testing entire systems made of several classes, several packages and even several external frameworks, such as application servers).</p>
</div>
<div class="paragraph">
<p>Writing a test is typically a three-step process:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Write the business logic of your test and insert <a href="annotations.html">TestNG annotations</a> in your code.</p>
</li>
<li>
<p>Add the information about your test (e.g. the class name, the groups you wish to run, etc…​) in a testng.xml file or in <code>build.xml</code>.</p>
</li>
<li>
<p><a href="../ant.html">Run TestNG</a>.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>You can find a quick example on the <a href="../welcome.html">Welcome page</a>.</p>
</div>
<div class="paragraph">
<p>The concepts used in this documentation are as follows:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>A suite is represented by one XML file. It can contain one or more tests and is defined by the <code><suite></code> tag.</p>
</li>
<li>
<p>A test is represented by <test> and can contain one or more TestNG classes.</p>
</li>
<li>
<p>A TestNG class is a Java class that contains at least one TestNG annotation. It is represented by the <class> tag and can contain one or more test methods.</p>
</li>
<li>
<p>A test method is a Java method annotated by <code>@Test</code> in your source.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>A TestNG test can be configured by <code>@BeforeXXX</code> and <code>@AfterXXX</code> annotations which allows to perform some Java logic before and after a certain point, these points being either of the items listed above.</p>
</div>
<div class="paragraph">
<p>The rest of this manual will explain the following:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>A list of all the annotations with a brief explanation. This will give you an idea of the various functionalities offered by TestNG but you will probably want to consult the section dedicated to each of these annotations to learn the details.</p>
</li>
<li>
<p>A description of the testng.xml file, its syntax and what you can specify in it.</p>
</li>
<li>
<p>A detailed list of the various features and how to use them with a combination of annotations and testng.xml.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_annotations">3.2. Annotations</h3>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Type</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotations</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Description</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation</p></td>
<td class="tableblock halign-left valign-top"><div class="content"><div class="ulist">
<ul>
<li>
<p><code>@BeforeSuite</code></p>
</li>
<li>
<p><code>@AfterSuite</code></p>
</li>
<li>
<p><code>@BeforeTest</code></p>
</li>
<li>
<p><code>@AfterTest</code></p>
</li>
<li>
<p><code>@BeforeGroups</code></p>
</li>
<li>
<p><code>@AfterGroups</code></p>
</li>
<li>
<p><code>@BeforeClass</code></p>
</li>
<li>
<p><code>@AfterClass</code></p>
</li>
<li>
<p><code>@BeforeMethod</code></p>
</li>
<li>
<p><code>@AfterMethod</code></p>
</li>
</ul>
</div></div></td>
<td class="tableblock halign-left valign-top"><div class="content"><div class="paragraph">
<p>Configuration information for a TestNG class:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>@BeforeSuite</code>: The annotated method will be run before all tests in this suite have run.</p>
</li>
<li>
<p><code>@AfterSuite</code>: The annotated method will be run after all tests in this suite have run.</p>
</li>
<li>
<p><code>@BeforeTest</code>: The annotated method will be run before any test method belonging to the classes inside the <test> tag is run.</p>
</li>
<li>
<p><code>@AfterTest</code>: The annotated method will be run after all the test methods belonging to the classes inside the <test> tag have run.</p>
</li>
<li>
<p><code>@BeforeGroups</code>: The list of groups that this configuration method will run before. This method is guaranteed to run shortly before the first test method that belongs to any of these groups is invoked.</p>
</li>
<li>
<p><code>@AfterGroups</code>: The list of groups that this configuration method will run after. This method is guaranteed to run shortly after the last test method that belongs to any of these groups is invoked.</p>
</li>
<li>
<p><code>@BeforeClass</code>: The annotated method will be run before the first test method in the current class is invoked.</p>
</li>
<li>
<p><code>@AfterClass</code>: The annotated method will be run after all the test methods in the current class have been run.</p>
</li>
<li>
<p><code>@BeforeMethod</code>: The annotated method will be run before each test method.</p>
</li>
<li>
<p><code>@AfterMethod</code>: The annotated method will be run after each test method.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p><strong>Behaviour of annotations in superclass of a TestNG class</strong></p>
</div>
<div class="paragraph">
<p>The annotations above will also be honored (inherited) when placed on a superclass of a TestNG class. This is useful for example to centralize test setup for multiple test classes in a common superclass.</p>
</div>
<div class="paragraph">
<p>In that case, TestNG guarantees that the "@Before" methods are executed in inheritance order (highest superclass first, then going down the inheritance chain), and the "@After" methods in reverse order (going up the inheritance chain).</p>
</div></div></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" rowspan="7"><p class="tableblock">Annotation Attributes</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>alwaysRun</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">For before methods (beforeSuite, beforeTest, beforeTestClass and beforeTestMethod, but not beforeGroups): If set to true, this configuration method will be run regardless of what groups it belongs to.
For after methods (afterSuite, afterClass, …​): If set to true, this configuration method will be run even if one or more methods invoked previously failed or was skipped.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>dependsOnGroups</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of groups this method depends on.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>dependsOnMethods</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of methods this method depends on.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>enabled</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether methods on this class/method are enabled.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>groups</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of groups this class/method belongs to.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>inheritGroups</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If true, this method will belong to groups specified in the @Test annotation at the class level.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>onlyForGroups</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Only for @BeforeMethod and @AfterMethod. If specified, then this setup/teardown method will only be invoked if the corresponding test method belongs to one of the listed groups.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>@DataProvider</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Marks a method as supplying data for a test method. The annotated method must return an <code>Object[][]</code> where each <code>Object[]</code> can be assigned the parameter list of the test method. The @Test method that wants to receive data from this <code>DataProvider</code> needs to use a dataProvider name equals to the name of this annotation.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" rowspan="2"><p class="tableblock">Annotation Attributes</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>name</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of this data provider. If it’s not supplied, the name of this data provider will automatically be set to the name of the method.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>parallel</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If set to true, tests generated using this data provider are run in parallel. Default value is false.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>@Factory</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Marks a method as a factory that returns objects that will be used by TestNG as Test classes. The method must return <code>Object[]</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>@Listeners</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Defines listeners on a test class.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation Attributes</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>value</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">An array of classes that extend org.testng.ITestNGListener.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>@Parameters</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Describes how to pass parameters to a @Test method.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation Attributes</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>value</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of variables used to fill the parameters of this method.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>@Test</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Marks a class or a method as part of the test.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" rowspan="16"><p class="tableblock">Annotation Attributes</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>alwaysRun</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If set to true, this test method will always be run even if it depends on a method that failed.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>dataProvider</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of the data provider for this test method.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>dataProviderClass</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The class where to look for the data provider. If not specified, the data provider will be looked on the class of the current test method or one of its base classes. If this attribute is specified, the data provider method needs to be static on the specified class.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>dependsOnGroups</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of groups this method depends on.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>dependsOnMethods</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of methods this method depends on.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>description</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The description for this method.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>enabled</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether methods on this class/method are enabled.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>expectedExceptions</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of exceptions that a test method is expected to throw. If no exception or a different than one on this list is thrown, this test will be marked a failure.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>groups</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of groups this class/method belongs to.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>invocationCount</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The number of times this method should be invoked.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>invocationTimeOut</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum number of milliseconds this test should take for the cumulated time of all the invocationcounts. This attribute will be ignored if invocationCount is not specified.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>priority</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The priority for this test method. Lower priorities will be scheduled first.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>successPercentage</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The percentage of success expected from this method</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>singleThreaded</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If set to true, all the methods on this test class are guaranteed to run in the same thread, even if the tests are currently being run with <code>parallel="methods"</code>. This attribute can only be used at the class level and it will be ignored if used at the method level. Note: this attribute used to be called sequential (now deprecated).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>timeOut</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum number of milliseconds this test should take.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>threadPoolSize</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The size of the thread pool for this method. The method will be invoked from multiple threads as specified by invocationCount.
NOTE: this attribute is ignored if invocationCount is not specified</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="_testng_xml">3.3. <code>testng.xml</code></h3>
<div class="paragraph">
<p>You can invoke TestNG in several different ways:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>With a <code>testng.xml</code> file</p>
</li>
<li>
<p>With <code>ant</code></p>
</li>
<li>
<p>From the command line</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>This section describes the format of <code>testng.xml</code> (you will find documentation on ant and the command line below).</p>
</div>
<div class="paragraph">
<p>The current DTD for testng.xml can be found on the main Web site: <a href="https://testng.org/testng-1.0.dtd">testng-1.0.dtd</a>.
Here is an example testng.xml file:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite1" verbose="1">
<test name="Nopackage">
<classes>
<class name="NoPackageTest"/>
</classes>
</test>
<test name="Regression1">
<classes>
<class name="test.sample.ParameterSample"/>
<class name="test.sample.ParameterTest"/>
</classes>
</test>
</suite></code></pre>
</div>
</div>
<div class="paragraph">
<p>You can specify package names instead of class names:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite1" verbose="1">
<test name="Regression1">
<packages>
<package name="test.sample"/>
</packages>
</test>
</suite></code></pre>
</div>
</div>
<div class="paragraph">
<p>In this example, TestNG will look at all the classes in the package <code>test.sample</code> and will retain only classes that have TestNG annotations.</p>
</div>
<div class="paragraph">
<p>You can also specify groups and methods to be included and excluded:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><test name="Regression1">
<groups>
<run>
<exclude name="brokenTests"/>
<include name="checkinTests"/>
</run>
</groups>
<classes>
<class name="test.IndividualMethodsTest">
<methods>
<include name="testMethod"/>
</methods>
</class>
</classes>
</test></code></pre>
</div>
</div>
<div class="paragraph">
<p>You can also define new groups inside <code>testng.xml</code> and specify additional details in attributes, such as whether to run the tests in parallel, how many threads to use, whether you are running JUnit tests, etc…​</p>
</div>
<div class="paragraph">
<p>By default, TestNG will run your tests in the order they are found in the XML file. If you want the classes and methods listed in this file to be run in an unpredictable order, set the <code>preserve-order</code> attribute to false.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><test name="Regression1" preserve-order="false">
<classes>
<class name="test.Test1">
<methods>
<include name="m1"/>
<include name="m2"/>
</methods>
</class>
<class name="test.Test2"/>
</classes>
</test></code></pre>
</div>
</div>
<div class="paragraph">
<p>Please see the DTD for a complete list of the features, or read on.</p>
</div>
</div>
<div class="sect2">
<h3 id="_running_testng">3.4. Running TestNG</h3>
<div class="paragraph">
<p>TestNG can be invoked in different ways:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Command line</p>
</li>
<li>
<p><a href="../ant.html">ant</a></p>
</li>
<li>
<p><a href="../eclipse.html">Eclipse</a></p>