-
Notifications
You must be signed in to change notification settings - Fork 17
/
Copy pathdocumentation.html
4993 lines (4833 loc) · 219 KB
/
documentation.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="Asciidoctor 2.0.18">
<title>TestNG Documentation</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700">
<link rel="stylesheet" href="./asciidoctor.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/prettify/r298/prettify.min.css">
</head>
<body class="article toc2 toc-left">
<div id="header">
<div id="toc" class="toc2">
<div id="toctitle">Table of Contents</div>
<ul class="sectlevel1">
<li><a href="#_testng_documentation">TestNG Documentation</a>
<ul class="sectlevel2">
<li><a href="#_introduction">Introduction</a></li>
<li><a href="#_annotations">Annotations</a></li>
<li><a href="#_testng_xml"><code>testng.xml</code></a></li>
<li><a href="#_running_testng">Running TestNG</a></li>
<li><a href="#_test_methods_test_classes_and_test_groups">Test methods, Test classes and Test groups</a></li>
<li><a href="#_parameters">Parameters</a></li>
<li><a href="#_dependencies">Dependencies</a></li>
<li><a href="#_factories">Factories</a></li>
<li><a href="#_class_level_annotations">Class level annotations</a></li>
<li><a href="#_ignoring_tests">Ignoring tests</a></li>
<li><a href="#_parallelism_and_time_outs">Parallelism and time-outs</a></li>
<li><a href="#_rerunning_failed_tests">Rerunning failed tests</a></li>
<li><a href="#_junit_tests">JUnit tests</a></li>
<li><a href="#_beanshell_and_advanced_group_selection">BeanShell and advanced group selection</a></li>
<li><a href="#_method_interceptors">Method Interceptors</a></li>
<li><a href="#_interceptors_for_data_providers">Interceptors for Data Providers</a></li>
<li><a href="#_testng_listeners">TestNG Listeners</a></li>
<li><a href="#_dependency_injection">Dependency injection</a></li>
<li><a href="#_listening_to_testng_lifecycle_events">Listening to TestNG lifecycle events</a></li>
<li><a href="#_listening_to_method_invocations">Listening to method invocations</a></li>
<li><a href="#_listening_to_configuration_invocations">Listening to configuration invocations</a></li>
<li><a href="#_listening_to_class_level_invocations">Listening to class level invocations</a></li>
<li><a href="#_listening_to_data_provider_invocations">Listening to data provider invocations</a></li>
<li><a href="#_listening_to_suite_level_invocations">Listening to Suite level invocations</a></li>
<li><a href="#_overriding_test_methods">Overriding test methods</a></li>
<li><a href="#_overriding_configuration_methods">Overriding configuration methods</a></li>
<li><a href="#_altering_suites_or_tests">Altering suites (or) tests</a></li>
</ul>
</li>
<li><a href="#_test_results">Test results</a>
<ul class="sectlevel2">
<li><a href="#_success_failure_and_assert">Success, failure and assert</a></li>
<li><a href="#_logging_and_results">Logging and results</a></li>
<li><a href="#_logging_listeners">Logging Listeners</a></li>
<li><a href="#_logging_reporters">Logging Reporters</a></li>
<li><a href="#_junit_reports">JUnit Reports</a></li>
<li><a href="#_reporter_api">Reporter API</a></li>
<li><a href="#_xml_reports">XML Reports</a></li>
<li><a href="#_testng_exit_codes">TestNG Exit Codes</a></li>
</ul>
</li>
<li><a href="#_yaml">YAML</a></li>
<li><a href="#_dry_run_for_your_tests">Dry Run for your tests</a></li>
<li><a href="#_jvm_arguments_in_testng">JVM Arguments in TestNG</a></li>
<li><a href="#_logging_framework_integration_in_testng">Logging framework integration in TestNG</a></li>
</ul>
</div>
</div>
<div id="content">
<div class="sect1">
<h2 id="_testng_documentation">TestNG Documentation</h2>
<div class="sectionbody">
<div class="sect2">
<h3 id="_introduction">Introduction</h3>
<div class="paragraph">
<p>TestNG is a testing framework designed to simplify a broad range of testing needs, from unit testing (testing a class in isolation of the others) to integration testing (testing entire systems made of several classes, several packages and even several external frameworks, such as application servers).</p>
</div>
<div class="paragraph">
<p>Writing a test is typically a three-step process:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Write the business logic of your test and insert <a href="annotations.html">TestNG annotations</a> in your code.</p>
</li>
<li>
<p>Add the information about your test (e.g. the class name, the groups you wish to run, etc…​) in a testng.xml file or in <code>build.xml</code>.</p>
</li>
<li>
<p><a href="../ant.html">Run TestNG</a>.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>You can find a quick example on the <a href="../welcome.html">Welcome page</a>.</p>
</div>
<div class="paragraph">
<p>The concepts used in this documentation are as follows:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>A suite is represented by one XML file. It can contain one or more tests and is defined by the <code><suite></code> tag.</p>
</li>
<li>
<p>A test is represented by <test> and can contain one or more TestNG classes.</p>
</li>
<li>
<p>A TestNG class is a Java class that contains at least one TestNG annotation. It is represented by the <class> tag and can contain one or more test methods.</p>
</li>
<li>
<p>A test method is a Java method annotated by <code>@Test</code> in your source.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>A TestNG test can be configured by <code>@BeforeXXX</code> and <code>@AfterXXX</code> annotations which allows to perform some Java logic before and after a certain point, these points being either of the items listed above.</p>
</div>
<div class="paragraph">
<p>The rest of this manual will explain the following:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>A list of all the annotations with a brief explanation. This will give you an idea of the various functionalities offered by TestNG but you will probably want to consult the section dedicated to each of these annotations to learn the details.</p>
</li>
<li>
<p>A description of the testng.xml file, its syntax and what you can specify in it.</p>
</li>
<li>
<p>A detailed list of the various features and how to use them with a combination of annotations and testng.xml.</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="_annotations">Annotations</h3>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Type</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotations</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Description</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation</p></td>
<td class="tableblock halign-left valign-top"><div class="content"><div class="ulist">
<ul>
<li>
<p><code>@BeforeSuite</code></p>
</li>
<li>
<p><code>@AfterSuite</code></p>
</li>
<li>
<p><code>@BeforeTest</code></p>
</li>
<li>
<p><code>@AfterTest</code></p>
</li>
<li>
<p><code>@BeforeGroups</code></p>
</li>
<li>
<p><code>@AfterGroups</code></p>
</li>
<li>
<p><code>@BeforeClass</code></p>
</li>
<li>
<p><code>@AfterClass</code></p>
</li>
<li>
<p><code>@BeforeMethod</code></p>
</li>
<li>
<p><code>@AfterMethod</code></p>
</li>
</ul>
</div></div></td>
<td class="tableblock halign-left valign-top"><div class="content"><div class="paragraph">
<p>Configuration information for a TestNG class:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>@BeforeSuite</code>: The annotated method will be run before all tests in this suite have run.</p>
</li>
<li>
<p><code>@AfterSuite</code>: The annotated method will be run after all tests in this suite have run.</p>
</li>
<li>
<p><code>@BeforeTest</code>: The annotated method will be run before any test method belonging to the classes inside the <test> tag is run.</p>
</li>
<li>
<p><code>@AfterTest</code>: The annotated method will be run after all the test methods belonging to the classes inside the <test> tag have run.</p>
</li>
<li>
<p><code>@BeforeGroups</code>: The list of groups that this configuration method will run before. This method is guaranteed to run shortly before the first test method that belongs to any of these groups is invoked.</p>
</li>
<li>
<p><code>@AfterGroups</code>: The list of groups that this configuration method will run after. This method is guaranteed to run shortly after the last test method that belongs to any of these groups is invoked.</p>
</li>
<li>
<p><code>@BeforeClass</code>: The annotated method will be run before the first test method in the current class is invoked.</p>
</li>
<li>
<p><code>@AfterClass</code>: The annotated method will be run after all the test methods in the current class have been run.</p>
</li>
<li>
<p><code>@BeforeMethod</code>: The annotated method will be run before each test method.</p>
</li>
<li>
<p><code>@AfterMethod</code>: The annotated method will be run after each test method.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p><strong>Behaviour of annotations in superclass of a TestNG class</strong></p>
</div>
<div class="paragraph">
<p>The annotations above will also be honored (inherited) when placed on a superclass of a TestNG class. This is useful for example to centralize test setup for multiple test classes in a common superclass.</p>
</div>
<div class="paragraph">
<p>In that case, TestNG guarantees that the "@Before" methods are executed in inheritance order (highest superclass first, then going down the inheritance chain), and the "@After" methods in reverse order (going up the inheritance chain).</p>
</div></div></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" rowspan="7"><p class="tableblock">Annotation Attributes</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>alwaysRun</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">For before methods (beforeSuite, beforeTest, beforeTestClass and beforeTestMethod, but not beforeGroups): If set to true, this configuration method will be run regardless of what groups it belongs to.
For after methods (afterSuite, afterClass, …​): If set to true, this configuration method will be run even if one or more methods invoked previously failed or was skipped.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>dependsOnGroups</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of groups this method depends on.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>dependsOnMethods</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of methods this method depends on.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>enabled</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether methods on this class/method are enabled.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>groups</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of groups this class/method belongs to.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>inheritGroups</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If true, this method will belong to groups specified in the @Test annotation at the class level.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>onlyForGroups</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Only for @BeforeMethod and @AfterMethod. If specified, then this setup/teardown method will only be invoked if the corresponding test method belongs to one of the listed groups.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>@DataProvider</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Marks a method as supplying data for a test method. The annotated method must return an <code>Object[][]</code> where each <code>Object[]</code> can be assigned the parameter list of the test method. The @Test method that wants to receive data from this <code>DataProvider</code> needs to use a dataProvider name equals to the name of this annotation.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" rowspan="2"><p class="tableblock">Annotation Attributes</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>name</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of this data provider. If it’s not supplied, the name of this data provider will automatically be set to the name of the method.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>parallel</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If set to true, tests generated using this data provider are run in parallel. Default value is false.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>@Factory</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Marks a method as a factory that returns objects that will be used by TestNG as Test classes. The method must return <code>Object[]</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>@Listeners</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Defines listeners on a test class.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation Attributes</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>value</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">An array of classes that extend org.testng.ITestNGListener.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>@Parameters</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Describes how to pass parameters to a @Test method.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation Attributes</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>value</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of variables used to fill the parameters of this method.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Annotation</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>@Test</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Marks a class or a method as part of the test.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top" rowspan="16"><p class="tableblock">Annotation Attributes</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>alwaysRun</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If set to true, this test method will always be run even if it depends on a method that failed.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>dataProvider</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The name of the data provider for this test method.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>dataProviderClass</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The class where to look for the data provider. If not specified, the data provider will be looked on the class of the current test method or one of its base classes. If this attribute is specified, the data provider method needs to be static on the specified class.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>dependsOnGroups</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of groups this method depends on.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>dependsOnMethods</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of methods this method depends on.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>description</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The description for this method.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>enabled</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether methods on this class/method are enabled.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>expectedExceptions</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of exceptions that a test method is expected to throw. If no exception or a different than one on this list is thrown, this test will be marked a failure.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>groups</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of groups this class/method belongs to.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>invocationCount</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The number of times this method should be invoked.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>invocationTimeOut</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum number of milliseconds this test should take for the cumulated time of all the invocationcounts. This attribute will be ignored if invocationCount is not specified.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>priority</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The priority for this test method. Lower priorities will be scheduled first.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>successPercentage</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The percentage of success expected from this method</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>singleThreaded</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If set to true, all the methods on this test class are guaranteed to run in the same thread, even if the tests are currently being run with <code>parallel="methods"</code>. This attribute can only be used at the class level and it will be ignored if used at the method level. Note: this attribute used to be called sequential (now deprecated).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>timeOut</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The maximum number of milliseconds this test should take.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>threadPoolSize</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The size of the thread pool for this method. The method will be invoked from multiple threads as specified by invocationCount.
NOTE: this attribute is ignored if invocationCount is not specified</p></td>
</tr>
</tbody>
</table>
</div>
<div class="sect2">
<h3 id="_testng_xml"><code>testng.xml</code></h3>
<div class="paragraph">
<p>You can invoke TestNG in several different ways:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>With a <code>testng.xml</code> file</p>
</li>
<li>
<p>With <code>ant</code></p>
</li>
<li>
<p>From the command line</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>This section describes the format of <code>testng.xml</code> (you will find documentation on ant and the command line below).</p>
</div>
<div class="paragraph">
<p>The current DTD for testng.xml can be found on the main Web site: <a href="https://testng.org/testng-1.0.dtd">testng-1.0.dtd</a>.
Here is an example testng.xml file:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite1" verbose="1">
<test name="Nopackage">
<classes>
<class name="NoPackageTest"/>
</classes>
</test>
<test name="Regression1">
<classes>
<class name="test.sample.ParameterSample"/>
<class name="test.sample.ParameterTest"/>
</classes>
</test>
</suite></code></pre>
</div>
</div>
<div class="paragraph">
<p>You can specify package names instead of class names:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Suite1" verbose="1">
<test name="Regression1">
<packages>
<package name="test.sample"/>
</packages>
</test>
</suite></code></pre>
</div>
</div>
<div class="paragraph">
<p>In this example, TestNG will look at all the classes in the package <code>test.sample</code> and will retain only classes that have TestNG annotations.</p>
</div>
<div class="paragraph">
<p>You can also specify groups and methods to be included and excluded:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><test name="Regression1">
<groups>
<run>
<exclude name="brokenTests"/>
<include name="checkinTests"/>
</run>
</groups>
<classes>
<class name="test.IndividualMethodsTest">
<methods>
<include name="testMethod"/>
</methods>
</class>
</classes>
</test></code></pre>
</div>
</div>
<div class="paragraph">
<p>You can also define new groups inside <code>testng.xml</code> and specify additional details in attributes, such as whether to run the tests in parallel, how many threads to use, whether you are running JUnit tests, etc…​</p>
</div>
<div class="paragraph">
<p>By default, TestNG will run your tests in the order they are found in the XML file. If you want the classes and methods listed in this file to be run in an unpredictable order, set the <code>preserve-order</code> attribute to false.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><test name="Regression1" preserve-order="false">
<classes>
<class name="test.Test1">
<methods>
<include name="m1"/>
<include name="m2"/>
</methods>
</class>
<class name="test.Test2"/>
</classes>
</test></code></pre>
</div>
</div>
<div class="paragraph">
<p>Please see the DTD for a complete list of the features, or read on.</p>
</div>
</div>
<div class="sect2">
<h3 id="_running_testng">Running TestNG</h3>
<div class="paragraph">
<p>TestNG can be invoked in different ways:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Command line</p>
</li>
<li>
<p><a href="../ant.html">ant</a></p>
</li>
<li>
<p><a href="../eclipse.html">Eclipse</a></p>
</li>
<li>
<p><a href="../idea.html">IntelliJ’s IDEA</a></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>This section only explains how to invoke TestNG from the command line.
Please click on one of the links above if you are interested in one of the other ways.</p>
</div>
<div class="paragraph">
<p>Assuming that you have TestNG in your class path, the simplest way to invoke TestNG is as follows:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="shell">java org.testng.TestNG testng1.xml [testng2.xml testng3.xml ...]</code></pre>
</div>
</div>
<div class="paragraph">
<p>You need to specify at least one XML file describing the TestNG suite you are trying to run.
Additionally, the following command-line switches are available:</p>
</div>
<div class="sect3">
<h4 id="_command_line_parameters">Command Line Parameters</h4>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Option</th>
<th class="tableblock halign-left valign-top">Argument</th>
<th class="tableblock halign-left valign-top">Documentation</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-configfailurepolicy</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>skip</code>,<code>continue</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether TestNG should continue to execute the remaining tests in the suite or skip them if an @Before* method fails. Default behavior is <code>skip</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-d</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A directory</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The directory where the reports will be generated (defaults to <code>test-output</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-dataproviderthreadcount</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The default number of threads to use for data providers when running tests in parallel.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This sets the default maximum number of threads to use for data providers when running tests in parallel. It will only take effect if the parallel mode has been selected (for example, with the <code>-parallel</code> option). This can be overridden in the suite definition.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-excludegroups</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A comma-separated list of groups.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of groups you want to be excluded from this run.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-groups</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A comma-separated list of groups.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The list of groups you want to run (e.g. "windows,linux,regression").</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-listener</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A comma-separated list of Java classes that can be found on your classpath.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Lets you specify your own test listeners. The classes need to implement <code>org.testng.ITestListener</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-usedefaultlisteners</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether to use the default listeners</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-methods</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A comma separated list of fully qualified class name and method. For example <code>com.example.Foo.f1,com.example.Bar.f2</code>.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Lets you specify individual methods to run.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-methodselectors</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A comma-separated list of Java classes and method priorities that define method selectors.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Lets you specify method selectors on the command line. For example: <code>com.example.Selector1:3,com.example.Selector2:2</code></p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-parallel</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>methods,tests,classes</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If specified, sets the default mechanism used to determine how to use parallel threads when running tests. If not set, default mechanism is not to use parallel threads at all. This can be overridden in the suite definition.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-reporter</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The extended configuration for a custom report listener.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Similar to the <code>-listener</code> option, except that it allows the configuration of JavaBeans-style properties on the reporter instance.
Example: <code>-reporter com.test.MyReporter:methodFilter=<strong>insert</strong>,enableFiltering=true</code>
You can have as many occurrences of this option, one for each reporter that needs to be added.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-sourcedir</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A semicolon separated list of directories.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The directories where your javadoc annotated test sources are. This option is only necessary if you are using javadoc type annotations. (e.g. <code>src/test</code> or <code>src/test/org/testng/eclipse-plugin;src/test/org/testng/testng</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-suitename</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The default name to use for a test suite.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This specifies the suite name for a test suite defined on the command line. This option is ignored if the <code>suite.xml</code> file or the source code specifies a different suite name. It is possible to create a suite name with spaces in it if you surround it with double-quotes <code>"like this"</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-testclass</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A comma-separated list of classes that can be found in your classpath.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A list of class files separated by commas (e.g. <code>org.foo.Test1,org.foo.test2</code>).</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-testjar</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A jar file.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Specifies a jar file that contains test classes. If a <code>testng.xml</code> file is found at the root of that jar file, it will be used, otherwise, all the test classes found in this jar file will be considered test classes.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-testname</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The default name to use for a test.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This specifies the name for a test defined on the command line. This option is ignored if the suite.xml file or the source code specifies a different test name. It is possible to create a test name with spaces in it if you surround it with double-quotes <code>"like this"</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-testnames</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A comma separated list of test names.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Only tests defined in a <test> tag matching one of these names will be run.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-testrunfactory</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A Java classes that can be found on your classpath.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Lets you specify your own test runners. The class needs to implement <code>org.testng.ITestRunnerFactory</code>.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-threadcount</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The default number of threads to use when running tests in parallel.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This sets the default maximum number of threads to use for running tests in parallel. It will only take effect if the parallel mode has been selected (for example, with the -parallel option). This can be overridden in the suite definition.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-xmlpathinjar</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">The path of the XML file inside the jar file.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">This attribute should contain the path to a valid XML file inside the test jar (e.g. <code>resources/testng.xml</code>). The default is <code>testng.xml</code>, which means a file called <code>testng.xml</code> at the root of the jar file. This option will be ignored unless <code>-testjar</code> is specified.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-shareThreadPoolForDataProviders</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Indicates if TestNG should use a global shared thread-pool (at suite level) for running data driven tests. TestNG will consider the value set for the configuration parameter <code>-dataproviderthreadcount</code> as the size of the thread pool.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-useGlobalThreadPool</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Indicates if TestNG should use a global shared thread-pool (at suite level) for running regular and data driven tests. TestNG will consider the value set for the configuration parameter <code>-threadcount</code> as the size of the thread pool.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-log</code> (or) <code>-verbose</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">a valid log level</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Level of verbosity to be used when logging messages.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-junit</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Should TestNG run in JUnit mode.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-mixed</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Mixed mode - autodetect the type of current test and run it with appropriate runner.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-objectfactory</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A string that represents a fully qualified class name.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Fully qualified class name that implements <code>org.testng.ITestObjectFactory</code> which can be used to create test class and listener instances.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-ignoreMissedTestNames</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Ignore missed test names given by <code>-testnames</code> and continue to run existing tests, if any.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-skipfailedinvocationcounts</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Should TestNG skip failed invocation counts for data driven tests and tests driven by invocation counts.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-testRunFactory</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A string that represents a fully qualified class name.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Fully qualified class name that implements <code>org.testng.ITestRunnerFactory</code> which can be used to create custom test runners for running tests.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-suitethreadpoolsize</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">An integer value that represents the thread pool size. When not specified, defaults to <code>1</code>.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Size of the thread pool to use to run suites.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-randomizesuites</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Whether to run suites in same order as specified in XML or not.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-alwaysrunlisteners</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Should method invocation listeners be run even for skipped methods.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-dependencyinjectorfactory</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A string that represents a fully qualified class name.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Fully qualified class name that implements org.testng.IInjectorFactory which can be used to handle with dependency injection.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">`-failwheneverythingskipped'</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Should TestNG fail execution if all tests were skipped and nothing was run.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-spilistenerstoskip</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A comma separated string that represents a fully qualified class name(s).</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Comma separated fully qualified class names of listeners that should be skipped from being wired in via Service Loaders.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-overrideincludedmethods</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Should TestNG exclude explicitly included test methods if they belong to any excluded groups as defined in the suite xml file.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-includeAllDataDrivenTestsWhenSkipping</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Should TestNG report all iterations of a data driven test as individual skips, in-case of upstream failures.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-propagateDataProviderFailureAsTestFailure</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Should TestNG consider failures in Data Providers as test failures.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-generateResultsPerSuite</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Should TestNG generate results on a per suite basis by creating a sub directory for each suite and dumping results into it.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-shareThreadPoolForDataProviders</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Should TestNG use a global Shared ThreadPool (At suite level) for running data providers.</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>-useGlobalThreadPool</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>true</code>,<code>false</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Should TestNG use a global Shared ThreadPool (At suite level) for running regular and data driven tests.</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>This documentation can be obtained by invoking TestNG without any arguments.</p>
</div>
<div class="paragraph">
<p>You can also put the command line switches in a text file, say <code>c:\command.txt</code>, and tell TestNG to use that file to retrieve its parameters:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="shell">C:> more c:\command.txt
-d test-output testng.xml
C:> java org.testng.TestNG @c:\command.txt</code></pre>
</div>
</div>
<div class="paragraph">
<p>Additionally, TestNG can be passed properties on the command line of the Java Virtual Machine, for example</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="shell">java -Dtestng.test.classpath="c:/build;c:/java/classes;" org.testng.TestNG testng.xml</code></pre>
</div>
</div>
<div class="paragraph">
<p>Here are the properties that TestNG understands:</p>
</div>
</div>
<div class="sect3">
<h4 id="_system_properties">System properties</h4>
<table class="tableblock frame-all grid-all stretch">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Property</th>
<th class="tableblock halign-left valign-top">Type</th>
<th class="tableblock halign-left valign-top">Documentation</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock"><code>testng.test.classpath</code></p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">A semicolon separated series of directories that contain your test classes.</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">If this property is set, TestNG will use it to look for your test classes instead of the class path. This is convenient if you are using the package tag in your XML file and you have a lot of classes in your classpath, most of them not being test classes.</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p><strong>Example:</strong></p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="shell">java org.testng.TestNG -groups windows,linux -testclass org.test.MyTest</code></pre>
</div>
</div>
<div class="paragraph">
<p>The ant task and <code>testng.xml</code> allow you to launch TestNG with more parameters (methods to include, specifying parameters, etc…​), so you should consider using the command line only when you are trying to learn about TestNG and you want to get up and running quickly.</p>
</div>
<div class="admonitionblock tip">
<table>
<tr>
<td class="icon">
<i class="fa icon-tip" title="Tip"></i>
</td>
<td class="content">
The command line flags that specify what tests should be run will be ignored if you also specify a testng.xml file, with the exception of <code>-groups</code> and <code>-excludegroups</code>, which will override all the group inclusions/exclusions found in <code>testng.xml</code>.
</td>
</tr>
</table>
</div>
</div>
<div class="sect3">
<h4 id="_running_tests_from_within_a_test_jar">Running tests from within a test jar</h4>
<div class="paragraph">
<p>TestNG can be provided with a jar that contains your test classes and you can execute the tests from within it.</p>
</div>
<div class="paragraph">
<p>Let’s see an example (We are going to use Maven for this example.)</p>
</div>
<div class="paragraph">
<p>We will be using <a href="https://maven.apache.org/plugins/maven-assembly-plugin/usage.html">maven-assembly-plugin</a> and <a href="https://maven.apache.org/plugins/maven-jar-plugin/usage.html">maven-jar-plugin</a> to demonstrate how to do this.</p>
</div>
<div class="paragraph">
<p>The project’s directory structure will look like below:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="bash">.
├── pom.xml
└── src
└── test
├── java
│ └── org
│ └── testng
│ ├── FirstTestCase.java
│ └── SecondTestCase.java
└── resources
├── suites
│ ├── suite1.xml
│ └── suite2.xml
└── test-jar-with-dependencies.xml</code></pre>
</div>
</div>
<div class="paragraph">
<p>The relevant <code><dependencies></code> section and the <code><plugins></code> section will look like below:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><dependencies>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>7.9.0</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<version>3.3.0</version>
<executions>
<execution>
<goals>
<goal>test-jar</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.6.0</version>
<configuration>
<descriptors>
<descriptor>src/test/resources/test-jar-with-dependencies.xml</descriptor>
</descriptors>
<archive>
<!-- We would like to create an executable jar so that we can execute it directly -->
<manifest>
<mainClass>org.testng.TestNG</mainClass>
</manifest>
</archive>
</configuration>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build></code></pre>
</div>
</div>
<div class="paragraph">
<p>The contents of <code>src/test/resources/test-jar-with-dependencies.xml</code> will look like below:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><assembly xmlns="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 http://maven.apache.org/xsd/assembly-1.1.0.xsd
https://maven.apache.org/plugins/maven-assembly-plugin/assembly/1.1.0 ">
<id>test-jar-with-dependencies</id>
<formats>
<format>jar</format>
</formats>
<includeBaseDirectory>false</includeBaseDirectory>
<dependencySets>
<dependencySet>
<outputDirectory>/</outputDirectory>
<useProjectArtifact>true</useProjectArtifact>
<useProjectAttachments>true</useProjectAttachments>
<unpack>true</unpack>
<scope>test</scope>
</dependencySet>
</dependencySets>
</assembly></code></pre>
</div>
</div>
<div class="paragraph">
<p>Now let’s build the jars using the command <code>mvn clean package</code>.</p>
</div>
<div class="paragraph">
<p>Now in order to run the tests, use the below command:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="bash">java -jar target/uber-testjar-demo-1.0-SNAPSHOT-test-jar-with-dependencies.jar -testjar target/uber-testjar-demo-1.0-SNAPSHOT-tests.jar</code></pre>
</div>
</div>
<div class="paragraph">
<p>This command causes TestNG to look for test classes in the jar and it executes all of them.</p>
</div>
<div class="paragraph">
<p>Here:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>target/uber-testjar-demo-1.0-SNAPSHOT-test-jar-with-dependencies.jar</code> - Represents the uber/fat jar which contains all the dependencies inside it.</p>
</li>
<li>
<p><code>target/uber-testjar-demo-1.0-SNAPSHOT-tests.jar</code> - Contains all the tests that we created.</p>
</li>
<li>
<p>-<code>-testjar</code> - This argument informs TestNG that it should look for test classes inside the jar and NOT in the current CLASSPATH.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>If you would like to execute a specific suite file that exists in the jar, then use the below command:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="bash">java -jar target/uber-testjar-demo-1.0-SNAPSHOT-test-jar-with-dependencies.jar -testjar target/uber-testjar-demo-1.0-SNAPSHOT-tests.jar -xmlpathinjar suites/suite2.xml</code></pre>
</div>
</div>
<div class="paragraph">
<p>Here we are specifying the path to the suite file using the command line argument <code>-xmlpathinjar</code>.</p>
</div>
<div class="paragraph">
<p>For more details, refer to <a href="https://stackoverflow.com/a/11787964">this stackoverflow post</a>.</p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="_test_methods_test_classes_and_test_groups">Test methods, Test classes and Test groups</h3>
<div class="sect3">
<h4 id="_test_methods">Test methods</h4>
<div class="paragraph">
<p>Test methods are annotated with @Test. Methods annotated with @Test that happen to return a value will be ignored, unless you set allow-return-values to true in your testng.xml:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><suite allow-return-values="true">
<!-- rest of the contents ignored for brevity -->
</suite></code></pre>
</div>
</div>
<div class="paragraph">
<p>or</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><test allow-return-values="true">