-
Notifications
You must be signed in to change notification settings - Fork 17
/
Copy pathparameters.html
516 lines (505 loc) · 20.3 KB
/
parameters.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="Asciidoctor 2.0.18">
<title>Parameters</title>
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Open+Sans:300,300italic,400,400italic,600,600italic%7CNoto+Serif:400,400italic,700,700italic%7CDroid+Sans+Mono:400,700">
<link rel="stylesheet" href="./asciidoctor.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/prettify/r298/prettify.min.css">
</head>
<body class="article toc2 toc-left">
<div id="header">
<div id="toc" class="toc2">
<div id="toctitle">Table of Contents</div>
<ul class="sectlevel2">
<li><a href="#_parameters">Parameters</a></li>
</ul>
</div>
</div>
<div id="content">
<div class="sect2">
<h3 id="_parameters">Parameters</h3>
<div class="literalblock">
<div class="content">
<pre>TestNG allows you to pass an arbitrary number of parameters to each of your tests using the `@Parameters` annotation.</pre>
</div>
</div>
<div class="paragraph">
<p>There are three ways to set these parameters</p>
</div>
<div class="ulist">
<ul>
<li>
<p>The testng.xml file</p>
</li>
<li>
<p>Programmatically</p>
</li>
<li>
<p>Java system properties</p>
</li>
</ul>
</div>
<div class="sect3">
<h4 id="_parameters_from_testng_xml">Parameters from testng.xml</h4>
<div class="paragraph">
<p>If you are using simple values for your parameters, you can specify them in your <code>testng.xml</code>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="java">@Parameters({ "first-name" })
@Test
public void testSingleString(String firstName) {
System.out.println("Invoked testString " + firstName);
assert "Cedric".equals(firstName);
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>In this code, we specify that the parameter <code>firstName</code> of your Java method should receive the value of the XML parameter called <code>first-name</code>. This XML parameter is defined in <code>testng.xml</code>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><suite name="My suite">
<parameter name="first-name" value="Cedric"/>
<test name="Simple example">
<!-- ... -->
</test>
</suite></code></pre>
</div>
</div>
<div class="paragraph">
<p>The same technique can be used for <code>@Before</code>/<code>@After</code> and <code>@Factory</code> annotations:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="java">@Parameters({ "datasource", "jdbc-driver" })
@BeforeMethod
public void beforeTest(String ds, String driver) {
m_dataSource = buildDataSource(); // look up the value of datasource
m_jdbcDriver = driver;
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>This time, the two Java parameter <code>ds</code> and <code>driver</code> will receive the value given to the properties <code>datasource</code> and <code>jdbc-driver</code> respectively.</p>
</div>
<div class="paragraph">
<p>Parameters can be declared optional with the {javadocs-base-url}/org/testng/annotations/Optional.html[org.testng.annotations.Optional] annotation:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="java">@Parameters("db")
@Test
public void testNonExistentParameter(@Optional("mysql") String db) {
//more code
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>If no parameter named "db" is found in your <code>testng.xml</code> file, your test method will receive the default value specified inside the <code>@Optional</code> annotation: "mysql".</p>
</div>
<div class="paragraph">
<p>The <code>@Parameters</code> annotation can be placed at the following locations:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>On any method that already has a <code>@Test</code>, <code>@Before</code>/<code>@After</code> or <code>@Factory</code> annotation.</p>
</li>
<li>
<p>On at most one constructor of your test class. In this case, TestNG will invoke this particular constructor with the parameters initialized to the values specified in <code>testng.xml</code> whenever it needs to instantiate your test class. This feature can be used to initialize fields inside your classes to values that will then be used by your test methods.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p><strong>Important:</strong></p>
</div>
<div class="ulist">
<ul>
<li>
<p>The XML parameters are mapped to the Java parameters in the same order as they are found in the annotation, and TestNG will issue an error if the numbers don’t match.</p>
</li>
<li>
<p>Parameters are scoped. In testng.xml, you can declare them either under:</p>
<div class="ulist">
<ul>
<li>
<p><code><suite></code> tag or</p>
</li>
<li>
<p><code><test></code> tag or</p>
</li>
<li>
<p><code><class></code> tag or</p>
</li>
<li>
<p><code><methods></code> tag.</p>
</li>
</ul>
</div>
</li>
<li>
<p>The order of precedence (lowest to highest) in terms of resolving values for parameters with same names is</p>
</li>
</ul>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="bash"><suite> --> <test> --> <class> --> <methods></code></pre>
</div>
</div>
<div class="ulist">
<ul>
<li>
<p>If two parameters have the same name, it’s the one defined in <code><methods></code> that has precedence. This is convenient if you need to specify a parameter applicable to all your tests and override its value only for certain test method.</p>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="_parameters_with_dataproviders">Parameters with DataProviders</h4>
<div class="paragraph">
<p>Specifying parameters in testng.xml might not be sufficient if you need to pass complex parameters, or parameters that need to be created from Java (complex objects, objects read from a property file or a database, etc…​). In this case, you can use a Data Provider to supply the values you need to test. A Data Provider is a method on your class that returns an array of array of objects. This method is annotated with <code>@DataProvider</code>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="java">//This method will provide data to any test method that declares that its Data Provider
//is named "test1"
@DataProvider(name = "test1")
public Object[][] createData1() {
return new Object[][] {
{ "Cedric", 36 },
{ "Anne", 37},
};
}
//This test method declares that its data should be supplied by the Data Provider
//named "test1"
@Test(dataProvider = "test1")
public void verifyData1(String n1, Integer n2) {
System.out.println(n1 + " " + n2);
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>will print</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="bash">Cedric 36
Anne 37</code></pre>
</div>
</div>
<div class="paragraph">
<p>A <code>@Test</code> method specifies its Data Provider with the <code>dataProvider</code> attribute. This name must correspond to a method on the same class annotated with <code>@DataProvider(name="…​")</code> with a matching name.</p>
</div>
<div class="paragraph">
<p>By default, the data provider will be looked for in the current test class or one of its base classes. If you want to put your data provider in a different class, it needs to be a static method or a class with a non-arg constructor, and you specify the class where it can be found in the <code>dataProviderClass</code> attribute:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="java">public class StaticProvider {
@DataProvider(name = "create")
public static Object[][] createData() {
return new Object[][] {
new Object[] { 42 }
};
}
}
public class MyTest {
@Test(dataProvider = "create", dataProviderClass = StaticProvider.class)
public void test(Integer n) {
// ...
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>The data provider supports injection too. TestNG will use the test context for the injection. The Data Provider method can return one of the following types:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>An array of array of objects (<code>Object[][]</code>) where the first dimension’s size is the number of times the test method will be invoked and the second dimension size contains an array of objects that must be compatible with the parameter types of the test method. This is the case illustrated by the example above.</p>
</li>
<li>
<p>An <code>Iterator<Object[]></code>. The only difference with <code>Object[][]</code> is that an <code>Iterator</code> lets you create your test data lazily. TestNG will invoke the iterator and then the test method with the parameters returned by this iterator one by one. This is particularly useful if you have a lot of parameter sets to pass to the method and you don’t want to create all of them upfront.</p>
<div class="ulist">
<ul>
<li>
<p>An array of objects (<code>Object[]</code>). This is similar to <code>Iterator<Object[]></code> but causes the test method to be invoked once for each element of the source array.</p>
</li>
<li>
<p>An <code>Iterator<Object>></code>. Lazy alternative of <code>Object[]</code>. Causes the test method to be invoked once for each element of the iterator.</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="paragraph">
<p>It must be said that return type is not limited to Object only thus <code>MyCustomData[][]</code> or <code>Iterator<Supplier></code> are also possible. The only limitation is that in case of iterator its parameter type can’t be explicitly parameterized itself. Here is an example of this feature:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="java">@DataProvider(name = "test1")
public Iterator<Object[]> createData() {
return new MyIterator(DATA);
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Using <code>MyCustomData[]</code> as a return type</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="java">@DataProvider(name = "test1")
public MyCustomData[] createData() {
return new MyCustomData[]{ new MyCustomData() };
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Or its lazy option with <code>Iterator<MyCustomData></code></p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="java">@DataProvider(name = "test1")
public Iterator<MyCustomData> createData() {
return Arrays.asList(new MyCustomData()).iterator();
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Parameter type (<code>Stream</code>) of <code>Iterator</code> can’t be explicitly parametrized</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="java">@DataProvider(name = "test1")
public Iterator<Stream> createData() {
return Arrays.asList(Stream.of("a", "b", "c")).iterator();
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>If you declare your <code>@DataProvider</code> as taking a <code>java.lang.reflect.Method</code> as first parameter, TestNG will pass the current test method for this first parameter. This is particularly useful when several test methods use the same <code>@DataProvider</code> and you want it to return different values depending on which test method it is supplying data for.</p>
</div>
<div class="paragraph">
<p>For example, the following code prints the name of the test method inside its <code>@DataProvider</code>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="java">@DataProvider(name = "dp")
public Object[][] createData(Method m) {
System.out.println(m.getName()); // print test method name
return new Object[][] { new Object[] { "Cedric" }};
}
@Test(dataProvider = "dp")
public void test1(String s) {
}
@Test(dataProvider = "dp")
public void test2(String s) {
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>and will therefore display:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="bash">test1
test2</code></pre>
</div>
</div>
<div class="paragraph">
<p>Data providers can run in parallel with the attribute parallel:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="java">@DataProvider(parallel = true)
public Object[][] getTestData() {
// ...
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Each of the parallel data providers running from an XML file runs with a thread pool which has a size of <code>10</code> by default. You can modify this value in the <code><suite></code> tag of your XML file:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><suite name="Suite1" data-provider-thread-count="20" >
<!-- content ignored for brevity -->
</suite></code></pre>
</div>
</div>
<div class="paragraph">
<p>If you want to run a few specific data providers in a different thread pool, you need to run them from a different XML file.</p>
</div>
</div>
<div class="sect3">
<h4 id="_retries_and_data_providers">Retries and data providers</h4>
<div class="paragraph">
<p>TestNG allows you to retry a data provider incase it has encountered any issues when calling it the first time.</p>
</div>
<div class="paragraph">
<p>This is similar to how regular test methods can be retried as explained in <a href="rerunning_failed_tests.html#_rerunning_failed_tests">this section</a>.</p>
</div>
<div class="paragraph">
<p>To be able to retry a data provider, the following needs to be done.</p>
</div>
<div class="ulist">
<ul>
<li>
<p>First we would need to implement the interface <code>org.testng.IRetryDataProvider</code>.</p>
</li>
<li>
<p>Next you would need to tie this implementation to the data provider annotation using the attribute <code>retryUsing</code> of the <code>@DataProvider</code> annotation.</p>
</li>
<li>
<p>With that we can now retry a failed data provider.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Here’s a sample retry implementation:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="java">import org.testng.IDataProviderMethod;
import org.testng.IRetryDataProvider;
import java.util.concurrent.atomic.AtomicInteger;
public class RetryDataProvider implements IRetryDataProvider {
private final AtomicInteger counter = new AtomicInteger(1);
@Override
public boolean retry(IDataProviderMethod dataProvider) {
boolean status = counter.getAndIncrement() <= 2;
String clazz = dataProvider.getMethod().getDeclaringClass().getName();
String dataProviderMethodName = dataProvider.getMethod().getName();
String methodName = clazz + "." + dataProviderMethodName + "()";
System.err.println("Retry the data provider method " + methodName + " ? " + status);
return status;
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Here’s how a test class that consumes this retry mechanism can look like:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="java">import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
public class RetriableDataProviderSample {
private boolean throwException = true;
@Test(dataProvider = "test-data")
public void sampleTestMethod(int input) {
System.err.println("Input value = " + input);
}
@DataProvider(retryUsing = RetryDataProvider.class, name = "test-data")
public Object[][] testDataSupplier() {
if (throwException) {
throwException = false;
System.err.println("Simulating a problem when invoking the data provider");
throw new IllegalStateException("Simulating a failure in data provider");
}
return new Object[][]{
{1}, {2}
};
}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>And when you run this sample, the output would look something like below:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="bash">SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Simulating a problem when invoking the data provider
Retry the data provider method org.testng.demo.RetriableDataProviderSample.testDataSupplier() ? true
Input value = 1
Input value = 2
===============================================
Default Suite
Total tests run: 2, Passes: 2, Failures: 0, Skips: 0
===============================================</code></pre>
</div>
</div>
</div>
<div class="sect3">
<h4 id="_controlling_threadpool_usage">Controlling ThreadPool Usage</h4>
<div class="paragraph">
<p>Starting from TestNG <code>7.9.0</code>, there are some additional ways in which the thread-pools that run the parallel tests can be controlled. For these new features to be consumed, update your suite file to use the <code>testng-1.1.dtd</code> (as seen below) so that your IDE can provide you with autocompletion:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="xml"><!DOCTYPE suite SYSTEM "https://testng.org/testng-1.1.dtd" >
<suite name="sample">
<!-- content ignored for brevity -->
</suite></code></pre>
</div>
</div>
<div class="ulist">
<ul>
<li>
<p><code>share-thread-pool-for-data-providers</code> - When this attribute is set to true at the suite level, TestNG will start using a shared thread pool for all the data driven tests in a given <code><suite></code>. The size of the thread pool is determined using the attribute <code>data-provider-thread-count</code>. This attribute has a default value of <code>false</code>.</p>
</li>
<li>
<p><code>use-global-thread-pool</code> - When this attribute is set to true at the suite level, TestNG will start using a common thread pool for running both your regular test methods and data driven test methods. The size of the thread pool is determined using the attribute thread-count. This attribute has a default value of <code>false</code>.</p>
</li>
</ul>
</div>
</div>
<div class="sect3">
<h4 id="_parameters_from_system_properties">Parameters from System Properties</h4>
<div class="paragraph">
<p>TestNG can be passed parameters on the command line of the Java Virtual Machine using system properties (<code>-D</code>). Parameters passed in this way are not required to be pre-defined in <code>testng.xml</code>, but will override any parameters defined there.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="prettyprint highlight"><code data-lang="bash">java -Dfirst-name=Cedrick -Dlast-name="von Braun" org.testng.TestNG testng.xml</code></pre>
</div>
</div>
<div class="paragraph">
<p>The Java system property variable is a string with no spaces that represents the name of the property. The value variable is a string that represents the value of the property. If the value is a string with spaces, then enclose it in quotation marks.</p>
</div>
<div class="admonitionblock tip">
<table>
<tr>
<td class="icon">
<i class="fa icon-tip" title="Tip"></i>
</td>
<td class="content">
In TestNG 6.x parameters defined in testng.xml could not be overwritten by system properties
</td>
</tr>
</table>
</div>
<div class="sect4">
<h5 id="_parameters_in_reports">Parameters in reports</h5>
<div class="paragraph">
<p>Parameters used to invoke your test methods are shown in the HTML reports generated by TestNG. Here is an example:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="pics/parameters.png" alt="parameters">
</div>
</div>
</div>
</div>
</div>
</div>
<div id="footer">
<div id="footer-text">
Last updated 2025-02-27 18:51:48 UTC
</div>
</div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/prettify/r298/run_prettify.min.js"></script>
</body>
</html>