TestNG supports many annotations and the philosophy behind using annotations is to bring the Test methods closer to any normal java method.

The test methods doesn’t need to follow any naming convention They should be allowed to accept parameters like any other java method The annotations allows further configuration using its attributes Test classes doesn’t need to extend any base test class

Below is the list of TestNG annotations:

@BeforeSuite will be executed before any tests declared inside a TestNG suite. @BeforeTest will be executed before each test section declared inside a TestNG suite. @BeforeClass will be executed before any of the test methods of a test class. @BeforeMethod will be executed before the execution of each test method. @BeforeGroups will run before any of the test method of the specified group is executed. groups attribute must contain the list of groups this method belongs to. @AfterSuite will be executed after any tests declared inside a TestNG suite. @AfterTest will be executed after each test section declared inside a TestNG suite. @AfterClass will be executed after the last test method of a test class. @AfterMethod will be executed after the execution of each test method. @AfterGroups will run after the last test method of the specified group is executed. groups attribute must contain the list of groups this method belongs to. @DataProvider method provides the data for a test method and must return a two dimensional object array (Object[ ][ ]) as data. @Factory method returns an array of class objects (Object[ ]). This is used to run a set of test cases with different values provided to the test class during its instantiation. @Test marks a class or a method as a test method. If used at class level, all the public methods of a class will be considered as a test method. @Listeners is defined at class level to specify an array of test listeners classes extending org.testng.ITestNGListener . @Parameters is used to pass parameters to a test method. These parameter values are provided in the testng.xml configuration file at runtime.

Configuration Annotations

Configuration annotations start with @Before and @After annotations. Each of these annotations run at a specific event in the TestNG lifecycle. To know those events it is important that we understand the testng.xml structure. It consists of:

suite which is the first element of testng.xml . A suite contains one or more test elements. A test contains one more test classes A class consists of one or more @test methods

A test is made of one or more classes and a class is made or one or more methods.

Let’s first run a simple testNg.xml which contains just a test class.

simpleTestng.xml:

<?xml version="1.0" encoding="UTF-8"?> <suite name="Suite"> <test name="UnitTesting"> <classes> <class name="com.javarticles.testng.TestA" /> </classes> </test> </suite>

As you see our testNg.xml is very simple, it just includes one test class TestA . Few things to note about:

You can always provide multiple methods of the same annotation type. For example, in the below class, I have two @BeforeSuite methods. The annotation methods are allowed to accept some special typed parameters like Method and ITestContext . TestNG takes the responsibility to feed in the runtime object. In our example, beforeTest test method accepts ITestContext , this helps us to know the TestNG’s test section name. We all the configuration annotation methods in our test class TestA . This will give us an idea of the sequence in which the methods are invoked.

TestA:

package com.javarticles.testng; import org.testng.ITestContext; import org.testng.annotations.AfterClass; import org.testng.annotations.AfterMethod; import org.testng.annotations.AfterSuite; import org.testng.annotations.AfterTest; import org.testng.annotations.BeforeClass; import org.testng.annotations.BeforeMethod; import org.testng.annotations.BeforeSuite; import org.testng.annotations.BeforeTest; import org.testng.annotations.Test; public class TestA { @BeforeSuite public void beforeSuite1() { System.out.println("TestA: beforeSuite1"); } @BeforeSuite public void beforeSuite2() { System.out.println("TestA: beforeSuite2"); } @BeforeTest public void beforeTest(ITestContext testContext) { System.out.println("TestA: beforeTest Test<" + testContext.getName() + ">"); } @BeforeClass public void beforeClass() { System.out.println("TestA: beforeClass"); } @BeforeMethod public void beforeMethod() { System.out.println("TestA: beforeMethod"); } @Test public void unitTest1A() { System.out.println("TestA: unitTest1A"); } @Test(groups="feature1") public void unitTest2A() { System.out.println("TestA: unitTest2A, feature1 test"); } @Test public void unitTest3A() { System.out.println("TestA: unitTest3A"); } @Test(groups="feature2") public void unitTest4A() { System.out.println("TestA: unitTest4A, feature2 test"); } @AfterMethod public void afterMethod() { System.out.println("TestA: afterMethod"); } @AfterClass public void afterClass() { System.out.println("TestA: afterClass"); } @AfterTest public void afterTest(ITestContext testContext) { System.out.println("TestA: afterTest Test<" + testContext.getName() + ">"); } @AfterSuite public void afterSuite() { System.out.println("TestA: afterSuite"); } }

Right click on the testNg.xml file and then click on ‘Run-as->TestNG Suite’.

As you can see from the output, the sequence in which the configurations methods run are:

@BeforeSuite methods @BeforeTest methods @BeforeClass methods @BeforeMethod methods @Test methods @AfterMethod methods @AfterClass methods @AfterTest methods @AfterSuite methods

@BeforeMethod , @Test and @AfterMethod repeat for each test method.

Output:

[TestNG] Running: C:\javarticles_ws\testNGAnnotations\simpleTestng.xml TestA: beforeSuite1 TestA: beforeSuite2 TestA: beforeTest Test TestA: beforeClass TestA: beforeMethod TestA: unitTest1A TestA: afterMethod TestA: beforeMethod TestA: unitTest2A, feature1 test TestA: afterMethod TestA: beforeMethod TestA: unitTest3A TestA: afterMethod TestA: beforeMethod TestA: unitTest4A, feature2 test TestA: afterMethod TestA: afterClass TestA: afterTest Test TestA: afterSuite =============================================== Suite Total tests run: 4, Failures: 0, Skips: 0 ===============================================

Multiple tests

You can also run multiple tests together. For example, I can group my tests into ‘Unit Tests’, ‘Functional Tests’ and ‘UI Tests’.

<?xml version="1.0" encoding="UTF-8"?> <suite name="Suite"> <test name="UnitTesting"> <classes> <class name="com.javarticles.testng.TestA" /> </classes> </test> <test name="FunctionalTesting"> <classes> <class name="com.javarticles.testng.TestA" /> <class name="com.javarticles.testng.TestB" /> </classes> </test> <test name="UITesting"> <classes> <class name="com.javarticles.testng.TestC" /> </classes> </test> </suite>

We have added two new test classes TestB and TestC .

TestB:

package com.javarticles.testng; import org.testng.ITestContext; import org.testng.annotations.AfterGroups; import org.testng.annotations.AfterMethod; import org.testng.annotations.AfterSuite; import org.testng.annotations.AfterTest; import org.testng.annotations.BeforeGroups; import org.testng.annotations.BeforeMethod; import org.testng.annotations.BeforeSuite; import org.testng.annotations.BeforeTest; import org.testng.annotations.Test; public class TestB { @BeforeSuite public void beforeSuite() { System.out.println("TestB: beforeSuite"); } @BeforeTest public void beforeTest(ITestContext testContext) { System.out.println("TestB: beforeTest Test<" + testContext.getName() + ">"); } @BeforeMethod public void beforeMethod() { System.out.println("TestB: beforeMethod"); } @Test public void scenario1B() { System.out.println("TestB: Scenario1B"); } @Test(groups="feature1") public void scenario2B() { System.out.println("TestB: Scenario2B, feature1 test"); } @AfterMethod public void afterMethod() { System.out.println("TestB: afterMethod"); } @AfterTest public void afterTest(ITestContext testContext) { System.out.println("TestB: afterTest Test<" + testContext.getName() + ">"); } @AfterSuite public void afterSuite() { System.out.println("TestB: afterSuite"); } @BeforeGroups(groups="feature1") public void beforeGroup() { System.out.println("TestB: beforeGroup for feature1"); } @AfterGroups(groups="feature1") public void afterGroup() { System.out.println("TestB: afterGroup for feature1"); } }

TestC:

package com.javarticles.testng; import org.testng.ITestContext; import org.testng.annotations.AfterGroups; import org.testng.annotations.AfterMethod; import org.testng.annotations.AfterSuite; import org.testng.annotations.AfterTest; import org.testng.annotations.BeforeGroups; import org.testng.annotations.BeforeMethod; import org.testng.annotations.BeforeSuite; import org.testng.annotations.BeforeTest; import org.testng.annotations.Test; public class TestC { @BeforeSuite public void beforeSuite() { System.out.println("TestC: beforeSuite"); } @BeforeTest(alwaysRun=true) public void beforeTest(ITestContext testContext) { System.out.println("TestC: beforeTest Test<" + testContext.getName() + ">"); } @BeforeMethod public void beforeMethod() { System.out.println("TestC: beforeMethod"); } @Test public void uiSceario1C() { System.out.println("TestC: uiSceario1C"); } @Test(groups="feature1") public void uiScenario2C() { System.out.println("TestC: uiScenario2C, feature1 test"); } @Test(groups="feature2") public void uiScenario3C() { System.out.println("TestC: uiScenario3C, feature2 test"); } @AfterMethod(groups="feature1") public void afterMethod() { System.out.println("TestC: afterMethod"); } @AfterTest(alwaysRun=true) public void afterTest(ITestContext testContext) { System.out.println("TestC: afterTest Test<" + testContext.getName() + ">"); } @AfterSuite public void afterSuite() { System.out.println("TestC: afterSuite"); } @BeforeGroups(groups="feature2") public void beforeGroup() { System.out.println("TestC: beforeGroup for feature2"); } @AfterGroups(groups="feature2") public void afterGroup() { System.out.println("TestC: afterGroup for feature2"); } }

That’s lot of output! If you note the sequence of execution remains same. We now see couple of new annotation methods @BeforeGroup and @AfterGroup methods. They fire whenever a test method is associated to a group is invoked.

Output:

[TestNG] Running: C:\javarticles_ws\testNGAnnotations\multiTestsTestng.xml TestA: beforeSuite1 TestA: beforeSuite2 TestB: beforeSuite TestC: beforeSuite TestA: beforeTest Test TestA: beforeClass TestA: beforeMethod TestA: unitTest1A TestA: afterMethod TestA: beforeMethod TestA: unitTest2A, feature1 test TestA: afterMethod TestA: beforeMethod TestA: unitTest3A TestA: afterMethod TestA: beforeMethod TestA: unitTest4A, feature2 test TestA: afterMethod TestA: afterClass TestA: afterTest Test TestA: beforeTest Test TestB: beforeTest Test TestA: beforeClass TestA: beforeMethod TestA: unitTest1A TestA: afterMethod TestB: beforeGroup for feature1 TestA: beforeMethod TestA: unitTest2A, feature1 test TestA: afterMethod TestA: beforeMethod TestA: unitTest3A TestA: afterMethod TestA: beforeMethod TestA: unitTest4A, feature2 test TestA: afterMethod TestA: afterClass TestB: beforeMethod TestB: Scenario1B TestB: afterMethod TestB: beforeMethod TestB: Scenario2B, feature1 test TestB: afterMethod TestB: afterGroup for feature1 TestA: afterTest Test TestB: afterTest Test TestC: beforeTest Test TestC: beforeMethod TestC: uiSceario1C TestC: afterMethod TestC: beforeMethod TestC: uiScenario2C, feature1 test TestC: afterMethod TestC: beforeGroup for feature2 TestC: beforeMethod TestC: uiScenario3C, feature2 test TestC: afterMethod TestC: afterGroup for feature2 TestC: afterTest Test TestA: afterSuite TestB: afterSuite TestC: afterSuite =============================================== Suite Total tests run: 13, Failures: 0, Skips: 0 ===============================================

Run groups

You can also group your tests based on your feature using <groups> element. For example,

groupTestng.xml:

<?xml version="1.0" encoding="UTF-8"?> <suite name="Suite"> <test name="Feature1"> <groups> <run> <include name="feature1" /> </run> </groups> <classes> <class name="com.javarticles.testng.TestA" /> <class name="com.javarticles.testng.TestB" /> <class name="com.javarticles.testng.TestC" /> </classes> </test> <test name="Feature2"> <groups> <run> <include name="feature2" /> </run> </groups> <classes> <class name="com.javarticles.testng.TestA" /> <class name="com.javarticles.testng.TestC" /> </classes> </test> </suite>

One thing to note when you are running the groups:

The configuration annotated methods won’t run unless you specify the group in the groups attribute or set alwaysRun to true. For example, in our TestC , we have included both the cases.

@AfterMethod(groups="feature1") public void afterMethod() { System.out.println("TestC: afterMethod"); } @AfterTest(alwaysRun=true) public void afterTest(ITestContext testContext) { System.out.println("TestC: afterTest Test<" + testContext.getName() + ">"); }

Output:

[TestNG] Running: C:\javarticles_ws\testNGAnnotations\groupTestng.xml TestC: beforeTest Test TestB: beforeGroup for feature1 TestA: unitTest2A, feature1 test TestB: Scenario2B, feature1 test TestC: uiScenario2C, feature1 test TestC: afterMethod TestB: afterGroup for feature1 TestC: afterTest Test TestC: beforeTest Test TestC: beforeGroup for feature2 TestA: unitTest4A, feature2 test TestC: uiScenario3C, feature2 test TestC: afterGroup for feature2 TestC: afterTest Test =============================================== Suite Total tests run: 5, Failures: 0, Skips: 0 ===============================================

Example of @Parameters

One of the important features of TestNG is data-driven testing.

One can pass parameter values to test methods as arguments following the below methods:

Using testng.xml configuration file. Through DataProviders

Parameterization is about defining the parameters and its value in testng.xml and then referring to the parameters using @Parameters annotation at method level. The attributes section of @Parameters will have the array of parameters used.

The parameters are defined in testng.xml using the parameter element. Its name and value are specified using the attributes name and value .

paramTestng.xml:

<?xml version="1.0" encoding="UTF-8"?> <suite name="Suite"> <parameter name="param1" value="One"></parameter> <parameter name="param2" value="1"></parameter> <test name="ParamTesting"> <classes> <class name="com.javarticles.testng.TestParam" /> </classes> </test> </suite>

Now we use these parameters in the test method.

TestParam:

package com.javarticles.testng; import org.testng.annotations.Parameters; import org.testng.annotations.Test; public class TestParam { @Parameters({"param1", "param2"}) @Test public void t(String p1, String p2) { System.out.println("t(" + p1 + ", " + p2 + ")" ); } }

Output:

[TestNG] Running: C:\javarticles_ws\testNGAnnotations\paramTestng.xml t(One, 1) =============================================== Suite Total tests run: 1, Failures: 0, Skips: 0 ===============================================

For more example on @Parameters , click here

Example of @DataProvider

One can provide data using @DataProvider annotated method.The method must return two dimensional object[][] array. The first array represents data set thus takes care of the number of time the tests has to run. The second array represents number of parameters. You can even provide a name to it else by default the method name becomes its name.

If a test is referring to a DataProvider for its parameter values then it has to specify the dataProvider attribute in its Test annotation.

A data-driven test will run once for each set of data passed-in and the data will be provided by the method annotated with @DataProvider .

In the below example, dp() is my data provider method. It returns two sets of data. Each set contains two parameter values. First set is ‘A’ and ‘a’. The second set is ‘B’ and ‘b’.

This is use to inject values to test method alphabets . Since the DataProvider has no name, we refer to it using the method name dp .

TestDataProvider:

package com.javarticles.testng; import org.testng.annotations.DataProvider; import org.testng.annotations.Test; public class TestDataProvider { @Test(dataProvider="dp") public void alphabets(String upperCase, String lowerCase) { System.out.println("alphabets(" + upperCase + ", " + lowerCase + ")"); } @DataProvider public Object[][] dp() { return new Object[][]{{"A", "a"}, {"B", "b"}}; } }

dataProviderTestng.xml:

<?xml version="1.0" encoding="UTF-8"?> <suite name="Suite"> <test name="DataProviderTesting"> <classes> <class name="com.javarticles.testng.TestDataProvider" /> </classes> </test> </suite>

Output:

[TestNG] Running: C:\javarticles_ws\testNGAnnotations\dataProviderTestng.xml alphabets(A, a) alphabets(B, b) =============================================== Suite Total tests run: 2, Failures: 0, Skips: 0 ===============================================

For more examples on @DataProvider , click here

Example of @Factory

We can use TestNG @Factory to create multiple test instances. The test class may either have the default no-arg constructor or constructor with one or more arguments.

In the below example, create is our @Factory method. It creates three instances of TestFactory with different set of data.

TestFactory:

package com.javarticles.testng; import org.testng.annotations.Factory; import org.testng.annotations.Test; public class TestFactory { private String param; public TestFactory(String param) { this.param = param; } @Test public void t() { System.out.println("TestFactory.t:" + param); } @Factory public Object[] create() { return new Object[] { new TestFactory("one"), new TestFactory("two"), new TestFactory("three")}; } }

factoryTestng.xml:

<?xml version="1.0" encoding="UTF-8"?> <suite name="Suite"> <test name="FactoryTesting"> <classes> <class name="com.javarticles.testng.TestFactory" /> </classes> </test> </suite>

When we run the factoryTestng.xml , we end up running the test case thrice, once for each test instance.

Output:

[TestNG] Running: C:\javarticles_ws\testNGAnnotations\factoryTestng.xml TestFactory.t:two TestFactory.t:three TestFactory.t:one =============================================== Suite Total tests run: 3, Failures: 0, Skips: 0 ===============================================

For more examples on @Factory , click here.

Example of @Listeners

In TestNG, all listeners extend the marker interface ITestNGListener . You can register listeners using @Listeners at class level. We specify here an array of test listeners class that extend org.testng.ITestNGListener .

Implement an ISuiteListener .

SuiteListener:

package com.javarticles.testng; import org.testng.ISuite; import org.testng.ISuiteListener; public class SuiteListener implements ISuiteListener { @Override public void onStart(ISuite suite) { System.out.println("Start of suite " + suite.getName()); } @Override public void onFinish(ISuite suite) { System.out.println("Finish of suite " + suite.getName()); } }

Using Listeners annotation specify the above listener.

TestListener:

package com.javarticles.testng; import org.testng.annotations.Listeners; import org.testng.annotations.Test; @Listeners({SuiteListener.class}) public class TestListener { @Test public void t() { System.out.println("test t()" ); } }

listenerTestng.xml:

<?xml version="1.0" encoding="UTF-8"?> <suite name="Suite"> <test name="TestingListener"> <classes> <class name="com.javarticles.testng.TestListener" /> </classes> </test> </suite>

Output:

[TestNG] Running: C:\javarticles_ws\testNGAnnotations\listenerTestng.xml Start of suite Suite test t() Finish of suite Suite =============================================== Suite Total tests run: 1, Failures: 0, Skips: 0 ===============================================

One particular pitfall TestNG may cause is, that TestNG does not guarantee that i.e. a @BeforeTest method is directly run before the Test. It only guarantees that it was executed at some point in time before starting the test. (Based on bentolor comments in Reddit).

If you want to run a specific method right before or after the invocation of a test method then implement a IInvokedMethodListener . You need to implement beforeInvocation and afterInvocation , compare the method name to make sure that the test method being invoked (or already invoked) is the one you are interested in.

For example, in the below InvokedTestMethodListenerExample , we want to run a method right before t2() and after t2() . We will implement IInvokedMethodListener and add it to our @Listeners annotation.

InvokedTestMethodListenerExample:

package com.javarticles.testng; import org.testng.annotations.AfterTest; import org.testng.annotations.BeforeTest; import org.testng.annotations.Listeners; import org.testng.annotations.Test; @Listeners(InvokedTestMethodListener.class) public class InvokedTestMethodListenerExample { @BeforeTest public void beforeTest1() { System.out.println("beforeTest1"); } @BeforeTest public void beforeTest2() { System.out.println("beforeTest2"); } @Test public void t1() { System.out.println("t1"); } @Test public void t2() { System.out.println("t2"); } @Test public void t3() { System.out.println("t3"); } @Test public void t4() { System.out.println("t4"); } @AfterTest public void afterTest1() { System.out.println("afterTest1"); } @AfterTest public void afterTest2() { System.out.println("afterTest2"); } }

InvokedTestMethodListener:

package com.javarticles.testng; import org.testng.IInvokedMethod; import org.testng.IInvokedMethodListener; import org.testng.ITestResult; public class InvokedTestMethodListener implements IInvokedMethodListener { @Override public void beforeInvocation(IInvokedMethod method, ITestResult testResult) { if ("t2".equals(method.getTestMethod().getMethodName())) { System.out.println("Invoked just before t2()"); } } @Override public void afterInvocation(IInvokedMethod method, ITestResult testResult) { if ("t2".equals(method.getTestMethod().getMethodName())) { System.out.println("Invoked just after t2()"); } } }

Output:

[TestNG] Running: C:\Users\mokkara\AppData\Local\Temp\testng-eclipse-1301133492\testng-customsuite.xml beforeTest1 beforeTest2 t1 Invoked just before t2() t2 Invoked just after t2() t3 t4 afterTest1 afterTest2 PASSED: t1 PASSED: t2 PASSED: t3 PASSED: t4

For more examples on TestNG listeners, click here

Download source code “testng annotations”

In this article, I showed you examples of TestNG annotations. You can download the source code here: testNGAnnotations.zip