Transcripted Summary
In this chapter, we will discuss the groups and dependsOnGroups attributes, and execute groups using an XML file.
# groups Attribute
The groups attribute is a way to add any number of test methods to a named list. This feature is beneficial because it’s an opportunity to run only those lists of test methods according to their group name. The group names can consist of a module of our application, a certain test type, or whatever else we decide. Plus, one test method can be a part of more than one group. For example, you can add the sign in test method to a group for smoke tests, and to another group for regression tests. A class can also be added to a group.
In this example, test1_SetUpChrome is added to a group called initialize.
@Test (groups = "initialize")
public void test1_SetUpChrome() {}
# dependsOnGroups Attribute
The dependsOnGroups attribute is similar to the dependsOnMethods attribute in that it allows us to define a group name that our test method depends on. Every method member of a group is guaranteed to be invoked before a method that depends on this group.
Let's make the test method test2_OpenOrangeHRM depend on the initialize group.
@Test (groups = "initialize")
public void test1_SetUpChrome() {}
@Test (dependsOnGroups = "initialize")
public void test2_OpenOrangeHRM() {}
We can also make test2_OpenOrangeHRM a part of a new group which we'll call env_ application.
@Test (groups = "initialize")
public void test1_SetUpChrome() {}
@Test (dependsOnGroups = "initialize", groups = "env_application")
public void test2_OpenOrangeHRM() {}
Let’s pretend there is a different URL for each environment such as a dev environment, QA environment, and production environment. We can have a group for each of these environments.
We are not required to add our test methods to a group. The remaining test methods do not belong to a group but we'll have them depend on a group.
@Test (groups = "initialize")
public void test1_SetUpChrome() {}
@Test (dependsOnGroups = "initialize", groups = "env_application")
public void test2_OpenOrangeHRM() {}
@Test (dependsOnGroups = "env_application")
public void test3_SignIn() {}
@Test (dependsOnGroups = "env_application")
public void test4_SearchUser() {}
@Test (dependsOnGroups = "env_application")
public void test5_SearchEmployee() {}
@Test (dependsOnGroups = "env_application")
public void test6_SearchCandidate() {}
@Test (dependsOnGroups = "env_application")
public void test7_SignOut() {}
When we run this, we see that all tests pass. But let's see what happens when there's a failure. We'll change our application's URL to an invalid one then run again.
@Test (groups = "initialize")
public void test1_SetUpChrome() {}
@Test (dependsOnGroups = "initialize", groups = "env_application")
public void test2_OpenOrangeHRM()
{
//Invalid URL
driver.get("https://opensource-demo.orangehrmlive1234.com/");
Assert.assertEquals(false, true, "Could Not Open OrangeHRM");
System.out.println("2. Open OrangeHRM");
}
@Test (dependsOnGroups = "env_application")
public void test3_SignIn() {}
@Test (dependsOnGroups = "env_application")
public void test4_SearchUser() {}
@Test (dependsOnGroups = "env_application")
public void test5_SearchEmployee() {}
@Test (dependsOnGroups = "env_application")
public void test6_SearchCandidate() {}
@Test (dependsOnGroups = "env_application")
public void test7_SignOut() {}
Console Output
Total test runs: 7, Failures: 1, Skips: 5
There's one failure and five skips, just like with the dependsOnMethods attribute.
# Execute Groups via XML File
TestNG provides a few ways to run groups at runtime. The main way is to use an XML file but we can also use the command line or Ant.
Let's look at a new class with six test methods.
import org.testng.annotations.Test;
public class Amazon_FakeSite
{
@Test (groups = "smoke")
public void test1_LogIn ()
{
System.out.println("1. Log In / Group = smoke" + "\n");
}
@Test (groups = { "smoke", "regression" } )
public void test2_SearchProducts ()
{
System.out.println("2. Search Products / Group = smoke, regression" + "\n");
}
@Test (groups = { "regression", "integration" , "defect.fix"} )
public void test3_PlaceOrder ()
{
System.out.println("3. Place Order / Group = regression, integration, defect.fix" + "\n");
}
@Test (groups = { "system", "defect.backlog" } )
public void test4_SendConfirmation ()
{
System.out.println("4. Send Confirmation / Group = system, defect.backlog" + "\n");
}
@Test (groups = { "regression", "defect.progress" } )
public void test5_ShipOrder ()
{
System.out.println("5. Ship Order / Group = regression, defect.progress" + "\n");
}
@Test
public void test6_LogOut ()
{
System.out.println("6. Log Out / Group = None" + "\n");
}
}
We've added test1 and test2 to a `smoke` group.
test2 is also a part of a regression group along with test3 and test 5.
In addition to the regression group, test3 also belongs to the integration and defect.fix groups. We have the option of being very descriptive with our group name so noted I added .fix as part of the group name indicating that this test method has a defect that should be fixed. When it is fixed, we can remove this test from the defect.fix group.
I've added test4 to the system and defect.backlog groups.
And test5 has been added to the regression and defect.progress groups.
test6 is not a part of a group.
Let’s execute some scenarios for groups at runtime using this XML file:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Test Automation Suite">
<test name="All Groups Test">
<groups>
<run></run>
</groups>
<classes>
<class name="com.testautomationu.chp6dependencytesting.Amazon_FakeSite"/>
</classes>
</test> <!-- All Groups Test -->
</suite> <!-- Test Automation Suite -->
Notice, the <groups> tag and <run> tag. The <groups> tag allows two tags: <run> and <define>.
The <run> tag lists the groups that can be included or excluded. This file does not include or exclude any groups. Therefore, all groups will show up.
The <define> tag helps us create a new group based on existing groups.
Running this XML file will execute all groups within the Amazon_FakeSite class.
The next xml file includes a regression group.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Test Automation Suite">
<test name="Include Regression Defect Groups Test">
<groups>
<run>
<include name = "regression"/>
</run>
</groups>
<classes>
<class name="com.testautomationu.chp6dependencytesting.Amazon_FakeSite"/>
</classes>
</test> <!-- All Groups Test -->
</suite> <!-- Test Automation Suite -->
When we run this, we see the following:
Console Output
2. Search Products / Group = smoke, regression
3. Place Order / Group = regression, integration, defect.fix
5. Ship Order / Group = regression, defect.progress
Notice only the tests that were apart of the regression group were executed, because this is what we specified in our XML file. These methods happen to be a part of other groups as well, but they were only executed because they belong to the regression group.
In the XML file, we can include more than one group. This example includes the smoke and system groups, which includes three methods from our class.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Test Automation Suite">
<test name="Include Smoke and System Groups Test">
<groups>
<run>
<include name = "smoke"/>
<include name = "system"/>
</run>
</groups>
<classes>
<class name="com.testautomationu.chp6dependencytesting.Amazon_FakeSite"/>
</classes>
</test> <!-- All Groups Test -->
</suite> <!-- Test Automation Suite -->
Console Output
1. Log In / Group = smoke
2. Search Products / Group = smoke, regression
4. Send Confirmation / Group = system, defect.backlog
Look what happens when we exclude a group, in this case `regression`.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Test Automation Suite">
<test name="Exclude Regression Groups Test">
<groups>
<run>
<exclude name = "regression"/>
</run>
</groups>
<classes>
<class name="com.testautomationu.chp6dependencytesting.Amazon_FakeSite"/>
</classes>
</test> <!-- All Groups Test -->
</suite> <!-- Test Automation Suite -->
Console Output
1. Log In / Group = smoke
4. Send Confirmation / Group = system, defect.backlog
6. Log Out / Group = None
As expected, no test methods which are assigned to the regression group have been executed.
We also have the ability to use regular expressions. In this XML file, defects has a regular expression using a dot and asterisk.
<exclude name = "defect.*"/>
Let's use that example where all groups that begin with defect. are excluded, and let's also indicate to include the regression group:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Test Automation Suite">
<test name="Include Regression - Exclude Defect Groups Test">
<groups>
<run>
<include name = "regression"/>
<exclude name = "defect.*"/>
</run>
</groups>
<classes>
<class name="com.testautomationu.chp6dependencytesting.Amazon_FakeSite"/>
</classes>
</test> <!-- All Groups Test -->
</suite> <!-- Test Automation Suite -->
Even though we have three tests as part of the regression group, only one of them are executed because it has filtered out the ones that are a part of group that begins with defect.
Console Output
2. Search Products / Group = smoke, regression
# Run Test Methods According To Group Name
Let’s recap the rules we covered when executing groups at runtime:
- a
<run>that has no<include>and no<exclude>will run all test methods - if we include a group then only test methods assigned to that group will run
- if we include more than one group then members of all included groups will be run
- excluding a group runs all test methods except the test methods assigned to that group
- we can use regular expressions to specify group names that should be included or excluded
- if a test method belongs to a group that is included but also belongs to another group that is excluded, then the test will not run because excluded wins
Up to this point, all of the test methods that we executed were from the same class. However, TestNG is not limited to executing a group using one class. We can execute groups using more than one class or even a package.
This XML file has two <class> tags.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Test Automation Suite">
<test name="Include 2 Classes Groups Test">
<groups>
<run>
<include name = "regression"/>
<include name = "env_application"/>
</run>
</groups>
<classes>
<class name="com.testautomationu.chp6dependencytesting.Amazon_FakeSite"/>
<class name="com.testautomationu.chp6dependencytesting.Groups_DependsOnGroups_PASS"/>
</classes>
</test> <!-- All Groups Test -->
</suite> <!-- Test Automation Suite -->
We have included the regression and env_application groups from both the Amazon_FakeSite and Groups_DependsOnGroups_PASS.
When we run this, all test methods that belong to these two groups and are within these two classes are executed.
