Driver behavior executing parallel TestNG selenium

问题:

I want to run selenium tests in TestNg in parallel that use the @dataprovider. Ideally tests are parallel by method (one test = one method) and not simple suite parallelism by browser. I have read somewhere that about 5 instances of ChromeDriver can be controlled at a time so I thought this should be possible. Later I plan to move to grid2. For developement I'm running things with IntelliJ Idea test runner by right-click + run on the XML config file.

I had problems running my tests in parallel (on grid2 and locally) so I created a sample of more or less what I want to do.

Here is my test class

package tests;

import org.openqa.selenium.By;
import org.openqa.selenium.Keys;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.interactions.Actions;
import org.testng.annotations.*;
import java.util.concurrent.TimeUnit;
import static org.testng.Assert.assertNotNull;

public class ParallelTest {
    public static final String SEARCH_TERMS = "search-terms";
    private WebDriver driver;

    @BeforeMethod
    @Parameters({"browser"})
    public void beforeMethod(@Optional("chrome") String browser){
        driver = getBrowser(browser);
        driver.manage().deleteAllCookies();
        driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
    }

    private WebDriver getBrowser(String browser) {
        if(browser.equals("chrome")){
            System.setProperty("webdriver.chrome.driver", "webdrivers\\chromedriver.exe");
            return new ChromeDriver();
        }
        return new FirefoxDriver();
    }

    @AfterMethod
    public void afterMethod(){
        driver.quit();
    }

    @Test(description = "Check parallel selenium works.",
          dataProvider = SEARCH_TERMS)
    public void parallelSeleniumTest(String searchTerm){
        driver.get("http://google.com");
        WebElement search = driver.findElement(By.id("gbqfq"));
        new Actions(driver)
                .sendKeys(search, searchTerm)
                .sendKeys(search, Keys.ENTER)
                .perform();
        String firstResult = driver.findElements(By.className("r")).get(0).getText();
        assertNotNull(firstResult);
        System.out.println(firstResult);
    }

    @DataProvider(name = SEARCH_TERMS, parallel = true)
    public Object[][] getSearchTerms(){
        return new Object[][]{
                {"google"},
                {"microsoft"},
                {"facebook"},
                {"amazon"},
                {"apple"},
                {"oracle"},
                {"yahoo"},
                {"jetbrains"},
                {"intellij idea"},
                {"selenium"},
                {"java"},
                {"testng"},
                {"code"}
        };
    }
}

I threw in some native events since I use them heavily in my test suite.

And here is the TestNg xml config file

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite thread-count="4" name="vfr6-ui-tests" parallel="methods">
    <test name="parallel-test-firefox">
        <parameter name="browser" value="firefox"/>
        <classes>
            <class name="tests.ParallelTest"/>
        </classes>
    </test>
    <test name="parallel-test-chrome">
        <parameter name="browser" value="chrome"/>
        <classes>
            <class name="tests.ParallelTest"/>
        </classes>
    </test>
</suite>

I read instantiating one driver per test tends to be the most maintainable. The problem is that the firefox test runs in serial while the chrome test spits out all of the data points as test cases, attempts to open a trove of browser instances, then everything fails. My tests will have either 10-25 or 300-500 data points (cycling between either clients or clients x products).

What is the best way to set up the driver, dataprovider, and test runner to achieve the best parallelism in running tests?

回答1:

I had the same experience about dataProvider. In my case I used dataProvider's (parallel=true) attribute though. There are two solutions to your problem.

1. Use dataProvider and in test class and use factory annotation for your constructor. In the factory annotation's attribute, use dataProvider="Your dataProvider's name". In the testng.xml, instead of parallel=methods, use parallel=instances.

   The drawback of the above approach is that when you get the report; may be it is maven- surefire, testng Eclipse report or reportNG report, you do not see parameters passed up front. To overcome this, you can use the following approach.

2. Create a factory class and instantiate your test class in the factory method using a for loop. (Start for loop from 0.) In the test class define a constructor which receives a parameter from factory class. Define a dataProvider in this test class which can use the parameter (data-point) received in the constructor. Define a BeforeMethod or BeforeClass which can use that parameter or data point and Your test methods should have the "dataProvider" attribute pointing to the desired dataProvider. Again, in testng.xml use parallel="instances".

   Also, use try/catch block for instantiating driver object and closing the browser. This will help you in avoiding skips due to failure of setUp of tearDown method.

回答2:

You don't really need to use a Factory. If I were you I would call this code within the dataprovider method:

driver = getBrowser(browser);

Then, return the driver instances as a 2nd column of args to the test method. Doing it that way allows the dataprovider to generate browser instances. To improve on that, you could instead use the builder design pattern, in the form of a DriverHelper class, that could replace the getBrowser method with a way to generate a much more specific driver configuration before passing the driver instance into the test method.

NOTE: Keep in mind that if you ever want to use Spring to load your drivers in the future, then this method won't work at all. In fact, you wont be able to use a DataProvider at all. But, if your not using Spring, then I would say this is the most elegant way of doing it.