Certain tests not reporting in TestNG

Question

I'm using TestNG and Selenium in Eclipse, in Java. I have a method test that calls the other methods as support for the actions it needs to perform. I have them marked as part of the test using @Test.

@BeforeTest
public void beforeTest() {
    driver.manage().window().maximize();
}

@Test
public static void checkValidity(String[] array, WebDriver driver){
    String partialURL = "";
    int brokenLinks = 0;
    for (int x=1; x<array.length; x+=2){
        partialURL = anonUserSitemapExperience.getPartialURL(driver, array[x]);
        if (partialURL.isEmpty()){
            System.err.println("The link \""+array[x]+"\" intended for the "+array[x-1]+" page is either broken or an external site.");
            brokenLinks++;
            brokenLinksTot++;
        }
        else{
            int found = anonUserSitemapExperience.findMatch(array, array.length, partialURL);
            if (found<1){
                System.err.println("A match was not found for "+array[x-1]+".\n"+array[x]+"\n"+partialURL);
                brokenLinks++;
                brokenLinksTot++;
            }
        }
    }
    System.err.println("\n"+brokenLinks+" broken link(s) was/were found.\n");
}

@Test
public static void openDropDowns(WebDriver driver){
    List<WebElement> dropdownArrows = driver.findElements(By.className("dropdownToggler"));
    Iterator<WebElement> itr = dropdownArrows.iterator();
    while(itr.hasNext()){
        try{
            itr.next().click();
        }
        catch(ElementNotVisibleException e){
        }
    }
}

@Test
public static String[] createArray(List<WebElement> list){
    String[] linkArray = new String[list.size()*2];
    int counter = 0;
    for (int x=1; x<linkArray.length; x+=2){
        linkArray[x] = list.get(counter).getAttribute("href");
        try{
            linkArray[x] = linkArray[x].replaceAll("%C2%AE", "®");
            linkArray[x] = linkArray[x].replaceAll("%20", " ");
            linkArray[x] = linkArray[x].replaceAll("%27", "'");
            linkArray[x] = linkArray[x].replaceAll("%C3%A4", "ä");
            linkArray[x] = linkArray[x].replaceAll("%C3%B6", "ö");
            linkArray[x] = linkArray[x].replaceAll("%C3%BC", "ü");
            linkArray[x] = linkArray[x].replaceAll("%C3%84", "Ä");
            linkArray[x] = linkArray[x].replaceAll("%C3%96", "Ö");
            linkArray[x] = linkArray[x].replaceAll("%C3%9C", "Ü");
            linkArray[x] = linkArray[x].replaceAll("%E2%80%93", "–");
            linkArray[x] = linkArray[x].replaceAll("%E2%84%A2", "™");
            linkArray[x] = linkArray[x].replaceAll("%25", "%");
            counter++;
        }
        catch(Exception e){
        }
    }
    int counter2 = 0;
    for (int x=0; x<linkArray.length; x+=2){
        linkArray[x] = list.get(counter2).getText();
        counter2++;
    }
    return linkArray;
}

@Test
public void test() {
    driver.get(siteUS);
    List<WebElement> topNavLinks = driver.findElement(By.className("topNavigationMenu")).findElements(By.className("menuLink"));
    int numLinks = topNavLinks.size();
    String[] topNavTitlesAndLinks = new String[numLinks*2];
    topNavTitlesAndLinks = createArray(topNavLinks);
    System.out.println("Filled titles and links array.");
    for (int x=1; x<topNavTitlesAndLinks.length; x+=2){
        driver.get(topNavTitlesAndLinks[x]);
        openDropDowns(driver);
        try{
            List<WebElement> menu = driver.findElement(By.className("asideNavigationMenu")).findElements(By.className("itemLink"));
            String[] menuArray = new String[menu.size()*2];
            menuArray = createArray(menu);
            checkValidity(menuArray, driver);
        }
        catch (Exception e){
            if (topNavTitlesAndLinks[x-1].contains("Endodontics")){
                WebElement element = driver.findElement(By.linkText("Endodontics"));
                Actions action = new Actions(driver);
                action.moveToElement(element).perform();
                WebElement subElement = driver.findElement(By.partialLinkText("Access"));
                action.moveToElement(subElement);
                action.click();
                action.perform();
                openDropDowns(driver);
                List<WebElement> menu = driver.findElement(By.className("asideNavigationMenu")).findElements(By.className("itemLink"));
                String[] menuArray = new String[menu.size()*2];
                menuArray = createArray(menu);
                checkValidity(menuArray, driver);
            }
        }
    }
}

@AfterTest
public void afterTest() {
    driver.close();
    driver.quit();

    System.err.println("\nTotal broken links found: "+brokenLinksTot);

    long time2 = System.currentTimeMillis();
    double timeMin =((double)(time2-time1)/60000);
    int timeMinTrunc = (int)timeMin;
    double timeSec = (timeMin%1)*60;

    System.out.println("\n\nAll tests finished in "+timeMinTrunc+" minutes and "+timeSec+" seconds.");
}

Problem is, when I run it, I don't get pass fails from the methods that are called by test. My output is as follows:

FAILED: test

SKIPPED: checkValidity

SKIPPED: openDropDowns

So how can I make all of the called methods report as well?

Answer 1

Only the top level @Test will report results. checkValidity, IMO, would not be a test case, it is more of a verification and would cause a test to pass or fail. Remember, @Test should denote a test case, a scenario, from start to finish. I should be able to look at a @Test and know exactly what you are trying to accomplish. In your example, it looks like you are trying to validate the entries in the listbox. Your test name should reflect that, and the way it is written, the first failure hit, the test will fail. If you need to validate each and every entry first before failing, then you will need to build your failure message appropriately, and not throw an exception until after all the iterations/clicks have completed.

Answer 2

1. Non-void methods are ignored by TestNG
2. You should not call test methods yourself - it is a job for TestNG
3. Rewrite your test class so that methods annotated with @Test reflect your scenario and follow in a row. You can then use any helper methods you want.

E.g.

@BeforeClass
public void before() {
    //blah blah precondition goes here
}

@Test(priority=0)
public void start() {
    // testy test 0 goes here
}

@Test(dependsOnMethods = "start")
public void next() {
    Sysout(doSomething());
}

public String doSomething() {
    return "Helper method which I call in some test method";
}

UPD

If you want to invoke 400 times:

@DataProvider(name = "400_links")
public static String[][] getLinks() {
   // find your own way to create links here
   return new String[][] {{"http://foo"}, {"http://bar"}, ... , {"http://xyz"}};
}

@Test(dataProvider = "400_links")
public void checkAllLinks(String link) {
   checkValidityOrWhatever(link);
}

public void checkValidityOrWhatever(String link) {
   //blah blah http ftw
}

It will be launched 400 times, if you have 400 links provided in getLinks() Every iteration will be reported by Testng. Failed iterations will not stop other invocations from launch.