How is an executor termination recursion in java?

Question

This is a program that reads information site for previous format it uses recursion and executor.It works fine,my problem is to test whether the program is completed and success notification.

public class NewClass { 

    static String levels[] = { "div.col-md-9 li a", "div#sidebar ul li a" };    
    static String links = "";   

    private void getRecursive(String href, int level, final ExecutorService executor) { 

        if (level > levels.length - 1) {    
            return; 
        }   

        Document doc;   
        try {   
            doc = Jsoup.connect(href).get();    
            Elements elements = doc.select(levels[level]);  
            final int flevel = ++level; 
            for (final Element element : elements) {    
                executor.execute(new Runnable() {   
                    @Override   
                    public void run() { 
                        if (!element.attr("href").isEmpty()) {  
                            links += element.attr("abs:href") + "\n";   
                            System.out.println(links);  
                            getRecursive(element.attr("abs:href"), flevel, executor);   
                        }   
                    }   
                }); 
            }   
        } catch (IOException e1) {  
            e1.printStackTrace();   
        }   
    }   

If levels.length = 1, the rule enforcer works well,but If levels.length>1 will appear error : Exception in thread "pool-1-thread-138" java.util.concurrent.RejectedExecutionException

    public static void main(String[] args) {    
    try {
        ExecutorService executor = Executors.newFixedThreadPool(5);
        new NewClass().getRecursive("http://www.java2s.com/", 0, executor);
        executor.shutdown();
        executor.awaitTermination(1, TimeUnit.HOURS);
        if (executor.isTerminated()) {
            JOptionPane.showMessageDialog(null, "Success");
        }
    } catch (Exception ex) {
        Logger.getLogger(NewClass.class.getName()).log(Level.SEVERE, null, ex);
    }
}   
}

Answer 1

The issue is that you're calling executor.execute after calling executor.shutdown(). The awaitTermination method only waits for tasks that were submitted before shutdown was called.

Here's why this is happening:

• Let's say the program starts on thread main.
• The first call to executor.execute is on main.
• Suppose the executor now runs on thread-pool-1, it also calls executor.execute.
• But before those new tasks execute, we jump back onto main and call executor.shutdown.
• The executor might start executing tasks again, maybe on thread-pool-2 this time, and it tries to call executor.execute. However, executor.shutdown() has already been called, so a java.util.concurrent.RejectedExecutionException is thrown.

There are a few ways to solve this:

1. Using an executor might be overkill in this instance; just doing it single threaded will probably be sufficient.
2. It may be possible to refactor the code so that the last executor.execute can call executor.shutdown. However, if there isn't a distinct last call, this won't be possible.

3. If you absolutely need to do this off the main thread, you could perform the entire recursive call in its own thread:

   public static void main(String[] args) {
       try {
           ExecutorService executor = Executors.newSingleThreadExecutor();
           executor.execute(new Runnable() {
               @Override
               public void run() {
                   NewClass.getRecursive("http://www.java2s.com/", 0);
               }
           });
           executor.shutdown();
           executor.awaitTermination(1, TimeUnit.HOURS);
           if (executor.isTerminated()) {
               JOptionPane.showMessageDialog(null, "Success");
           }
       }
       catch (Exception ex)
       {
           Logger.getLogger(NewClass.class.getName()).log(Level.SEVERE, null, ex);
       }
   }
   

   and

   private static class NewClass {
   
       static String levels[] = { "div.col-md-9 li a", "div#sidebar ul li a" };
       static String links = "";
   
       private static void getRecursive(String href, int level) {
   
           if (level > levels.length - 1) {
               return;
           }
   
           Document doc;
           try {
               doc = Jsoup.connect(href).get();
               Elements elements = doc.select(levels[level]);
               final int flevel = ++level;
               for (final Element element : elements) {
                   if (!element.attr("href").isEmpty()) {
                       links += element.attr("abs:href") + "\n";
                       System.out.println(links);
                       getRecursive(element.attr("abs:href"), flevel);
                   }
               }
           }
           catch (IOException e1) {
               e1.printStackTrace();
           }
       }
   }
   

Answer 2

You can't submit any new tasks after you shutdown the ExecutorService, the recursion seems to stop after you've processed all the levels (you don't submit any new tasks after that), you can do something like this:

if (level > levels.length - 1) {    
    executor.shutdown();
    return; 
}