Antlr Handling Exceptions

Question

I have developed a complex grammar using Antlr 3 using AST tree. ANTLR generates the Lexer and Parser. The problem is that when the user enters a syntax that is not valid for example, the grammar is expecting ';'. The user does not enter this, then in my Eclipse IDE I get the following Exception:

 line 1:24 mismatched input '<EOF>' expecting ';'

How can this Exception be handled because I try to catch this Exception, but the Exception is not catched. Is this an Exception at all? I don't seem to understand why this Exception is not caught. I tried to find out, however the Antlr website seems to be down for some time now.

I looked at the following: ANTLR exception handling with "$", Java and followed that example, but when the Lexer generates the code by adding the RuntimeException(), I get unreachable code.

I am not sure what to do.

When I try to get the number of syntax errors from the parser it displays 0.

EDIT:

I have found a solution that works by looking at: ANTLR not throwing errors on invalid input

However, when I try to get the Exception message back, it is null. Have I set everything up correctly? Please see sample grammar:

grammar i;

options {
output=AST;
}

@header {
package com.data;
}

@rulecatch {
    catch(RecognitionException e) {
        throw e;
   }
}

// by having these below it makes no difference
/**@parser::members {
    @Override
    public void reportError(RecognitionException e) {
        throw new RuntimeException("Exception : " + " " + e.getMessage());
    }
}

@lexer::members {
    @Override
    public void reportError(RecognitionException e) {
       throw new RuntimeException("Exception : " + " " + e.getMessage());
    }
}*/

EDIT:

Please see what I have so far:

grammar i;

options {
output=AST;
}

@header {
package com.data;
}

@rulecatch {
    // ANTLR does not generate its normal rule try/catch
    catch(RecognitionException e) {
        throw e;
    }
}

@parser::members {
    @Override
    public void displayRecognitionError(String[] tokenNames, RecognitionException e) {
        String hdr = getErrorHeader(e);
        String msg = getErrorMessage(e, tokenNames);
        throw new RuntimeException(hdr + ":" + msg);
    }
}

@lexer::members {
    @Override
    public void displayRecognitionError(String[] tokenNames, RecognitionException e) {
        String hdr = getErrorHeader(e);
        String msg = getErrorMessage(e, tokenNames);
        throw new RuntimeException(hdr + ":" + msg);
    }
}

operatorLogic   : 'AND' | 'OR';
value       : STRING;
query       : (select)*;
select      : 'SELECT'^ functions 'FROM table' filters?';';
operator    : '=' | '!=' | '<' | '>' | '<=' | '>=';
filters : 'WHERE'^ conditions;
members : STRING operator value;
conditions  : (members (operatorLogic members)*);
functions   : '*';
STRING  : ('a'..'z'|'A'..'Z')+;
WS      : (' '|'\t'|'\f'|'\n'|'\r')+ {skip();}; // handle white space between keywords

public class Processor {

public Processor() {

}

/**
 * This method builds the MQL Parser.
 * @param args the args.
 * @return the built IParser.
 */
private IParser buildMQLParser(String query) {
    CharStream cs = new ANTLRStringStream(query);
    // the input needs to be lexed
    ILexer lexer = new ILexer(cs);
          CommonTokenStream tokens = new CommonTokenStream();
    IParser parser = new IParser(tokens);
    tokens.setTokenSource(lexer);
    // use the ASTTreeAdaptor so that the grammar is aware to build tree in AST format
    parser.setTreeAdaptor((TreeAdaptor) new ASTTreeAdaptor().getASTTreeAdaptor());
return parser;
}

/**
 * This method parses the MQL query.
 * @param query the query.
 */
public void parseMQL(String query) {
    IParser parser = buildMQLParser(query);
    CommonTree commonTree = null;
    try {
                     commonTree = (CommonTree) parser.query().getTree();
                    }
    catch(Exception e) {
        System.out.println("Exception :" + " " + e.getMessage());
    }
}
}

public class ASTTreeAdaptor {

public ASTTreeAdaptor() {

}

/**
 * This method is used to create a TreeAdaptor.
 * @return a treeAdaptor.
 */
public Object getASTTreeAdaptor() {
    TreeAdaptor treeAdaptor = new CommonTreeAdaptor() {
        public Object create(Token payload) {
        return new CommonTree(payload);
        }
    };
    return treeAdaptor; 
}
}

So when I enter the following: SELECT * FROM table

without a ';' I get the a MismatchedTokenException:

catch(Exception e) {
     System.out.println("Exception : " + " " e);
}

When I try:

e.getMessage();

it returns null.

Answer 1

Try overriding displayRecognitionError instead:

@parser::members { 
   ...

    @Override    
    public void displayRecognitionError(String[] tokenNames, RecognitionException e) {
        String hdr = getErrorHeader(e);
        String msg = getErrorMessage(e, tokenNames);
        throw new RuntimeException(hdr + ":" + msg);
    }
    ...
}
//same code in @lexer::members

If you want to track errors rather than abort, you could create a handler interface to track them:

@parser::members { 
   ...
    private YourErrorTrackerInterface errorTracker;

    //getter/setter for errorTracker here        

    @Override    
    public void displayRecognitionError(String[] tokenNames, RecognitionException e) {
        String hdr = getErrorHeader(e);
        String msg = getErrorMessage(e, tokenNames);
        if (errorTracker != null){
          errorTracker.addError(e, tokenNames, hdr, msg);
        }
    }
    ...
}
//same code in @lexer::members

The error tracker could then decide whether to throw an exception or continue.

---

The code above allows you to track "recoverable" errors, errors that ANTLR can skip over. There are still scenarios that produce unrecoverable errors, such as SELECT * FROM table (without the ending ;). In that case, you'll have to catch the exceptions in parseMQL or somewhere around there. (You could try writing your own recovery code, but I wouldn't recommend that you do.)

Here is a modified parseMQL that shows the two different types of parsing errors. Note that I removed the call to getMessage because not all exceptions derived off of RecognitionException fill it in.

public void parseMQL(String query) {
    iParser parser = buildMQLParser(query);
    CommonTree commonTree = null;
    try {
        commonTree = (CommonTree) parser.query().getTree();
    } catch (MismatchedTokenException e){
        //not production-quality code, just forming a useful message
        String expected = e.expecting == -1 ? "<EOF>" : iParser.tokenNames[e.expecting];
        String found = e.getUnexpectedType() == -1 ? "<EOF>" : iParser.tokenNames[e.getUnexpectedType()];

        System.out.println("Fatal mismatched token exception: expected " + expected + " but was " + found);   

    } catch (RecognitionException e) {
        System.out.println("Fatal recognition exception " + e.getClass().getName()
                + " : " + e);

    } catch (Exception e) {
        System.out.println("Other exception : " + e.getMessage());
    }
}

Input SELECT * FROM table produces message "Fatal mismatched token exception: expected ';' but was <EOF>". This exception was produced directly by ANTLR.

Input SELECT FROM table; produces message "Other exception : line 1:7:missing '*' at 'FROM table'". This exception was produced by the code above.

Answer 2

If I understand correctly you want to handle your language syntax errors. This is how I have this setup on my project.

/**
 * Adapter need for ANTL to recognize our custom nodes
 * 
 * @author Greg
 */
public class PhantomTreeAdaptor extends CommonTreeAdaptor{

    @Override
    public Object create(Token payload){
        return new ASTNode(payload);
    }

    @Override
    public Object dupNode(Object old){
        return (old == null) ? null : ((ASTNode) old).dupNode();
    }

    @Override
    public Object errorNode(TokenStream input, Token start, Token stop, RecognitionException e){
        return new ASTErrorNode(input, start, stop, e);
    }
}

Here is the Error Node

/**
 * This is our custom Error node used by the adapter.
 * 
 * @author Greg
 */
public class ASTErrorNode extends ASTNode {

    org.antlr.runtime.tree.CommonErrorNode delegate;

    public ASTErrorNode(TokenStream input, Token start, Token stop, RecognitionException e) {

        delegate = new CommonErrorNode(input, start, stop, e);

    }

    public boolean isNil() {
        return delegate.isNil();
    }

    public int getType() {
        return delegate.getType();
    }

    public String getText() {
        return delegate.getText();
    }

    public String toString() {

        return delegate.toString();
    }

}

And this is how this is all glued together.

 final PhantomSQLLexer lex = new PhantomSQLLexer(input);

        final CommonTokenStream tokens = new CommonTokenStream(lex);
        final PhantomSQLParser g = new PhantomSQLParser(tokens);
        g.setTreeAdaptor(new PhantomTreeAdaptor());
        final start_rule_return r = g.start_rule();
        if (g.getNumberOfSyntaxErrors() == 0) {
            if (LOGGER.isDebugEnabled()) {
                LOGGER.debug("tree=" + ((Tree) r.tree).toStringTree());
                LOGGER.debug("-------------------------------------------");
            }
            final ASTNode root = r.tree;
            exec(root);            
        }
        else {
            LOGGER.debug("Error parsing input");
        }

We simply created our lexer and parser then we setup our parser with custom Tree adapter (PhantomTreeAdaptor). From there we can check if we have errors in our custom code.