Map input to ast types in boost spirit

Question

I want to implement color highlighting of input string that is being fed to given spirit grammar. Is there an easy (or any, if not easy) way to map given character from input into rule/ast type that it matches? Preferable in form of array/vector of rule/ast types where index is index of character of input string. Or maybe better - iterator rages to ast types.

Answer 1

Of course there is. Several answers on this site demonstrate similar things. You'll have to decide how you want to treat subrules.

A random example using on_success:

Live On Coliru

//#define BOOST_SPIRIT_DEBUG
#include <boost/spirit/include/qi.hpp>
#include <iomanip>
namespace qi = boost::spirit::qi;

using It     = std::string::const_iterator;
using R      = boost::iterator_range<It>;
using RuleId = void const*;

struct token {
    R what;
    RuleId r_id;
}; 

struct assocociate_f {
    std::vector<token>& into;
    RuleId r_id = nullptr;

    template <typename Ctx>
    void operator()(qi::unused_type, Ctx& ctx) const {
        using boost::fusion::at_c;
        into.push_back({at_c<0>(ctx.attributes), r_id});
    }
};

int main() {
    qi::rule<It, R()> numlit, ident, oper;
    qi::rule<It, R(), qi::space_type> simple, expr;

    numlit = qi::raw[qi::double_];
    ident  = qi::raw[qi::lexeme[qi::alpha >> *qi::alnum]];
    simple = qi::raw[(numlit | ident | '(' >> expr >> ')')];
    oper   = qi::raw[qi::char_("-+*/%")];
    expr   = qi::raw[simple >> *(oper >> expr)];

    std::vector<token> collect;
    qi::on_success(numlit, assocociate_f{collect, &numlit});
    qi::on_success(ident,  assocociate_f{collect, &ident});
    qi::on_success(oper,   assocociate_f{collect, &oper});
    //qi::on_success(simple, assocociate_f{collect, &simple});
    //qi::on_success(expr,   assocociate_f{collect, &expr});

    BOOST_SPIRIT_DEBUG_NODES((numlit)(ident)(simple)(expr));

    auto idof = [&](token const& tok) -> std::string {
        auto match = [&](auto const& x) { return tok.r_id == static_cast<void const*>(&x); };
        if (match(numlit)) return "numeric literal";
        if (match(ident))  return "identifier";
        if (match(simple)) return "simple expression";
        if (match(expr))   return "expression";
        if (match(oper))   return "operator";
        return "other";
    };

    for (std::string const input : { "3 * pi + (13/47 - 5)" }) {
        std::cout << std::setw(20) << "input: " << input << "\n";
        It f = input.begin(), l = input.end();

        if (qi::phrase_parse(f, l, expr, qi::space)) {
            for (auto& tok : collect) {
                std::cout 
                    << std::setw(20) << idof(tok) + ": "
                    << std::setw(tok.what.begin() - input.begin() + 1) << tok.what
                    << "\n";
            }
        } else {
            std::cout << "Parse failed\n";
        }

        if (f!=l) {
            std::cout << "Remaining: '" << std::string(f,l) << "'\n";
        }
    }
}

Prints

             input: 3 * pi + (13/47 - 5)
   numeric literal: 3
          operator:   *
        identifier:     pi
          operator:        +
   numeric literal:           13
          operator:             /
   numeric literal:              47
          operator:                 -
   numeric literal:                   5

Uncommenting the extra

//qi::on_success(simple, assocociate_f{collect, &simple});
//qi::on_success(expr,   assocociate_f{collect, &expr});

You get: Live On Coliru

             input: 3 * pi + (13/47 - 5)
   numeric literal: 3
 simple expression: 3
          operator:   *
        identifier:     pi
 simple expression:     pi
          operator:        +
   numeric literal:           13
 simple expression:           13
          operator:             /
   numeric literal:              47
 simple expression:              47
          operator:                 -
   numeric literal:                   5
 simple expression:                   5
        expression:                   5
        expression:              47 - 5
        expression:           13/47 - 5
 simple expression:          (13/47 - 5)
        expression:          (13/47 - 5)
        expression:     pi + (13/47 - 5)
        expression: 3 * pi + (13/47 - 5)

More

A more funky example would be How to provider user with autocomplete suggestions for given boost::spirit grammar? - where string_view or string_ref is being used instead of iterator_range. Also, that "collapses" adjacent ranges to result in more usable ranges.

Other related examples:

• Cross-platform way to get line number of an INI file where given option was found
• boost spirit parse with the source