It's a good idea to use boost::program_options

I have to deal with a lot of files with a well defined syntax and semantic, for example:

the first line it's an header with special info
the other lines are containing a key value at the start of the line that are telling you how to parse and deal with the content of that line
if there is a comment it starts with a given token
etc etc ...

now boost::program_options, as far as I can tell, does pretty much the same job, but I only care about importing the content of those text file, without any extra work in between, just parse it and store it in my data structure .

the key step for me is that I would like to be able to do this parsing with:

regular expressions since I need to detect different semantics and I can't really imagine another way to do this
error checking ( corrupted file, unmatched keys even after parsing the entire file, etc etc ... )

so, I can use this library for this job ? There is a more functional approach ?

回答1:

Okay, a starting point for a Spirit grammar

_Name    = "newmtl" >> lexeme [ +graph ];
_Ns      = "Ns"     >> double_;
_Ka      = "Ka"     >> double_ >> double_ >> double_;
_Kd      = "Kd"     >> double_ >> double_ >> double_;
_Ks      = "Ks"     >> double_ >> double_ >> double_;
_d       = "d"      >> double_;
_illum  %= "illum"  >> qi::int_ [ _pass = (_1>=0) && (_1<=10) ];

comment  = '#' >> *(char_ - eol);

statement=
         comment
       | _Ns    [ bind(&material::_Ns, _r1) = _1 ]
       | _Ka    [ bind(&material::_Ka, _r1) = _1 ]
       | _Kd    [ bind(&material::_Kd, _r1) = _1 ]
       | _Ks    [ bind(&material::_Ks, _r1) = _1 ]
       | _d     [ bind(&material::_d,  _r1) = _1 ]
       | _illum [ bind(&material::_illum, _r1) = _1 ]
       ;

_material = -comment % eol
        >> _Name [ bind(&material::_Name, _val) = _1 ] >> eol
        >> -statement(_val) % eol;

start = _material % -eol;

I only implemented the MTL file subset grammar from your sample files.

Note: This is rather a simplistic grammar. But, you know, first things first. In reality I'd probably consider using the keyword list parser from the spirit repository. It has facilities to 'require' certain number of occurrences for the different 'field types'.

Note: Spirit Karma (and some ~50 other lines of code) are only here for demonstrational purposes.

With the following contents of untitled.mtl

# Blender MTL File: 'None'
# Material Count: 2

newmtl None
Ns 0
Ka 0.000000 0.000000 0.000000
Kd 0.8 0.8 0.8
Ks 0.8 0.8 0.8
d 1
illum 2
# Added just for testing:

newmtl Demo
Ns 1
Ks 0.9 0.9 0.9
d 42
illum 7

The output reads

phrase_parse -> true
remaining input: ''
void dump(const T&) [with T = std::vector<blender::mtl::material>]
-----
material {
    Ns:0
    Ka:{r:0,g:0,b:0}
    Kd:{r:0.8,g:0.8,b:0.8}
    Ks:{r:0.8,g:0.8,b:0.8}
    d:1
    illum:2(Highlight on)
}
material {
    Ns:1
    Ka:(unspecified)
    Kd:(unspecified)
    Ks:{r:0.9,g:0.9,b:0.9}
    d:42
    illum:7(Transparency: Refraction on/Reflection: Fresnel on and Ray trace on)
}
-----

Here's the listing

#define BOOST_SPIRIT_USE_PHOENIX_V3
#define BOOST_SPIRIT_DEBUG

#include <boost/fusion/adapted.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/karma.hpp> // for debug output/streaming
#include <boost/spirit/include/phoenix.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>

namespace qi = boost::spirit::qi;
namespace phx= boost::phoenix;

namespace wavefront { namespace obj
{
} }

namespace blender { namespace mtl // material?
{
    struct Ns { int exponent; }; // specular exponent
    struct Reflectivity { double r, g, b; };

    using Name = std::string;
    using Ka   = Reflectivity;
    using Kd   = Reflectivity;
    using Ks   = Reflectivity;

    using dissolve_factor = double;
    enum class illumination_model {
            color,          // 0     Color on and Ambient off
            color_ambient,  // 1     Color on and Ambient on
            highlight,      // 2     Highlight on
            reflection_ray, // 3     Reflection on and Ray trace on
            glass_ray,      // 4     Transparency: Glass on
                            //       Reflection: Ray trace on
            fresnel_ray,    // 5     Reflection: Fresnel on and Ray trace on
            refract_ray,    // 6     Transparency: Refraction on
                            //       Reflection: Fresnel off and Ray trace on
            refract_ray_fresnel,// 7 Transparency: Refraction on
                            //       Reflection: Fresnel on and Ray trace on
            reflection,     // 8     Reflection on and Ray trace off
            glass,          // 9     Transparency: Glass on
                            //       Reflection: Ray trace off
            shadow_invis,   // 10    Casts shadows onto invisible surfaces
    };

    struct material
    {
        Name                                _Name;
        boost::optional<Ns>                 _Ns;
        boost::optional<Reflectivity>       _Ka;
        boost::optional<Reflectivity>       _Kd;
        boost::optional<Reflectivity>       _Ks;
        boost::optional<dissolve_factor>    _d;
        boost::optional<illumination_model> _illum;
    };

    using mtl_file = std::vector<material>;

    ///////////////////////////////////////////////////////////////////////
    // Debug output helpers
    std::ostream& operator<<(std::ostream& os, blender::mtl::illumination_model o)
    {
        using blender::mtl::illumination_model;
        switch(o)
        {
            case illumination_model::color:               return os << "0(Color on and Ambient off)";
            case illumination_model::color_ambient:       return os << "1(Color on and Ambient on)";
            case illumination_model::highlight:           return os << "2(Highlight on)";
            case illumination_model::reflection_ray:      return os << "3(Reflection on and Ray trace on)";
            case illumination_model::glass_ray:           return os << "4(Transparency: Glass on/Reflection: Ray trace on)";
            case illumination_model::fresnel_ray:         return os << "5(Reflection: Fresnel on and Ray trace on)";
            case illumination_model::refract_ray:         return os << "6(Transparency: Refraction on/Reflection: Fresnel off and Ray trace on)";
            case illumination_model::refract_ray_fresnel: return os << "7(Transparency: Refraction on/Reflection: Fresnel on and Ray trace on)";
            case illumination_model::reflection:          return os << "8(Reflection on and Ray trace off)";
            case illumination_model::glass:               return os << "9(Transparency: Glass on/Reflection: Ray trace off)";
            case illumination_model::shadow_invis:        return os << "10(Casts shadows onto invisible surfaces)";
            default: return os << "ILLEGAL VALUE";
        }
    }

    std::ostream& operator<<(std::ostream& os, blender::mtl::Reflectivity const& o)
    {
        return os << "{r:" << o.r << ",g:" << o.g << ",b:" << o.b << "}";
    }

    std::ostream& operator<<(std::ostream& os, blender::mtl::material const& o)
    {
        using namespace boost::spirit::karma;
        return os << format("material {"
                "\n\tNs:"    << (auto_  | "(unspecified)")
                << "\n\tKa:"    << (stream | "(unspecified)")
                << "\n\tKd:"    << (stream | "(unspecified)")
                << "\n\tKs:"    << (stream | "(unspecified)")
                << "\n\td:"     << (stream | "(unspecified)")
                << "\n\tillum:" << (stream | "(unspecified)")
                << "\n}", o);
    }
} }

BOOST_FUSION_ADAPT_STRUCT(blender::mtl::Reflectivity,(double, r)(double, g)(double, b))
BOOST_FUSION_ADAPT_STRUCT(blender::mtl::Ns, (int, exponent))
BOOST_FUSION_ADAPT_STRUCT(blender::mtl::material,
        (boost::optional<blender::mtl::Ns>, _Ns)
        (boost::optional<blender::mtl::Ka>, _Ka)
        (boost::optional<blender::mtl::Kd>, _Kd)
        (boost::optional<blender::mtl::Ks>, _Ks)
        (boost::optional<blender::mtl::dissolve_factor>, _d)
        (boost::optional<blender::mtl::illumination_model>, _illum))

namespace blender { namespace mtl { namespace parsing
{
    template <typename It>
        struct grammar : qi::grammar<It, qi::blank_type, mtl_file()>
    {
        template <typename T=qi::unused_type> using rule = qi::rule<It, qi::blank_type, T>;

        rule<Name()>               _Name;
        rule<Ns()>                 _Ns;
        rule<Reflectivity()>       _Ka;
        rule<Reflectivity()>       _Kd;
        rule<Reflectivity()>       _Ks;
        rule<dissolve_factor()>    _d;
        rule<illumination_model()> _illum;

        rule<mtl_file()> start;
        rule<material()> _material;
        rule<void(material&)> statement;
        rule<> comment;

        grammar() : grammar::base_type(start)
        {
            using namespace qi;
            using phx::bind;
            using blender::mtl::material;

            _Name    = "newmtl" >> lexeme [ +graph ];
            _Ns      = "Ns"     >> double_;
            _Ka      = "Ka"     >> double_ >> double_ >> double_;
            _Kd      = "Kd"     >> double_ >> double_ >> double_;
            _Ks      = "Ks"     >> double_ >> double_ >> double_;
            _d       = "d"      >> double_;
            _illum  %= "illum"  >> qi::int_ [ _pass = (_1>=0) && (_1<=10) ];

            comment  = '#' >> *(char_ - eol);

            statement=
                    comment
                | _Ns    [ bind(&material::_Ns, _r1) = _1 ]
                | _Ka    [ bind(&material::_Ka, _r1) = _1 ]
                | _Kd    [ bind(&material::_Kd, _r1) = _1 ]
                | _Ks    [ bind(&material::_Ks, _r1) = _1 ]
                | _d     [ bind(&material::_d,  _r1) = _1 ]
                | _illum [ bind(&material::_illum, _r1) = _1 ]
                ;

            _material = -comment % eol
                    >> _Name [ bind(&material::_Name, _val) = _1 ] >> eol
                    >> -statement(_val) % eol;

            start = _material % -eol;

            BOOST_SPIRIT_DEBUG_NODES(
                    (start)
                    (statement)
                    (_material)
                    (_Name) (_Ns) (_Ka) (_Kd) (_Ks) (_d) (_illum)
                    (comment))
        }

};

} } }

#include <fstream>

template <typename T>
void dump(T const& data)
{
    using namespace boost::spirit::karma;
    std::cout << __PRETTY_FUNCTION__
        << "\n-----\n"
        << format(stream % eol, data)
        << "\n-----\n";
}

void testMtl(const char* const fname)
{
    std::ifstream mtl(fname, std::ios::binary);
    mtl.unsetf(std::ios::skipws);
    boost::spirit::istream_iterator f(mtl), l;

    using namespace blender::mtl::parsing;
    static const grammar<decltype(f)> p;

    blender::mtl::mtl_file data;
    bool ok = qi::phrase_parse(f, l, p, qi::blank, data);

    std::cout << "phrase_parse -> " << std::boolalpha << ok << "\n";
    std::cout << "remaining input: '" << std::string(f,l) << "'\n";

    dump(data);
}

int main()
{
    testMtl("untitled.mtl");
}

回答2:

Yes, at least if you config file as simple as map of key-value pairs (something like simple .ini).

From documentation:

The program_options library allows program developers to obtain program options, that is (name, value) pairs from the user, via conventional methods such as command line and config file.

...

Options can be read from anywhere. Sooner or later the command line will be not enough for your users, and you'll want config files or maybe even environment variables. These can be added without significant effort on your part.

See "multiple sources" sample for details.

But, if you need (or could probably need in the future) a more sophisticated config files (XML, JSON or binary for example), it is worth to use standalone library.

回答3:

It's most likely possible, but not necessarily convenient. If you want to parse anything you want to use parser - whether you use existing one or write one yourself depends on what you are parsing.

If there is no way to parse your format with any existing tool then just write your own parser. You can use lex/flex/flex++ with yacc/bison/bison++ or boost::spirit.

I think in a long run learning to maintain you own parser will be more useful that forcefully adjusting boost::program_options config, but not as convenient as using some existing parser already matching your needs.