Very poor boost::lexical_cast performance

Question

Windows XP SP3. Core 2 Duo 2.0 GHz. I'm finding the boost::lexical_cast performance to be extremely slow. Wanted to find out ways to speed up the code. Using /O2 optimizations on visual c++ 2008 and comparing with java 1.6 and python 2.6.2 I see the following results.

Integer casting:

c++: 
std::string s ;
for(int i = 0; i < 10000000; ++i)
{
    s = boost::lexical_cast<string>(i);
}

java:
String s = new String();
for(int i = 0; i < 10000000; ++i)
{
    s = new Integer(i).toString();
}

python:
for i in xrange(1,10000000):
    s = str(i)

The times I'm seeing are

c++: 6700 milliseconds

java: 1178 milliseconds

python: 6702 milliseconds

c++ is as slow as python and 6 times slower than java.

Double casting:

c++:
std::string s ;
for(int i = 0; i < 10000000; ++i)
{
    s = boost::lexical_cast<string>(d);
}

java:
String s = new String();
for(int i = 0; i < 10000000; ++i)
{
    double d = i*1.0;
    s = new Double(d).toString();
}

python:
for i in xrange(1,10000000):
    d = i*1.0
    s = str(d)

The times I'm seeing are

c++: 56129 milliseconds

java: 2852 milliseconds

python: 30780 milliseconds

So for doubles c++ is actually half the speed of python and 20 times slower than the java solution!!. Any ideas on improving the boost::lexical_cast performance? Does this stem from the poor stringstream implementation or can we expect a general 10x decrease in performance from using the boost libraries.

Answer 1

You could specialize lexical_cast for int and double types. Use strtod and strtol in your's specializations.

namespace boost {
template<>
inline int lexical_cast(const std::string& arg)
{
    char* stop;
    int res = strtol( arg.c_str(), &stop, 10 );
    if ( *stop != 0 ) throw_exception(bad_lexical_cast(typeid(int), typeid(std::string)));
    return res;
}
template<>
inline std::string lexical_cast(const int& arg)
{
    char buffer[65]; // large enough for arg < 2^200
    ltoa( arg, buffer, 10 );
    return std::string( buffer ); // RVO will take place here
}
}//namespace boost

int main(int argc, char* argv[])
{
    std::string str = "22"; // SOME STRING
    int int_str = boost::lexical_cast<int>( str );
    std::string str2 = boost::lexical_cast<std::string>( str_int );

    return 0;
}

This variant will be faster than using default implementation, because in default implementation there is construction of heavy stream objects. And it is should be little faster than printf, because printf should parse format string.

Answer 2

I use this very fast solution for POD types...

namespace DATATYPES {

    typedef std::string   TString;
    typedef char*         TCString;
    typedef double        TDouble;
    typedef long          THuge;
    typedef unsigned long TUHuge;
};

namespace boost {

template<typename TYPE>
inline const DATATYPES::TString lexical_castNumericToString(

                                const TYPE& arg, 
                                const DATATYPES::TCString fmt) {

    enum { MAX_SIZE = ( std::numeric_limits<TYPE>::digits10 + 1 )  // sign
                                                            + 1 }; // null
    char buffer[MAX_SIZE] = { 0 };

    if (sprintf(buffer, fmt, arg) < 0) {
        throw_exception(bad_lexical_cast(typeid(TYPE),
                                         typeid(DATATYPES::TString)));
    }
    return ( DATATYPES::TString(buffer) );
}

template<typename TYPE>
inline const TYPE lexical_castStringToNumeric(const DATATYPES::TString& arg) {

    DATATYPES::TCString end = 0;
    DATATYPES::TDouble result = std::strtod(arg.c_str(), &end);

    if (not end or *end not_eq 0) {
        throw_exception(bad_lexical_cast(typeid(DATATYPES::TString),
                                         typeid(TYPE)));
    }
    return TYPE(result);
}

template<>
inline DATATYPES::THuge lexical_cast(const DATATYPES::TString& arg) {
    return (lexical_castStringToNumeric<DATATYPES::THuge>(arg));
}

template<>
inline DATATYPES::TString lexical_cast(const DATATYPES::THuge& arg) {
    return (lexical_castNumericToString<DATATYPES::THuge>(arg,"%li"));
}

template<>
inline DATATYPES::TUHuge lexical_cast(const DATATYPES::TString& arg) {
    return (lexical_castStringToNumeric<DATATYPES::TUHuge>(arg));
}

template<>
inline DATATYPES::TString lexical_cast(const DATATYPES::TUHuge& arg) {
    return (lexical_castNumericToString<DATATYPES::TUHuge>(arg,"%lu"));
}

template<>
inline DATATYPES::TDouble lexical_cast(const DATATYPES::TString& arg) {
    return (lexical_castStringToNumeric<DATATYPES::TDouble>(arg));
}

template<>
inline DATATYPES::TString lexical_cast(const DATATYPES::TDouble& arg) {
    return (lexical_castNumericToString<DATATYPES::TDouble>(arg,"%f"));
}

} // end namespace boost

Answer 3

As Barry said, lexical_cast is very general, you should use a more specific alternative, for example check out itoa (int->string) and atoi (string -> int).