How to make generic computations over heterogeneou

Question

PREMISE:

After playing around with variadic templates a little bit, I realized that achieving anything which goes slightly beyond the trivial meta-programming tasks soon becomes pretty cumbersome. In particular, I found myself wishing for a way to perform generic operations over an argument pack such as iterate, split, loop in a std::for_each-like fashion, and so on.

After watching this lecture by Andrei Alexandrescu from C++ and Beyond 2012 on the desirability of static if into C++ (a construct borrowed from the D Programming Language) I had the feeling that some sort of static for would come handy as well - and I feel more of these static constructs could bring benefit.

So I started wondering if there is a way to achieve something like this for argument packs of a variadic template function (pseudo-code):

template<typename... Ts>
void my_function(Ts&&... args)
{
    static for (int i = 0; i < sizeof...(args); i++) // PSEUDO-CODE!
    {
        foo(nth_value_of<i>(args));
    }
}

Which would get translated at compile-time into something like this:

template<typename... Ts>
void my_function(Ts&&... args)
{
    foo(nth_value_of<0>(args));
    foo(nth_value_of<1>(args));
    // ...
    foo(nth_value_of<sizeof...(args) - 1>(args));
}

In principle, static_for would allow for even more elaborate processing:

template<typename... Ts>
void foo(Ts&&... args)
{
    constexpr s = sizeof...(args);

    static for (int i = 0; i < s / 2; i++)
    {
        // Do something
        foo(nth_value_of<i>(args));
    }

    static for (int i = s / 2; i < s; i++)
    {
        // Do something different
        bar(nth_value_of<i>(args));
    }
}

Or for a more expressive idiom like this one:

template<typename... Ts>
void foo(Ts&&... args)
{
    static for_each (auto&& x : args)
    {
        foo(x);
    }
}

RELATED WORK:

I did some search on the Web and found out that something does indeed exist:

• This link describes how to convert a parameter pack into a Boost.MPL vector, but that only goes half the way (if not less) towards the goal;
• this question on SO seems to call for a similar and slightly related meta-programming feature (splitting an argument pack into two halves) - actually, there are several questions on SO which seem to be related to this issue, but none of the answer I have read solves it satisfactorily IMHO;
• Boost.Fusion defines algorithms for converting an argument pack into a tuple, but I would prefer:
  1. not to create unnecessary temporaries to hold arguments that can (and should be) perfectly forwarded to some generic algorithms;
  2. have a small, self-contained library to do that, while Boost.Fusion is likely to include way more stuff than is needed to address this issue.

QUESTION:

Is there a relatively simple way, possibly through some template meta-programming, to achieve what I am looking for without incurring in the limitations of the existing approaches?

Answer 1

Since I was not happy with what I found, I tried to work out a solution myself and ended up writing a small library which allows formulating generic operations on argument packs. My solution has the following features:

• Allows iterating over all or some elements of an argument pack, possibly specified by computing their indices on the pack;
• Allows forwarding computed portions of an argument pack to variadic functors;
• Only requires including one relatively short header file;
• Makes extensive use of perfect forwarding to allow for heavy inlining and avoids unnecessary copies/moves to allow for minimum performance loss;
• The internal implementation of the iterating algorithms relies on Empty Base Class Optimization for minimizing memory consumption;
• It is easy (relatively, considering it's template meta-programming) to extend and adapt.

I will first show what can be done with the library, then post its implementation.

USE CASES

Here is an example of how the for_each_in_arg_pack() function can be used to iterate through all the arguments of a pack and pass each argument in input to some client-supplied functor (of course, the functor must have a generic call operator if the argument pack contains values of heterogenous types):

// Simple functor with a generic call operator that prints its input. This is used by the
// following functors and by some demonstrative test cases in the main() routine.
struct print
{
    template<typename T>
    void operator () (T&& t)
    {
        cout << t << endl;
    }
};

// This shows how a for_each_*** helper can be used inside a variadic template function
template<typename... Ts>
void print_all(Ts&&... args)
{
    for_each_in_arg_pack(print(), forward<Ts>(args)...);
}

The print functor above can also be used in more complex computations. In particular, here is how one would iterate on a subset (in this case, a sub-range) of the arguments in a pack:

// Shows how to select portions of an argument pack and 
// invoke a functor for each of the selected elements
template<typename... Ts>
void split_and_print(Ts&&... args)
{
    constexpr size_t packSize = sizeof...(args);
    constexpr size_t halfSize = packSize / 2;

    cout << "Printing first half:" << endl;
    for_each_in_arg_pack_subset(
        print(), // The functor to invoke for each element
        index_range<0, halfSize>(), // The indices to select
        forward<Ts>(args)... // The argument pack
        );

    cout << "Printing second half:" << endl;
    for_each_in_arg_pack_subset(
        print(), // The functor to invoke for each element
        index_range<halfSize, packSize>(), // The indices to select
        forward<Ts>(args)... // The argument pack
        );
}

Sometimes, one may just want to forward a portion of an argument pack to some other variadic functor instead of iterating through its elements and pass each of them individually to a non-variadic functor. This is what the forward_subpack() algorithm allows doing:

// Functor with variadic call operator that shows the usage of for_each_*** 
// to print all the arguments of a heterogeneous pack
struct my_func
{
    template<typename... Ts>
    void operator ()(Ts&&... args)
    {
        print_all(forward<Ts>(args)...);
    }
};

// Shows how to forward only a portion of an argument pack 
// to another variadic functor
template<typename... Ts>
void split_and_print(Ts&&... args)
{
    constexpr size_t packSize = sizeof...(args);
    constexpr size_t halfSize = packSize / 2;

    cout << "Printing first half:" << endl;
    forward_subpack(my_func(), index_range<0, halfSize>(), forward<Ts>(args)...);

    cout << "Printing second half:" << endl;
    forward_subpack(my_func(), index_range<halfSize, packSize>(), forward<Ts>(args)...);
}

For more specific tasks, it is of course possible to retrieve specific arguments in a pack by indexing them. This is what the nth_value_of() function allows doing, together with its helpers first_value_of() and last_value_of():

// Shows that arguments in a pack can be indexed
template<unsigned I, typename... Ts>
void print_first_last_and_indexed(Ts&&... args)
{
    cout << "First argument: " << first_value_of(forward<Ts>(args)...) << endl;
    cout << "Last argument: " << last_value_of(forward<Ts>(args)...) << endl;
    cout << "Argument #" << I << ": " << nth_value_of<I>(forward<Ts>(args)...) << endl;
}

If the argument pack is homogeneous on the other hand (i.e. all arguments have the same type), a formulation such as the one below might be preferable. The is_homogeneous_pack<> meta-function allows determining whether all the types in a parameter pack are homogeneous, and is mainly meant to be used in static_assert() statements:

// Shows the use of range-based for loops to iterate over a
// homogeneous argument pack
template<typename... Ts>
void print_all(Ts&&... args)
{
    static_assert(
        is_homogeneous_pack<Ts...>::value, 
        "Template parameter pack not homogeneous!"
        );

    for (auto&& x : { args... })
    {
        // Do something with x...
    }

    cout << endl;
}

Finally, since lambdas are just syntactic sugar for functors, they can be used as well in combination with the algorithms above; however, until generic lambdas will be supported by C++, this is only possible for homogeneous argument packs. The following example also shows the usage of the homogeneous-type<> meta-function, which returns the type of all arguments in a homogeneous pack:

 // ...
 static_assert(
     is_homogeneous_pack<Ts...>::value, 
     "Template parameter pack not homogeneous!"
     );
 using type = homogeneous_type<Ts...>::type;
 for_each_in_arg_pack([] (type const& x) { cout << x << endl; }, forward<Ts>(args)...);

This is basically what the library allows doing, but I believe it could even be extended to carry out more complex tasks.

IMPLEMENTATION

Now comes the implementation, which is a bit tricky in itself so I will rely on comments to explain the code and avoid making this post too long (perhaps it already is):

#include <type_traits>
#include <utility>

//===============================================================================
// META-FUNCTIONS FOR EXTRACTING THE n-th TYPE OF A PARAMETER PACK

// Declare primary template
template<int I, typename... Ts>
struct nth_type_of
{
};

// Base step
template<typename T, typename... Ts>
struct nth_type_of<0, T, Ts...>
{
    using type = T;
};

// Induction step
template<int I, typename T, typename... Ts>
struct nth_type_of<I, T, Ts...>
{
    using type = typename nth_type_of<I - 1, Ts...>::type;
};

// Helper meta-function for retrieving the first type in a parameter pack
template<typename... Ts>
struct first_type_of
{
    using type = typename nth_type_of<0, Ts...>::type;
};

// Helper meta-function for retrieving the last type in a parameter pack
template<typename... Ts>
struct last_type_of
{
    using type = typename nth_type_of<sizeof...(Ts) - 1, Ts...>::type;
};

//===============================================================================
// FUNCTIONS FOR EXTRACTING THE n-th VALUE OF AN ARGUMENT PACK

// Base step
template<int I, typename T, typename... Ts>
auto nth_value_of(T&& t, Ts&&... args) ->
    typename std::enable_if<(I == 0), decltype(std::forward<T>(t))>::type
{
    return std::forward<T>(t);
}

// Induction step
template<int I, typename T, typename... Ts>
auto nth_value_of(T&& t, Ts&&... args) ->
    typename std::enable_if<(I > 0), decltype(
        std::forward<typename nth_type_of<I, T, Ts...>::type>(
            std::declval<typename nth_type_of<I, T, Ts...>::type>()
            )
        )>::type
{
    using return_type = typename nth_type_of<I, T, Ts...>::type;
    return std::forward<return_type>(nth_value_of<I - 1>((std::forward<Ts>(args))...));
}

// Helper function for retrieving the first value of an argument pack
template<typename... Ts>
auto first_value_of(Ts&&... args) ->
    decltype(
        std::forward<typename first_type_of<Ts...>::type>(
            std::declval<typename first_type_of<Ts...>::type>()
            )
        )
{
    using return_type = typename first_type_of<Ts...>::type;
    return std::forward<return_type>(nth_value_of<0>((std::forward<Ts>(args))...));
}

// Helper function for retrieving the last value of an argument pack
template<typename... Ts>
auto last_value_of(Ts&&... args) ->
    decltype(
        std::forward<typename last_type_of<Ts...>::type>(
            std::declval<typename last_type_of<Ts...>::type>()
            )
        )
{
    using return_type = typename last_type_of<Ts...>::type;
    return std::forward<return_type>(nth_value_of<sizeof...(Ts) - 1>((std::forward<Ts>(args))...));
}

//===============================================================================
// METAFUNCTION FOR COMPUTING THE UNDERLYING TYPE OF HOMOGENEOUS PARAMETER PACKS

// Used as the underlying type of non-homogeneous parameter packs
struct null_type
{
};

// Declare primary template
template<typename... Ts>
struct homogeneous_type;

// Base step
template<typename T>
struct homogeneous_type<T>
{
    using type = T;
    static const bool isHomogeneous = true;
};

// Induction step
template<typename T, typename... Ts>
struct homogeneous_type<T, Ts...>
{
    // The underlying type of the tail of the parameter pack
    using type_of_remaining_parameters = typename homogeneous_type<Ts...>::type;

    // True if each parameter in the pack has the same type
    static const bool isHomogeneous = std::is_same<T, type_of_remaining_parameters>::value;

    // If isHomogeneous is "false", the underlying type is the fictitious null_type
    using type = typename std::conditional<isHomogeneous, T, null_type>::type;
};

// Meta-function to determine if a parameter pack is homogeneous
template<typename... Ts>
struct is_homogeneous_pack
{
    static const bool value = homogeneous_type<Ts...>::isHomogeneous;
};

//===============================================================================
// META-FUNCTIONS FOR CREATING INDEX LISTS

// The structure that encapsulates index lists
template <unsigned... Is>
struct index_list
{
};

// Collects internal details for generating index ranges [MIN, MAX)
namespace detail
{
    // Declare primary template for index range builder
    template <unsigned MIN, unsigned N, unsigned... Is>
    struct range_builder;

    // Base step
    template <unsigned MIN, unsigned... Is>
    struct range_builder<MIN, MIN, Is...>
    {
        typedef index_list<Is...> type;
    };

    // Induction step
    template <unsigned MIN, unsigned N, unsigned... Is>
    struct range_builder : public range_builder<MIN, N - 1, N - 1, Is...>
    {
    };
}

// Meta-function that returns a [MIN, MAX) index range
template<unsigned MIN, unsigned MAX>
using index_range = typename detail::range_builder<MIN, MAX>::type;

//===============================================================================
// CLASSES AND FUNCTIONS FOR REALIZING LOOPS ON ARGUMENT PACKS

// Implementation inspired by @jogojapan's answer to this question:
// http://stackoverflow.com/questions/14089637/return-several-arguments-for-another-function-by-a-single-function

// Collects internal details for implementing functor invocation
namespace detail
{
    // Functor invocation is realized through variadic inheritance.
    // The constructor of each base class invokes an input functor.
    // An functor invoker for an argument pack has one base class
    // for each argument in the pack

    // Realizes the invocation of the functor for one parameter
    template<unsigned I, typename T>
    struct invoker_base
    {
        template<typename F, typename U>
        invoker_base(F&& f, U&& u) { f(u); }
    };

    // Necessary because a class cannot inherit the same class twice
    template<unsigned I, typename T>
    struct indexed_type
    {
        static const unsigned int index = I;
        using type = T;
    };

    // The functor invoker: inherits from a list of base classes.
    // The constructor of each of these classes invokes the input
    // functor with one of the arguments in the pack.
    template<typename... Ts>
    struct invoker : public invoker_base<Ts::index, typename Ts::type>...
    {
        template<typename F, typename... Us>
        invoker(F&& f, Us&&... args)
            :
            invoker_base<Ts::index, typename Ts::type>(std::forward<F>(f), std::forward<Us>(args))...
        {
        }
    };
}

// The functor provided in the first argument is invoked for each
// argument in the pack whose index is contained in the index list
// specified in the second argument
template<typename F, unsigned... Is, typename... Ts>
void for_each_in_arg_pack_subset(F&& f, index_list<Is...> const& i, Ts&&... args)
{
    // Constructors of invoker's sub-objects will invoke the functor.
    // Note that argument types must be paired with numbers because the
    // implementation is based on inheritance, and one class cannot
    // inherit the same base class twice.
    detail::invoker<detail::indexed_type<Is, typename nth_type_of<Is, Ts...>::type>...> invoker(
        f,
        (nth_value_of<Is>(std::forward<Ts>(args)...))...
        );
}

// The functor provided in the first argument is invoked for each
// argument in the pack
template<typename F, typename... Ts>
void for_each_in_arg_pack(F&& f, Ts&&... args)
{
    for_each_in_arg_pack_subset(f, index_range<0, sizeof...(Ts)>(), std::forward<Ts>(args)...);
}

// The functor provided in the first argument is given in input the
// arguments in whose index is contained in the index list specified
// as the second argument.
template<typename F, unsigned... Is, typename... Ts>
void forward_subpack(F&& f, index_list<Is...> const& i, Ts&&... args)
{
    f((nth_value_of<Is>(std::forward<Ts>(args)...))...);
}

// The functor provided in the first argument is given in input all the
// arguments in the pack.
template<typename F, typename... Ts>
void forward_pack(F&& f, Ts&&... args)
{
    f(std::forward<Ts>(args)...);
}

CONCLUSION

Of course, even though I provided my own answer to this question (and actually because of this fact), I am curious to hear if alternative or better solutions exist which I have missed - apart from the ones mentioned in the "Related Works" section of the question.

Answer 2

Using an enumerate solution (ala Python).

Usage:

void fun(int i, size_t index, size_t size) {
    if (index != 0) {
        std::cout << ", ";
    }

    std::cout << i;

    if (index == size - 1) {
        std::cout << "\n";
    }
} // fun

enumerate(fun, 2, 3, 4);

// Expected output: "2, 3, 4\n"
// check it at: http://liveworkspace.org/code/1cydbw$4

Code:

// Fun: expects a callable of 3 parameters: Arg, size_t, size_t
// Arg: forwarded argument
// size_t: index of current argument
// size_t: number of arguments
template <typename Fun, typename... Args, size_t... Is>
void enumerate_impl(Fun&& fun, index_list<Is...>, Args&&... args) {
    std::initializer_list<int> _{
        (fun(std::forward<Args>(args), Is, sizeof...(Is)), 0)...
    };
    (void)_; // placate compiler, only the side-effects interest us
}

template <typename Fun, typename... Args>
void enumerate(Fun&& fun, Args&&... args) {
    enumerate_impl(fun,
                   index_range<0, sizeof...(args)>(),
                   std::forward<Args>(args)...);
}

The range builder (pilferred from your solution):

// The structure that encapsulates index lists
template <size_t... Is>
struct index_list
{
};

// Collects internal details for generating index ranges [MIN, MAX)
namespace detail
{
    // Declare primary template for index range builder
    template <size_t MIN, size_t N, size_t... Is>
    struct range_builder;

    // Base step
    template <size_t MIN, size_t... Is>
    struct range_builder<MIN, MIN, Is...>
    {
        typedef index_list<Is...> type;
    };

    // Induction step
    template <size_t MIN, size_t N, size_t... Is>
    struct range_builder : public range_builder<MIN, N - 1, N - 1, Is...>
    {
    };
}

// Meta-function that returns a [MIN, MAX) index range
template<size_t MIN, size_t MAX>
using index_range = typename detail::range_builder<MIN, MAX>::type;

Answer 3

Let me post this code, based on the discussion:

#include <initializer_list>
#define EXPAND(EXPR) std::initializer_list<int>{((EXPR),0)...}

// Example of use:
#include <iostream>
#include <utility>

void print(int i){std::cout << "int: " << i << '\n';}
int print(double d){std::cout << "double: " << d << '\n';return 2;}

template<class...T> void f(T&&...args){
  EXPAND(print(std::forward<T>(args)));
}

int main(){
  f();
  f(1,2.,3);
}

I checked the generated code with g++ -std=c++11 -O1 and main only contains 3 calls to print, there is no trace of the expansion helpers.

Answer 4

After reading a few other posts and tinkering for a while I came up with the following (somewhat similar to the above, but the implementation is a little different). I wrote this using the Visual Studio 2013 compiler.

Usage using a lambda expression -

static_for_each()(
    [](std::string const& str)
    {
        std::cout << str << std::endl;
    }, "Hello, ", "Lambda!");

The downside when using a lambda is the parameters must be of the same type declared in the lambda's parameter list. This means it will only work with one type. If you want to use a templated function, you can use the next example.

Usage using struct wrapper functor -

struct print_wrapper
{
    template <typename T>
    void operator()(T&& str)
    {
        std::cout << str << " ";
    }
};

// 
// A little test object we can use.
struct test_object
{
    test_object() : str("I'm a test object!") {}
    std::string str;
};

std::ostream& operator<<(std::ostream& os, test_object t)
{
    os << t.str;
    return os;
}

//
// prints: "Hello, Functor! 1 2 I'm a test object!"
static_for_each()(print_wrapper(), "Hello,", "Functor!", 1, 2.0f, test_object());

This allows you to pass in any types you'd like and operate on them using the functor. I found this pretty clean and works nicely for what I wanted. You can also use it with a function parameter pack like this -

template <typename T, typename... Args>
void call(T f, Args... args)
{
    static_for_each()(f, args...);
}

call(print_wrapper(), "Hello", "Call", "Wrapper!");

Here is the implementation -

// 
// Statically iterate over a parameter pack 
// and call a functor passing each argument.
struct static_for_each
{
private:
    // 
    // Get the parameter pack argument at index i.
    template <size_t i, typename... Args>
    static auto get_arg(Args&&... as) 
    -> decltype(std::get<i>(std::forward_as_tuple(std::forward<Args>(as)...)))
    {
        return std::get<i>(std::forward_as_tuple(std::forward<Args>(as)...));
    }

    //
    // Recursive template for iterating over 
    // parameter pack and calling the functor.
    template <size_t Start, size_t End>
    struct internal_static_for
    {
        template <typename Functor, typename... Ts>
        void operator()(Functor f, Ts&&... args)
        {
            f(get_arg<Start>(args...));
            internal_static_for<Start + 1, End>()(f, args...);
        }
    };

    //
    // Specialize the template to end the recursion.
    template <size_t End>
    struct internal_static_for<End, End>
    {
        template <typename Functor, typename... Ts>
        void operator()(Functor f, Ts&&... args){}
    };

public:
    // 
    // Publically exposed operator()(). 
    // Handles template recursion over parameter pack.
    // Takes the functor to be executed and a parameter 
    // pack of arguments to pass to the functor, one at a time.
    template<typename Functor, typename... Ts>
    void operator()(Functor f, Ts&&... args)
    {
        // 
        // Statically iterate over parameter
        // pack from the first argument to the
        // last, calling functor f with each 
        // argument in the parameter pack.
        internal_static_for<0u, sizeof...(Ts)>()(f, args...);
    }
};

Hope people find this useful :-)

Answer 5

The ... notation does have some interesting options, like:

template<typename T>
int print(const T& x) {
  std::cout << "<" << x << ">";
  return 0;
}

void pass(...) {}

template<typename... TS>
void printall(TS... ts){
  pass(print(ts)...);
}

Unfortunately, I don't know of any way to enforce the order in which the print functions are called (reverse, on my compiler). Note that print needs to return something.

This trick can be useful if you don't care about order.