Transducers – what are they? This is term from Clojure world. There is a classic video from inventor of this term Rich Hickey, and here is some manual page from closure’s docs. But I’ll try to explain all this the simplest possible way so you don’t have to watch all that stuff.

Also I will show my way to implement transducers using C++14 and a lot of examples how to use them.

The idea is simple – if You have sequential process and each inner step could be represented as functional composition of reducing functions – we could make ‘composer tools’ for it (high order functions) and call them transducers.

Perfect analogy from real world – baggage queue in airport. Each bag should go through several operations – check bag, wrap bag, get weight, pass bag, etc. Each iteration could be represented as some reducing function. And transducer is the way to make composition of operations – transduce one reducing function into another (producing desired composition).

So we can write chain of map, reduce, filter, limit, etc but in form of transducers. Instead of performing iterations each of them will produce composition of actions for single sequence element. And finally, when we execute whole functional chain there will be only one cycle through range.

There are already working libs for C++ available so you can play with transducers – one of them is open-source lib called Atria. Enjoy presentation about Atria from cppcon 2015.

But actually it’s not so complicated to implement your own version of transducers. My main aim was to integrate them into my working functional data processing scheme (which is described here). And also create the best possible syntax sugar to make life sweet.

Notice My implementation is a bit different from atria’s one – the difference will be explained below.

USAGE EXAMPLES

Let’s start with basic examples (which were at my slides from MeetingCPP 2015). Even if you are not familiar with basic ideas of functional processing the samples will still be quite readable.

First example is just composition of filter and map for primitive types. I took integers only as example, of course it’s not limited to them. The following example takes only even numbers from vector of ints and than adds one to each of them. Generally this could be done writing one or two circles with if condition inside, which would not be so readable. But we could use transducer here:

vector<int> input{1,2,3,4,5,6,7}; auto comp = tr | tfilter([](int x){ return (x % 2 == 0); }) | tmap([](int x){ return x+1; }); auto result = into(vector<int>(), comp, output_rf, input); // 3 5 7 1 2 3 4 vector < int > input { 1 , 2 , 3 , 4 , 5 , 6 , 7 } ; auto comp = tr | tfilter ( [ ] ( int x ) { return ( x % 2 == 0 ) ; } ) | tmap ( [ ] ( int x ) { return x + 1 ; } ) ; auto result = into ( vector < int > ( ) , comp , output_rf , input ) ; // 3 5 7

Second line creates transducer comp (which can be reusable), and last line is performing actual iteration. Output_rf is standard reducing function which just adds new element into provided list. Function into takes four arguments – first is initial state of accumulator, second one is composite transducer, third is initial reducing function and the last one is input range.

We can write same operation as one line:

auto result = input | into_vector << (tr | tfilter([](int x){ return (x % 2 == 0); }) | tmap([](auto x){ return x+1; })); // 3 5 7 1 2 auto result = input | into_vector < < ( tr | tfilter ( [ ] ( int x ) { return ( x % 2 == 0 ) ; } ) | tmap ( [ ] ( auto x ) { return x + 1 ; } ) ) ; // 3 5 7

Here we switched into with into_vector, which assumes that we accumulating items into empty vector. Once again, there is only one cycle inside this chain. As You can see this line is quite readable.

We could have multiple inputs:

// multiple inputs vector<int> input1{1,2,3,4}; vector<int> input2{6,7,8,9}; auto result = into_vector(tr | tmap([](int x, int y){ return x + y; }), input1, input2); // 7 9 11 13 1 2 3 4 5 // multiple inputs vector < int > input1 { 1 , 2 , 3 , 4 } ; vector < int > input2 { 6 , 7 , 8 , 9 } ; auto result = into_vector ( tr | tmap ( [ ] ( int x , int y ) { return x + y ; } ) , input1 , input2 ) ; // 7 9 11 13

Also we could have any range as input:

// any range as source auto result = into_vector(tr | tfilter([](int x){ return (x % 5 == 0);}), ints(100)); // 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 1 2 3 // any range as source auto result = into_vector ( tr | tfilter ( [ ] ( int x ) { return ( x % 5 == 0 ) ; } ) , ints ( 100 ) ) ; // 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95

Any class which has begin()/end() interface could be used here.

Another one-liner to show that accumulator could be just one integer which could be used in place:

LOG << "Count: " << into(0, tr | tfilter(is_even), tr_count, ints(2000)) << NL; // Count: 1000 1 2 LOG < < "Count: " < < into ( 0 , tr | tfilter ( is_even ) , tr_count , ints ( 2000 ) ) < < NL ; // Count: 1000

When you need to stop iterating due to some reason it’s possible to extend our transducers to make early termination. There are several ways to do so – this will be discussed further. Now the most obvious example – limit count of elements inside final accumulator.

Also we don’t have to put anything into result storage – here we just perform some function for each element using nullptr as accumulator.

vector<int>{1,2,3,4,5,6,7} | into << nullptr << (tr | tfilter(is_even) | tenumerate() | tlimit(2)) << tr_each([](int n, int x){ LOG << n << ":" << x << NL; }) ; // 0:2 // 1:4 1 2 3 4 vector < int > { 1 , 2 , 3 , 4 , 5 , 6 , 7 } | into < < nullptr < < ( tr | tfilter ( is_even ) | tenumerate ( ) | tlimit ( 2 ) ) < < tr_each ( [ ] ( int n , int x ) { LOG < < n < < ":" < < x < < NL ; } ) ; // 0:2 // 1:4

As transducers could deal with any sequence of data (like UI events, network events, etc) it would be nice to have ability to resume iteration process.

auto enumerateStrings = (tr | tenumerate() | tmap([](int n, string s){ return s + " " + SS::toString(n); }))(output_rf); auto result = fn_tr_reduce(vector<string>(), enumerateStrings, vector<string>{"a","b","c"}); // a 0 b 1 c 2 fn_tr_reduce_more(result, enumerateStrings, vector<string>{"e","d"}); // a 0 b 1 c 2 e 3 d 4 1 2 3 4 5 6 auto enumerateStrings = ( tr | tenumerate ( ) | tmap ( [ ] ( int n , string s ) { return s + " " + SS : : toString ( n ) ; } ) ) ( output_rf ) ; auto result = fn_tr_reduce ( vector < string > ( ) , enumerateStrings , vector < string > { "a" , "b" , "c" } ) ; // a 0 b 1 c 2 fn_tr_reduce_more ( result , enumerateStrings , vector < string > { "e" , "d" } ) ; // a 0 b 1 c 2 e 3 d 4

As You can see in this example enumeration of strings was continued from right index. Sweet.

If You are not familiar with functional data processing concept at all, such examples might seem not so obvious. But trust me – after on hour playing with this syntax, everything will be perfectly readable.

Enough of examples – let’s proceed to possible implementation details:

IMPLEMENTATION

What do we need to implement transducers? Actually main essence could be written as several lines using C++14!

auto tr_map = [] (auto fn) { return [=] (auto step) { return [=] (auto& out, auto&& ...ins) { return step(out, fn(std::forward<decltype(ins)>(ins)...)); }; }; }; 1 2 3 4 5 6 7 auto tr_map = [ ] ( auto fn ) { return [ = ] ( auto step ) { return [ = ] ( auto & out, auto&& ...ins) { return step(out, fn(std::forward<decltype(ins)>(ins)...)); } ; } ; } ;

Function which returns function which returns another function… looks totally crazy.

But this is not only problem – Microsoft’s compiler produced internal compiler error on this part of code. So I peeked at Atria implementation and rewrote this code, converting each lambda into templated functor! Instead of several lines I got this (as I need cross-platform solution):

namespace fn_detail { template <typename G, typename... Ts> struct tr_transducer { std::tuple<Ts...> params; tr_transducer(Ts ...ts) : params(std::make_tuple(ts...)){} template<typename RF> auto operator() (RF&& step) const { return this->make(std::forward<RF>(step), std::make_index_sequence<sizeof...(Ts)>()); } template<typename RF, std::size_t...indexes_t> auto make(RF step, std::index_sequence<indexes_t...>) const -> typename G::template apply<RF, Ts...> { return { std::forward<RF>(step), std::get<indexes_t>(params)... }; } }; struct tr_map_gen { template <typename ReducingFnT, typename MappingT> struct apply { ReducingFnT step; MappingT mapping; template <typename StateT, typename ...InputTs> bool operator() (StateT& out, InputTs&& ...ins) { return step(out, mapping(std::forward<decltype(ins)>(ins)...)); } }; }; } template <typename T> auto tr_map(T f){ return fn_detail::tr_transducer<fn_detail::tr_map_gen, T>(f); } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 namespace fn_detail { template < typename G , typename . . . Ts > struct tr_transducer { std : : tuple < Ts . . . > params ; tr_transducer ( Ts . . . ts ) : params ( std : : make_tuple ( ts . . . ) ) { } template < typename RF > auto operator ( ) ( RF && step) const { return this->make(std::forward<RF>(step), std::make_index_sequence<sizeof...(Ts)>()); } template < typename RF , std : : size_t . . . indexes_t > auto make ( RF step , std : : index_sequence < indexes_t . . . > ) const - > typename G : : template apply < RF , Ts . . . > { return { std : : forward < RF > ( step ) , std : : get < indexes_t > ( params ) . . . } ; } } ; struct tr_map_gen { template < typename ReducingFnT , typename MappingT > struct apply { ReducingFnT step ; MappingT mapping ; template < typename StateT , typename . . . InputTs > bool operator ( ) ( StateT & out, InputTs&& ...ins) { return step(out, mapping(std::forward<decltype(ins)>(ins)...)); } } ; } ; } template < typename T > auto tr_map ( T f ) { return fn_detail : : tr_transducer < fn_detail : : tr_map_gen , T > ( f ) ; }

If You are using only Clang you can stay with previous implementation and be happy.

This implementation already contains several assumptions. First it contains the way I choose to implement early termination. Each step() returns bool which is false when we need to terminate. Just that simple. And accumulator is passed by reference to step function.

This could be implemented by wrapping result state itself into wrapper struct and this way is described here – video.

Also I want to compose my transducers using functional chain, which was described here. But there is one important difference – when we compose reducing functions, the order of application is reversed. So we need to create reversed functional chain.

/// The inversed chain of functors ... is actualy just a tuple of functors template <typename... FNs> class fn_chain_reversed { private: const std::tuple<FNs...> functions; template <std::size_t I, typename Arg> inline typename std::enable_if<I == sizeof...(FNs) - 1, decltype(std::get<I>(functions)(std::declval<Arg>())) >::type call(Arg arg) const { return std::get<I>(functions)(std::forward<Arg>(arg)); } template <std::size_t N, std::size_t I, typename Arg> struct final_type : final_type<N-1, I+1, decltype(std::get<I>(functions)(std::declval<Arg>())) > {}; template <std::size_t I, typename Arg> struct final_type<0, I, Arg> { using type = decltype(std::get<I>(functions)(std::declval<Arg>())); }; template <std::size_t I, typename Arg> inline typename std::enable_if<I < sizeof...(FNs) - 1, typename final_type<sizeof...(FNs) - 1 - I, I, Arg>::type >::type call(Arg arg) const { return this->call<I+1>(std::get<I>(functions)(std::forward<Arg>(arg))); } static const bool isFunctionalChain = true; public: fn_chain_reversed() : functions(std::tuple<>()) {} fn_chain_reversed(std::tuple<FNs...> functions) : functions(functions) {} // add function into chain (inversed order) template< typename F > inline auto add(const F& f) const -> fn_chain_reversed<F,FNs...> { return fn_chain_reversed<F,FNs...>(std::tuple_cat(std::make_tuple(f), functions)); } // call whole functional chain template <typename Arg> inline auto operator()(Arg arg) const -> decltype(this->call<0,Arg>(arg)) { return call<0>(std::forward<Arg>(arg)); } }; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 /// The inversed chain of functors ... is actualy just a tuple of functors template < typename . . . FNs > class fn_chain_reversed { private : const std : : tuple < FNs . . . > functions ; template < std : : size_t I , typename Arg > inline typename std : : enable_if < I == sizeof . . . ( FNs ) - 1 , decltype ( std : : get < I > ( functions ) ( std : : declval < Arg > ( ) ) ) > : : type call ( Arg arg ) const { return std : : get < I > ( functions ) ( std : : forward < Arg > ( arg ) ) ; } template < std : : size_t N , std : : size_t I , typename Arg > struct final_type : final_type < N - 1 , I + 1 , decltype ( std : : get < I > ( functions ) ( std : : declval < Arg > ( ) ) ) > { } ; template < std : : size_t I , typename Arg > struct final_type < 0 , I , Arg > { using type = decltype ( std : : get < I > ( functions ) ( std : : declval < Arg > ( ) ) ) ; } ; template < std : : size_t I , typename Arg > inline typename std : : enable_if < I < sizeof . . . ( FNs ) - 1 , typename final_type < sizeof . . . ( FNs ) - 1 - I , I , Arg > : : type > : : type call ( Arg arg ) const { return this - > call < I + 1 > ( std : : get < I > ( functions ) ( std : : forward < Arg > ( arg ) ) ) ; } static const bool isFunctionalChain = true ; public : fn_chain_reversed ( ) : functions ( std : : tuple < > ( ) ) { } fn_chain_reversed ( std : : tuple < FNs . . . > functions ) : functions ( functions ) { } // add function into chain (inversed order) template < typename F > inline auto add ( const F & f) const -> fn_chain_reversed<F,FNs...> { return fn_chain_reversed<F,FNs...>(std::tuple_cat(std::make_tuple(f), functions)); } // call whole functional chain template < typename Arg > inline auto operator ( ) ( Arg arg ) const - > decltype ( this - > call < 0 , Arg > ( arg ) ) { return call < 0 > ( std : : forward < Arg > ( arg ) ) ; } } ;

And to support piping into such chain we need to overload operator like this:

template<typename... FNs, typename F> inline auto operator|(fn_chain_reversed<FNs...> && transducer, F&& rf) -> decltype(transducer.add(rf)) { return transducer.add(std::forward<F>(rf)); } 1 2 3 4 5 template < typename . . . FNs , typename F > inline auto operator | ( fn_chain_reversed < FNs . . . > && transducer, F&& rf) -> decltype(transducer.add(rf)) { return transducer.add(std::forward<F>(rf)); }

One more thing – if we have general functional piping overload (more details here), this overload will produce errors during compilation as compiler will not know what to do – just call your chain using provided argument or push new element into chain. To solve this ambiguous situation we just need to add some type traits and SFINAE:

template<typename T, typename... FNs> struct fn_isNotFunctionalChain{ static const bool value = true; }; template<> struct fn_isNotFunctionalChain< fn_chain_reversed<> >{ static const bool value = false; }; //... template<typename T, class F, typename = std::enable_if_t<fn_isNotFunctionalChain<F>::value> > auto operator|(const F& f, T&& param) -> decltype(f(param)) { return f(std::forward<T>(param)); } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 template < typename T , typename . . . FNs > struct fn_isNotFunctionalChain { static const bool value = true ; } ; template < > struct fn_isNotFunctionalChain < fn_chain_reversed < > > { static const bool value = false ; } ; //... template < typename T , class F , typename = std : : enable_if_t < fn_isNotFunctionalChain < F > : : value > > auto operator | ( const F & f, T&& param) -> decltype(f(param)) { return f(std::forward<T>(param)); }

Ok, let’s implement main iteration now:

template <typename RF, typename A, std::size_t ...Indices, typename ...Ranges> auto fn_accum_impl(std::index_sequence<Indices...>, RF&& step, A& out, Ranges... ranges) { auto firsts = std::make_tuple(std::begin(ranges)...); auto lasts = std::make_tuple(std::end(ranges)...); bool ok = true; while ((fn_detail::tuple_all_neq(firsts, lasts)) && (ok)) { ok = step(out, std::forward< decltype(*std::begin(ranges)) >(*std::get<Indices>(firsts))...); fn_detail::noop(++std::get<Indices>(firsts)...); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 template < typename RF , typename A , std : : size_t . . . Indices , typename . . . Ranges > auto fn_accum_impl ( std : : index_sequence < Indices . . . > , RF && step, A& out, Ranges... ranges) { auto firsts = std::make_tuple(std::begin(ranges)...); auto lasts = std : : make_tuple ( std : : end ( ranges ) . . . ) ; bool ok = true ; while ( ( fn_detail : : tuple_all_neq ( firsts , lasts ) ) && (ok)) { ok = step(out, std::forward< decltype(*std::begin(ranges)) >(*std::get<Indices>(firsts))...); fn_detail : : noop ( ++ std : : get < Indices > ( firsts ) . . . ) ; } }

Only complication here is the need to support several input ranges. So we pack iterators into tuple and increment them all as one step. Also we need additional function to compare two tuples to know when to stop (which I grabbed from atria):

template <std::size_t Index, std::size_t Max> struct tuple_all_neq_t { template <typename Tuple1T, typename Tuple2T> bool operator()(Tuple1T&& t1, Tuple2T&& t2) { return std::get<Index>(std::forward<Tuple1T>(t1)) != std::get<Index>(std::forward<Tuple2T>(t2)) && tuple_all_neq_t<Index + 1, Max>{} ( std::forward<Tuple1T>(t1), std::forward<Tuple2T>(t2)); } }; template <std::size_t Max> struct tuple_all_neq_t<Max, Max> { template <typename Tuple1T, typename Tuple2T> bool operator()(Tuple1T&&, Tuple2T&&) { return true; } }; template <typename Tuple1T, typename Tuple2T> bool tuple_all_neq(Tuple1T&& t1, Tuple2T&& t2) { constexpr auto size1 = std::tuple_size< std::decay_t<Tuple1T> >::value; constexpr auto size2 = std::tuple_size< std::decay_t<Tuple2T> >::value; using impl_t = tuple_all_neq_t<0u, (size1 > size2 ? size2 : size1)>; return impl_t{} ( std::forward<Tuple1T>(t1), std::forward<Tuple2T>(t2)); } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 template < std : : size_t Index , std : : size_t Max > struct tuple_all_neq_t { template < typename Tuple1T , typename Tuple2T > bool operator ( ) ( Tuple1T && t1, Tuple2T&& t2) { return std::get<Index>(std::forward<Tuple1T>(t1)) != std::get<Index>(std::forward<Tuple2T>(t2)) && tuple_all_neq_t<Index + 1, Max>{} ( std::forward<Tuple1T>(t1), std::forward<Tuple2T>(t2)); } } ; template < std : : size_t Max > struct tuple_all_neq_t < Max , Max > { template < typename Tuple1T , typename Tuple2T > bool operator ( ) ( Tuple1T &&, Tuple2T&&) { return true; } } ; template < typename Tuple1T , typename Tuple2T > bool tuple_all_neq ( Tuple1T && t1, Tuple2T&& t2) { constexpr auto size1 = std::tuple_size< std::decay_t<Tuple1T> >::value; constexpr auto size2 = std : : tuple_size < std : : decay_t < Tuple2T > > : : value ; using impl_t = tuple_all_neq_t < 0u , ( size1 > size2 ? size2 : size1 ) > ; return impl_t { } ( std : : forward < Tuple1T > ( t1 ) , std : : forward < Tuple2T > ( t2 ) ) ; }

All what’s left is to create several more convenient interfaces to call main iteration cycle. into and into_vector are probably the most obvious ways to use transducers from first examples, but they all just call main iteration function – fn_accum_impl.

template <typename T, typename RF, typename C, typename... Ins> auto fn_tr_transduce(C init, T&& transducer, RF&& reducingFunction, Ins... ins) { C out = init; fn_accum_impl(std::make_index_sequence<sizeof...(Ins)>{}, transducer(reducingFunction), out, (std::forward<Ins>(ins))...); return std::move(out); }; template <typename RF, typename C, typename... Ins> auto fn_into_vector(RF step, C input, Ins... ins) { return fn_tr_transduce(std::vector<std::decay_t<decltype(*std::begin(input))>>(), step, [] (auto& out, auto input) { out.push_back(input); return true; }, std::forward<C>(input), (std::forward<Ins>(ins))...); }; template <typename T, typename C, typename... Ins> auto fn_tr_reduce(C&& init, T&& step, Ins... ins) { C&& out = std::forward<C>(init); fn_accum_impl(std::make_index_sequence<sizeof...(Ins)>{}, std::forward<T>(step), out, (std::forward<Ins>(ins))...); return std::move(out); }; #define tr fn_chain_reversed<>() fn_make_universal(tr_reduce, fn_tr_reduce); fn_make_universal(into, fn_tr_transduce); fn_make_universal(into_vector, fn_into_vector); 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 template < typename T , typename RF , typename C , typename . . . Ins > auto fn_tr_transduce ( C init , T && transducer, RF&& reducingFunction, Ins... ins) { C out = init; fn_accum_impl ( std : : make_index_sequence < sizeof . . . ( Ins ) > { } , transducer ( reducingFunction ) , out , ( std : : forward < Ins > ( ins ) ) . . . ) ; return std : : move ( out ) ; } ; template < typename RF , typename C , typename . . . Ins > auto fn_into_vector ( RF step , C input , Ins . . . ins ) { return fn_tr_transduce ( std : : vector < std : : decay_t < decltype ( * std : : begin ( input ) ) > > ( ) , step , [ ] ( auto & out, auto input) { out.push_back(input); return true ; } , std : : forward < C > ( input ) , ( std : : forward < Ins > ( ins ) ) . . . ) ; } ; template < typename T , typename C , typename . . . Ins > auto fn_tr_reduce ( C && init, T&& step, Ins... ins) { C&& out = std::forward<C>(init); fn_accum_impl ( std : : make_index_sequence < sizeof . . . ( Ins ) > { } , std : : forward < T > ( step ) , out , ( std : : forward < Ins > ( ins ) ) . . . ) ; return std : : move ( out ) ; } ; #define tr fn_chain_reversed<>() fn_make_universal ( tr_reduce , fn_tr_reduce ) ; fn_make_universal ( into , fn_tr_transduce ) ; fn_make_universal ( into_vector , fn_into_vector ) ;

The macro fn_make_universal converts functor into ‘universal’ functor which supports piping and currrying (that topic is described here).

Now we can just add a bit more of sample transducers (filter, enumerate, limit, each):

struct tr_filter_gen { template <typename ReducingFnT, typename FilterT> struct apply { ReducingFnT step; FilterT pred; template <typename StateT, typename ...InputTs> bool operator() (StateT& out, InputTs&& ...ins) { if (pred(std::forward<decltype(ins)>(ins)...)) return step(out, std::forward<decltype(ins)>(ins)...); else return true; } }; }; struct tr_enumerate_gen { template <typename ReducingFnT, typename N> struct apply { ReducingFnT step; int n; template <typename StateT, typename ...InputTs> bool operator() (StateT& out, InputTs&& ...ins) { return step(out, n++, std::forward<decltype(ins)>(ins)...); } }; }; struct tr_limit_gen { template <typename ReducingFnT, typename N1, typename N2> struct apply { ReducingFnT step; int n; int limit; template <typename StateT, typename ...InputTs> bool operator() (StateT& out, InputTs&& ...ins) { return (n++ > limit) ? false : step(out, std::forward<decltype(ins)>(ins)...); } }; }; template <typename T> struct tr_each_impl{ T step; template <typename StateT, typename ...InputTs> bool operator() (StateT& out, InputTs&& ...ins) { step(ins...); return true; } }; template <typename T> auto tr_filter(T pred) { return fn_detail::tr_transducer<fn_detail::tr_filter_gen, T>(pred); } static auto tr_enumerate(int n = 0) { return fn_detail::tr_transducer<fn_detail::tr_enumerate_gen, int>(n); } static auto tr_limit(int limit){ return fn_detail::tr_transducer<fn_detail::tr_limit_gen, int, int>(1, limit); } template <typename T> auto tr_each(T step){ return fn_detail::tr_each_impl<T>{step}; }; const auto tr_count = [](auto&s, auto ...ins) { s++; return true; }; fn_make_universal(tmap, tr_map); fn_make_universal(tfilter, tr_filter); fn_make_universal(teach, tr_each); fn_make_universal(tlimit, tr_limit); fn_make_universal(tenumerate, tr_enumerate); 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 struct tr_filter_gen { template < typename ReducingFnT , typename FilterT > struct apply { ReducingFnT step ; FilterT pred ; template < typename StateT , typename . . . InputTs > bool operator ( ) ( StateT & out, InputTs&& ...ins) { if (pred(std::forward<decltype(ins)>(ins)...)) return step(out, std::forward<decltype(ins)>(ins)...); else return true ; } } ; } ; struct tr_enumerate_gen { template < typename ReducingFnT , typename N > struct apply { ReducingFnT step ; int n ; template < typename StateT , typename . . . InputTs > bool operator ( ) ( StateT & out, InputTs&& ...ins) { return step(out, n++, std::forward<decltype(ins)>(ins)...); } } ; } ; struct tr_limit_gen { template < typename ReducingFnT , typename N1 , typename N2 > struct apply { ReducingFnT step ; int n ; int limit ; template < typename StateT , typename . . . InputTs > bool operator ( ) ( StateT & out, InputTs&& ...ins) { return (n++ > limit) ? false : step(out, std::forward<decltype(ins)>(ins)...); } } ; } ; template < typename T > struct tr_each_impl { T step ; template < typename StateT , typename . . . InputTs > bool operator ( ) ( StateT & out, InputTs&& ...ins) { step(ins...); return true ; } } ; template < typename T > auto tr_filter ( T pred ) { return fn_detail : : tr_transducer < fn_detail : : tr_filter_gen , T > ( pred ) ; } static auto tr_enumerate ( int n = 0 ) { return fn_detail : : tr_transducer < fn_detail : : tr_enumerate_gen , int > ( n ) ; } static auto tr_limit ( int limit ) { return fn_detail : : tr_transducer < fn_detail : : tr_limit_gen , int , int > ( 1 , limit ) ; } template < typename T > auto tr_each ( T step ) { return fn_detail : : tr_each_impl < T > { step } ; } ; const auto tr_count = [ ] ( auto &s, auto ...ins) { s++; return true ; } ; fn_make_universal ( tmap , tr_map ) ; fn_make_universal ( tfilter , tr_filter ) ; fn_make_universal ( teach , tr_each ) ; fn_make_universal ( tlimit , tr_limit ) ; fn_make_universal ( tenumerate , tr_enumerate ) ;

You could implement any custom transducer using this implementation as sketch.

CONCLUSION

Transducers combine long functional data processing chains into one iteration cycle. This is crucial when you need to process large amount of data or have sequence of realtime events.

Transducers are quite readable and simple to use (when you are familiar with primitive functional data processing chains). You need to write core of transducers only once (or You can grab one of libraries which already have suitable implementation).

I hope the benefits are obvious from samples. As one of ways to simplicity transducers were included into my presentation at MeetingCPP 2015 about fighting complexity in modern C++ (slides).