Easily one of the weirder new constructs within the new ES6 spec, the ES6 generator answers a perplexing question: What would happen if you stopped a loop in the middle of its execution, went off and did something else for a while, then came back?

It turns out that you can actually do quite a bit.

A generator, at first blush, looks very much like a function with a loop in it:

function * iter() { var i = 0; while (i != 3){ yield i; i++ } }

There are two differences. The first is the appearance of the * between function and the name (function * iter), with the second being the presence of the yield keyword followed by an expression (yield i;). The generator is itself actually short for iterator generator – by calling it as a function the result is a second function that contains an internal iterator that can then be invoked, one item after the previous, using the next() call:

var count = iter(); console.log(count().next()); // 0 console.log(count().next()); // 1 console.log(count().next()); // 2 console.log(count().next()); // null console.log(count().next()); // null

The yield statement in the original iter() function acts like a return statement in a normal function, returning the current state of an expression before processing that expression in some manner.

So what, exactly, is an iterator? In Javascript, as with most languages, there has always been a need to do batch processing of a sequence of items. The oldest solution for this was the hoary C standby:

for (i=0; someTestOn(i); i++){// doSomethingWith(i)} This worked pretty well with arrays: var arr = [1, 2, 3, 4, 5]; for (i=0; i != arr.length;i++){ var item = arr[i]; doSomethingWith(item); }

It got much more problematic with objects, necessitating the creation of the in keyword:

for (var key in obj){ var item = arr[key]; doSomethingWith(item); }

Those got to be troublesome when the object in question was an array, as suddenly the array was revealed as an object with its index as key values. DOM interactions became even more nettlesome:

for (var index =0; index != parentNode.childNodes.length;index++){ var node=parentNode.childNodes.item(index); doSomethingWithNode(node); }

This doesn’t even begin to cover various APIs for getting at data from a database, or accessing the next item from a news feed or working with the hideous monstrosities of the XPath APIs.

Eventually, the solution became obvious – rather than trying to accommodate the hundreds of different ways of iterating over a sequence after twenty five years, put the means to do so in the developer’s hands. Thus the generator was born.

A generator will take a function which can then be updated with new state information, returns a result, and moves to the next state. If the state becomes null then the iteration stops. A little more complex iterator showcases this nicely:

function * range(a,b){ var i = a; while(i != b){ yield i; i++ } }; var seq = range(3,8);

Notice here the slightly more traditional functional form, and the fact that the generator now has parameters. The index variable is first set to the value in a, the yield operator returns the index variable, then the index is incremented.

To iterate over this, you can use the while() syntax:

while (n = seq.next().value){ console.log(n) }

This will generate the sequence 3 to 7 (the upper bound is exclusive). However, even that syntax can be simplified:

for (obj of range(3,8)){ console.log(obj) } or even: Array.from( range(3,8) ).forEach( (obj) => console.log(obj))

where the Array.from() function takes the iterator and converts into an array, before using arrow notation to map it to an operation.

Check out this Codepen example if you want to see everything this post covers in action:

See the Pen Iterators by Kurt Cagle (@kurt_cagle) on CodePen.

Generalizing Iterators

One of the key takeaways here is that not only has the index variable disappeared in all of these approaches, but the whole implementation of the iterator has been nicely encapsulated. The users of our iterators do not need to know what happens on the inside. This means that you can do quite a bit with such iterators.

For instance, suppose that you had an array of values that you wanted to iterate through. The generator for this is just a minor modification of the range() iteration discussed above:

function * arrIterate(arr){ var i = 0; while(i != arr.length){ yield arr[i]; i++ } }; for (obj of arrIterate(["red","green","blue"])){ console.log(obj) } //"red" //"green" //"blue"

As it turns out, this is in fact the iterator that lets you walk through an array in the first place. However, suppose that when the last item was reached, rather than terminating, you wanted to repeat this until a specific count was reached, in effect creating a controlled infinite loop:

function * infinite(arr,count){ var i = 0; var j = 0; while(true){ yield arr[i]; i = (i+1 == arr.length)?0:i+1; j++; if (j == count){break} } } for (obj of infinite(arr,10){ console.log(obj); } //"red" //"green" //"blue" //"red" //...five more times //"red"

Here, the iterator is defined with two indexes, one to control the array index, the second the number of times the iterator is invoked. Critically, the break keyword serves to automatically terminate the iterator loop, and by extension, the iteration itself.

Note that this can still be simplified and generalized to handle any break condition.

function * infinite(arr,loopCond){ let i = j = 0; while(true){ j = i % arr.length; yield arr[j]; i++; if (!loopCond(i,arr)){break} } } for (obj of infinite(arr,function(index,arr){return index < 10;})){ console.log(obj); }

This is a powerful construct. It is an infinite loop, until the loop condition function loopCond() becomes false. Moreover, because the iterator loop returns control to the Javascript thread, this can be used to refresh the page, perform an ajax call, check for events or similar type of activity that won’t block UI control.

Iterators as Data Servers

Iterators can also be used as forward running data servers.

For instance, let’s say that you had a comma separated value file of the form:

var data = `name, age, department, id Jane, 25, information technology, jd125 Mark, 37, accounting, ml399 Harry, 51, marketing, hh356`

(for convenience, it’ll be treated here as a string)

The following iterator (* records) will take the CSV file and convert it into an iterated set of map objects, using split(“

”) to split on line boundaries and split(“,”) to split on comma field boundaries:

function * records(dataStr){ let lines = dataStr.split("

"); let headers = lines.shift().split(","); let fieldCount = headers.length; let i = 0; while(i != lines.length){ let fields = lines[i].split(","); let map = new Map(); for (j=0;j!=fieldCount;j++){ map.set(headers[j],fields[j]) } yield map; i++; } }

This can then be used to generate a report using template literals:

var report = []; function itemReport(record){ return `${record.get('name')}, who is ${record.get('age') }, works in ${record.get('department')}.`}; for (record of records(data)){ report.push(itemReport(record)) } console.log(report.join("

")) // Jane, who is 25, works in information technology. // Mark, who is 37, works in accounting. // Harry, who is 51, works in marketing.

Summary

One of the most powerful aspects of this approach is that you can use generators to encapsulate where the data is coming from – XML, JSON, relational recordsets from SQL queries, SPARQL, etc., with the iterators returning arrays of maps as a default conversion. It makes writing such code not only easier but also more generic – if you don’t have to worry about the initial form of your data, then templates and transformers can be written that can handle a far broader set of data. Given Javascript’s increasing role in the data management space, this can only be a good thing

Generators and iterators provide native level capabilities for iterating through any sequence of items, and provide a framework to unify one of the most frequently performed operations in programming. This should make them an invaluable addition to your developer’s toolbox.

For working examples similar to those covered here, please check out my CodePen “Iterators“.