Maybe some times you have overused the Array methods map, reduce, filter, find, etc. and that could make out applications to use more memory of what they should be using.

Let’s see an example:

const myArray = [1,2,3,4,5,6,7,8,9,10] myArray

.map(x => x * x)

.filter(x => x % 3 === 0)

Sure you have done something similar to this several times, and what is the “problem” here, we are creating new arrays every time we chain and array method. That means the .map creates another array with the x² in every index ([ 1, 4, 9, 16, 25, 36, 49, 64, 81, 100 ]) and after that create another filtered one [ 9, 36, 81 ] in this case you maybe don’t have any problem of performance of memory use, but what if myArray is a very large one? you are going to create another big one again, and again until you have your wanted result.

In those cases, the Lazy evaluation with iterables could be useful. But first, how can we create an iterator and how works.

What is an iterator?

An iterator is an Object that defines a sequence and maybe return a value when the sequence is finished, those sequences could be Finite or Infinite.

The iterator Object will need a property function called Symbol.iterator which should return another object with a next() method. That next method will be the one that returns every value of the sequence and if the sequence is finished or not using an Object with the property value and done as

const JustOneIterator = {

[Symbol.iterator]: function () {

return {

next() {

return { done: false, value: 1 }

}

}

}

} const iterator = JustOneIterator[Symbol.iterator]()

iterator.next() // { done: false, value: 1 }

iterator.next() // { done: false, value: 1 }

iterator.next() // { done: false, value: 1 }

iterator.next() // { done: false, value: 1 }

// this will never return done: true

In this example, you can see and Infinite sequence of ones.

Let’s try a Finite sequence with a range of number

const rangeIterator = (from, to = Infinity, step = 1) => ({

[Symbol.iterator]: function () {

let done = false

let value = 0

return {

next () {

value = from

done = from > to

from = !done ? from + step : value

return { done: done, value: value }

}

}

}

}) const iterator = rangeIterator(0, 4)[Symbol.iterator]()

iterator.next() // { done: false, value: 0 }

iterator.next() // { done: false, value: 1 }

iterator.next() // { done: false, value: 2 }

iterator.next() // { done: false, value: 3 }

iterator.next() // { done: false, value: 4 }

iterator.next() // { done: true, value: 5} var range = rangeIterator(0, 4) const arr = [...range] // [0, 1, 2, 3, 4]

Because this is a Finite iterator we can spread into an Array or use the Array.from method to create the array from the iterator.

Array.from(range) // [0, 1, 2, 3, 4]

Now how can we create our own iterables objects in order to map, reduce, filter, etc.

With this Utils now we can do something like:

With a finite array

const cubicPar = compose(

map.bind(null, x => x * x * x),

filter.bind(null, x => x %2 === 0)

) const fiveElements = take(10, cubicPar([1,2,3,4,5,6,7,8,9,10])) getValue(fiveElements)

// [ 8, 64, 216, 512, 1000 ]

With an infinite Sequence

const Numbers = generate(function () {

let n = 0

return {

next: () => ({ done: false, value: n++ })

}

}) const cubirParRest = compose(

map.bind(null, x => x * x * x),

filter.bind(null, x => x %2 === 0),

take.bind(null, 10),

rest

) Array.from(cubirParRest(Numbers))

// [ 8, 64, 216, 512, 1000, 1728, 2744, 4096, 5832 ]

Now that we manage to create some Iterators from those Utils, we don’t want to always compose our functions and instead we want to chain methods. We can add a dependency on the generate method

const LazyIterable = {

value: function () {

return Array.from(this)

},

map: function (fn) {

return map(fn, this)

},

reduce: function (fn, seed) {

return reduce(fn, seed, this)

},

filter: function (fn) {

return filter(fn, this)

},

find: function (fn) {

return filter(fn, this).first()

},

first: function () {

return first(this)

},

rest: function () {

return rest(this)

},

until: function (fn) {

return until(fn, this)

},

take: function (numberToTake) {

return take(numberToTake, this)

}

} const Numbers = generate(function () {

let n = 0

return {

next: () => ({ done: false, value: n++ })

}

}, LazyIterable) Numbers

.map(x => x * x * x)

.filter(x => x %2 === 0)

.take(10)

.rest()

.value() // [ 8, 64, 216, 512, 1000, 1728, 2744, 4096, 5832 ]

We could do a Fibonacci sequence

const Fibonacci = generate(function () {

let n1 = 0

let n2 = 1

let value

return {

next: () => {

[value, n1, n2] = [n1, n2, n1 + n2]

return { value }

}

}

}, LazyIterable) Fibonacci

.take(10)

.value()

// [ 0, 1, 1, 2, 3, 5, 8, 13, 21, 34 ] Fibonacci

.map(x => x * 2)

.filter(x => x % 4 === 0)

.take(10)

.value()

// returns 0, 4, 16, 68, 288, 1220, 5168, 21892, 92736, 392836 ]

We can even create and LazyArray Object that extends from Array

const LazyIterableDescriptor = Object.keys(LazyIterable).reduce((acc, key) => {

acc[key] = { value: LazyIterable[key] }

return acc

}, {}) function LazyArray () {

this.push.apply(this, arguments)

} LazyArray.prototype = Object.create(Array.prototype, Object.assign({

constructor: {

value: LazyArray

}

}, LazyIterableDescriptor)) LazyArray.from = function (iterable) {

const lazy = new LazyArray() for (let element of iterable) {

lazy.push(element)

}

return lazy

} const lazy = new LazyArray(1,2,3,4,5)

// or LazyArray.from([1,2,3,4,5]) lazy.value() // [1,2,3,4,5]

lazy.push(6) // 6

lazy.value() // [1,2,3,4,5,6] lazy

.map(x => x * x * x)

.value()

// [ 1, 8, 27, 64, 125, 216 ]

We could create another type of Object as Stash, LinkedList and more, and all of them could be use Lazy Iteration.

I hope this information could be useful for some of you, and you could start using Laziness in your applications.

About performance

If we run this code (in my computer)

function arrayOfNumbers (n) {

const array = []

let i = 0

while (i < n) {

array.push(i)

i++

}

return array

} const array = arrayOfNumbers(100000000) let i = 0

while (i < 1000) {

i++

array

.map(x => x * x)

.filter(x => x % 2 === 0)

.reduce((acc, x) => acc + x)

}

I got this

$ time node __tests__/array.js

rss 67.5 MB

heapTotal 54.04 MB

heapUsed 32.11 MB

external 0.01 MB real 0m57.297s

user 0m54.974s

sys 0m1.851s

But if I run this

const { LazyArray } = require('../lib/perezoso') function arrayOfNumbers (n) {

const array = new LazyArray()

let i = 0

while (i < n) {

array.push(i)

i++

}

return array

} const lazyArray = arrayOfNumbers(100000000) let i = 0

while (i < 1000) {

i++

lazyArray

.map(x => x * x)

.filter(x => x % 2 === 0)

.reduce((acc, x) => acc + x)

}

Finished in

$ time node __tests__/lazy2.js

rss 33.38 MB

heapTotal 19.21 MB

heapUsed 9.36 MB

external 0.01 MB real 0m19.202s

user 0m18.774s

sys 0m0.692s

But changing everything for iterators

const { Lazy } = require('../lib/perezoso') const Numbers = Lazy.generate(function () {

let n = 0

return {

next () {

return { value: n++ }

}

}

}) const lazyArray = Numbers.take(100000000) let i = 0

while (i < 1000) {

i++

lazyArray

.map(x => x * x)

.filter(x => x % 2 === 0)

.reduce((acc, x) => acc + x)

}

I got this

$ time node __tests__/lazy.js

rss 32.72 MB

heapTotal 18.23 MB

heapUsed 9.16 MB

external 0.01 MB real 0m12.556s

user 0m12.218s

sys 0m0.728s

Another important thing is if I used a bigger arrayOfNumbers(100000000) by *100 (two more 0) the Array version and the LazyArray version fails with a memory error but finished if I use the only iterator example (3rdone)

May the force be with you

Update 09 May 2019

Using the functional approach of perezoso.js, I got a very similar result of memory consumption to the one using the complete Lazy object but a little bit faster.

$ time node scripts/funtional.js

rss 32.92 MB

heapTotal 18.71 MB

heapUsed 10.12 MB

external 0.01 MB real 0m7.166s

user 0m6.907s

sys 0m0.557s $ time node scripts/funtional2.js

rss 32.45 MB

heapTotal 17.23 MB

heapUsed 7.23 MB

external 0.01 MB real 0m9.077s

user 0m8.793s

sys 0m0.679s

Here are all the https://github.com/highercomve/perezoso-js/tree/master/scripts