Welcome back to my series on coroutines. Today we will have a look at the only two higher level functions we get from Kotlin directly: buildIterator and buildSequence . As one can be implemented calling the other (which is actually done in the original implementations) we will only look into buildIterator .

buildSequence is really just:

public fun <T> buildSequence(builderAction: suspend SequenceBuilder<T>.() -> Unit): Sequence<T> = Sequence { buildIterator(builderAction) }

Here we also see, that coroutines can get a receiver, which is pretty important for our two functions.

Usage

First, let’s have a look at how these functions are used.

import kotlin.coroutines.experimental.buildIterator import kotlin.coroutines.experimental.buildSequence //sampleStart fun main(args: Array<String>) { [mark]buildIterator[/mark] { println("yielding 23") yield(23) println("yielding 1 to 6") yieldAll(1..6) println("yielding 42") yield(42) println("done") }.forEach(::println) } //sampleEnd

As we can see our receiver provides us with two more functions: yield and yieldAll . These suspend our coroutine until the next value from the sequence/iterator is queried. So all in all we get a lazily generated sequence/iterator. These two are actually pretty similar to generators in Python (which uses a yield keyword).

Here is another practical example with a potentially infinite generator:

import kotlin.coroutines.experimental.buildIterator import kotlin.coroutines.experimental.buildSequence //sampleStart fun fibonacci(): Sequence<Int> { var a = 0 var b = 1 return buildSequence { while (true) { a = b.also { b = a + b } yield(a) } } } fun main(args: Array<String>) { // ignore the first 5 values // and print the next 10 for (n in fibonacci().drop(5).take(10)) { println(n) } } //sampleEnd

Restrictions

Sadly yield and yieldAll are the only suspending functions we can call in our coroutine scope. This has two reasons:

Functions like delay shown in the previous article require the coroutine context to provide certain features.

shown in the previous article require the coroutine context to provide certain features. SequenceBuilder (our receiver) is annotated with @RestrictsSuspension , which prevents us from calling any suspending functions other than the ones it provides.

The second one sounds like an arbitrary restriction, but is rather useful considering the first one. For example calling delay in a coroutine context which does not support it, would result in errors at quite confusing locations. Therefore just telling the developer, that the function cannot be called in this scope is way more helpful.

Actually we can also define custom suspending functions that can be called. These need to be extension functions to SequenceBuilder though which makes them inherit the restriction. Therefore they also may only call yield , yieldAll and other extension functions of SequenceBuilder .

Implementation

From the “Usage” section we can take how our API needs to look and behave. So let’s start with the sequence builder:

@RestrictsSuspension abstract class SequenceBuilder<T> { abstract suspend fun yield(value: T) // this is a simplified version // the original has an extra optimization for yielding multiple values at once // to lower the number of suspensions suspend fun yieldAll(values: Iterator<T>) { for (value in values) { yield(value) } } suspend fun yieldAll(values: Iterable<T>) = yieldAll(values.iterator()) }

As we want the values in our built iterator to be lazily generated we will need some kind of state machine:

So the flow will be like this: Our iterator starts out “Not Ready”. When we call hasNext the first time, the coroutine would be resumed. If we then called yield , the state would switch to “Ready”, we would store the yielded value, and true would be returned by hasNext . A following call to next would then return the stored value and switch the state back to “Not Ready”. If we called hasNext and the coroutine reached its end the state would be switched to “Done” and hasNext would return false . Calling hasNext on a “Done” iterator will then always return false , while calling next will throw a NoSuchElementException .

So here is an implementation of the aforementioned concept:

// this thing implements both sides: // * the interface the coroutine sees // * the interface the outside caller sees // but either side can only see a part of it class SequenceBuilderIterator<T> : Iterator<T>, Continuation<Unit>, SequenceBuilder<T>() { // we also implement Continuation<Unit>, // so we can capture the completion and the failure of our coroutine enum class State { NotReady, Ready, Done, Failed // this one was not mentioned above. Ofcourse our coroutine might throw an exception... } // initial state private var state: State = State.NotReady // possible results private var exception: Throwable? = null private var value: T? = null // the continuation created from our coroutine // we will see the reason for this being public later var nextStep: Continuation<Unit>? = null // we suspend the coroutine // thanks to @RestrictsSuspension this can only be done here, in SequenceBuilderIterator override suspend fun yield(value: T) = suspendCoroutine<Unit> { continuation -> // we save where we need to continue the coroutine // set the state // and store the value nextStep = continuation state = State.Ready this.value = value } // no fancy stuff supported override val context: CoroutineContext get() = EmptyCoroutineContext // this is called when the coroutine scope ends override fun resume(value: Unit) { state = State.Done } // this is called when an uncaught exception leaves the coroutine scope override fun resumeWithException(exception: Throwable) { state = State.Failed this.exception = exception } // only NotReady leads to the evaluation of the next item override fun hasNext(): Boolean { // this loop runs twice at most, as after resume the state is definitely one of the top 3 cases while (true) { when (state) { State.Ready -> return true State.Done -> return false State.Failed -> throw RuntimeException(exception) State.NotReady -> nextStep!!.resume(Unit) } } } override fun next(): T { // next might be called without hasNext having been called beforehand if (hasNext()) { state = State.NotReady return value!! } else { throw NoSuchElementException() } } }

As you can see it was possible to transfer the state machine model to code quite easily. The following implementation of buildIterator is pretty trivial and mostly equivalent to the original:

fun <T> buildIterator(builder: suspend SequenceBuilder<T>.() -> Unit): Iterator<T> { val iterator = SequenceBuilderIterator<T>() // nextStep is a public property because of the circular dependency iterator.nextStep = builder.createCoroutine(iterator, iterator) return iterator }

Working example

Let’s return to the examples from the start. There, the original functions from the Kotlin standard library are imported. Now we’ll see our own implementation in action:

import kotlin.coroutines.experimental.Continuation import kotlin.coroutines.experimental.CoroutineContext import kotlin.coroutines.experimental.EmptyCoroutineContext import kotlin.coroutines.experimental.RestrictsSuspension import kotlin.coroutines.experimental.createCoroutine import kotlin.coroutines.experimental.suspendCoroutine @RestrictsSuspension abstract class SequenceBuilder<T> { abstract suspend fun yield(value: T) suspend fun yieldAll(values: Iterator<T>) { for (value in values) { yield(value) } } suspend fun yieldAll(values: Iterable<T>) = yieldAll(values.iterator()) } class SequenceBuilderIterator<T> : Iterator<T>, Continuation<Unit>, SequenceBuilder<T>() { enum class State { NotReady, Ready, Done, Failed } private var state: State = State.NotReady private var exception: Throwable? = null private var value: T? = null var nextStep: Continuation<Unit>? = null override suspend fun yield(value: T) = suspendCoroutine<Unit> { continuation -> nextStep = continuation state = State.Ready this.value = value } override val context: CoroutineContext get() = EmptyCoroutineContext override fun resume(value: Unit) { state = State.Done } override fun resumeWithException(exception: Throwable) { state = State.Failed this.exception = exception } override fun hasNext(): Boolean { while (true) { when (state) { State.Ready -> return true State.Done -> return false State.Failed -> throw RuntimeException(exception) State.NotReady -> nextStep!!.resume(Unit) } } } override fun next(): T { if (hasNext()) { state = State.NotReady return value!! } else { throw NoSuchElementException() } } } fun <T> buildIterator(builder: suspend SequenceBuilder<T>.() -> Unit): Iterator<T> { val iterator = SequenceBuilderIterator<T>() iterator.nextStep = builder.createCoroutine(iterator, iterator) return iterator } fun <T> buildSequence(builderAction: suspend SequenceBuilder<T>.() -> Unit): Sequence<T> = Sequence { buildIterator(builderAction) } //sampleStart fun fibonacci(): Sequence<Int> { var a = 0 var b = 1 return buildSequence { while (true) { a = b.also { b = a + b } yield(a) } } } fun main(args: Array<String>) { buildIterator<Int> { println("yielding 23") yield(23) println("yielding 1 to 6") yieldAll(1..6) println("yielding 42") yield(42) println("done") }.forEach(::println) for (n in fibonacci().drop(5).take(10)) { println(n) } } //sampleEnd

See also