In the real world, at some point, we all have to deal with data that is fetched from a backend service, page by page. It's called paging. Everybody does this! Nothing to be ashamed of ;-) Here is one of the ways in which a Swift pattern (well, not specific to Swift, but visible in Swift) can be used to build a neat solution that allows me to fetch data in chunks and display it in the application.

This approach involves Generators. A generator is a state machine that produces new data continuously. So instead of getting all the data at once, I can ask for chunks of data, calling the generator over and over to generate new data. For example, in Swift GeneratorType is a data source for SequenceType .

It's a very simple idea with a neat interface, yet it's powerful. It's all about asking for more data by calling next() - it advances to the next element and returns it, or nil if no next element exists.

generator.next()

GeneratorType protocol is defined as follows:

protocol GeneratorType { typealias Element mutating func next() -> Element? }

... this is good for a sequence, but it's not good enough for asynchronous operations. That's why I created a similar Generator, but this one is ready to handle asynchronously generated data (like from network requests).

I'll demonstrate how to build a simple generator, which in conjunction with specialized functions can simply do the job. The job is: fetch data in chunks (pages).

Let's start.

The Generator

At first I defined AsyncGeneratorType protocol

protocol AsyncGeneratorType { typealias Element typealias Fetch mutating func next(fetchNextBatch: Fetch) }

Element describes the type of data to generate

describes the type of data to generate Fetch is a closure type, defines a type for data fetching function

is a closure type, defines a type for data fetching function next(fetchNextBatch: Fetch) is the function to generate the next page of data

Now, let's build a generator that conforms to the AsyncGeneratorType protocol we've just defined, where:

Element is an array of generic type T .

is an array of generic type . Fetch is a closure with offset , limit and completion parameters. Completion is called after data is fetched successfully by the function - fetching may be an asynchronous operation.

typealias Element = Array<T> typealias Fetch = (offset: Int, limit: Int, completion: (result: Element) -> Void) -> Void

Keep the current offset and page size limit:

var offset:Int let limit: Int init(offset: Int = 0, limit: Int = 25) { self.offset = offset self.limit = limit }

Finally implement a next() function, the core of the paging:

func next(fetchNextBatch: Fetch) { fetchNextBatch(offset: self.offset, limit: self.limit) { (items) in self.offset += items.count } }

The next() function gets a function (closure) as a parameter. That function will be called every time a new page is fetched from the underlying storage. Because fetching may be an asynchronous operation, after data is fetched, the completion handler has to be called with the resulting items. At this point the generator will update internal state and prepare to serve the next page.

Here's the code of the generator:

class PagingGenerator<T>: AsyncGeneratorType { typealias Element = Array<T> typealias Fetch = (offset: Int, limit: Int, completion: (result: Element) -> Void) -> Void var offset:Int let limit: Int init(startOffset: Int = 0, limit: Int = 25) { self.offset = startOffset self.limit = limit } func next(fetchNextBatch: Fetch) { fetchNextBatch(offset: offset, limit: limit) { [unowned self] (items) in self.offset += items.count } } }

Generate pages of data

What I love about generators is their simple interface. Once defined - it just works. Once I instantiate a generator:

var paging = PagingGenerator<Contact>(startOffset: 0, limit: 25)

I can simply ask for the next page every time I need it:

paging.next(...)

In my case (paging) I need a function that defines actual data fetching. The example function fetchNextBatch() below:

Downloads data from the website Updates a local list of results Returns the items back to the completion closure

private func fetchNextBatch(offset: Int, limit: Int, completion: (Array<Contact>) -> Void) -> Void { if let remotelyFetched = downloadGithubUsers(offset) { self.contacts += remotelyFetched completion(remotelyFetched) } }

This is a specialised function that is responsible only for getting the data and returning the result. From now on it's all I have to implement to make the paging work with data from different sources.

Every time I ask for the next page, the generator will fetch it, preprocess it and add the result to my local list. Requesting next page looks like this now:

paging.next(fetchNextBatch)

Specialised functions

Actually, it's not convenient to have fetchNextBatch() responsible for fetching AND updating the UI. Let me change the interface a little to separate these two things into specialised functions. First, update the protocol with an additional parameter onFinish , called after new data is fetched:

func next(fetchNextBatch: Fetch, onFinish: ((Element) -> Void)? = nil) { fetchNextBatch(offset: self.offset, limit: self.limit) { (items) in onFinish?(items) self.offset += items.count } }

Then I can use it to update my UI:

private func updateDataSource(items: Array<Contact>) { self.contacts += items }

by calling it like this:

paging.next(fetchNextBatch, onFinish: updateDataSource)

Result

Here's a final version of PagingGenerator :

protocol AsyncGeneratorType { typealias Element typealias Fetch mutating func next(fetchNextBatch: Fetch, onFinish: ((Element) -> Void)?) } /// Generator is the class because struct is captured in asynchronous operations so offset won't update. class PagingGenerator<T>: AsyncGeneratorType { typealias Element = Array<T> typealias Fetch = (offset: Int, limit: Int, completion: (result: Element) -> Void) -> Void var offset:Int let limit: Int init(startOffset: Int = 0, limit: Int = 25) { self.offset = startOffset self.limit = limit } func next(fetchNextBatch: Fetch, onFinish: ((Element) -> Void)? = nil) { fetchNextBatch(offset: offset, limit: limit) { [unowned self] (items) in onFinish?(items) self.offset += items.count } } }

It's not really important which version of Swift this is. What I want to show is the way in which I can deal with data that is downloaded in chunks (pages), in a way that's appropriate for Swift structures.

I can use it to incrementally update a UITableView data source, like this:

class ViewController: UIViewController { @IBOutlet var tableView: UITableView! private var paging = PagingGenerator<Contact>(startOffset: 0, limit: 25) private var contacts = [Contact]() { didSet { tableView.reloadData() } } override func viewDidLoad() { super.viewDidLoad() paging.next(fetchNextBatch, onFinish: updateDataSource) // first page } } //MARK: Paging extension ViewController { private func fetchNextBatch(offset: Int, limit: Int, completion: (Array<Contact>) -> Void) -> Void { if let remotelyFetched = downloadGithubUsersPage(offset) { completion(remotelyFetched) } } private func updateDataSource(items: Array<Contact>) { self.contacts += items } } //MARK: UITableViewDataSource extension ViewController: UITableViewDataSource { func tableView(tableView: UITableView, numberOfRowsInSection section: Int) -> Int { return contacts.count } func tableView(tableView: UITableView, cellForRowAtIndexPath indexPath: NSIndexPath) -> UITableViewCell { let cell = UITableViewCell(style: UITableViewCellStyle.Default, reuseIdentifier: nil) let contact = contacts[indexPath.row] cell.textLabel?.text = "\(contact.firstName) \(contact.lastName)" return cell } } //MARK: UITableViewDelegate extension ViewController: UITableViewDelegate { func tableView(tableView: UITableView, willDisplayCell cell: UITableViewCell, forRowAtIndexPath indexPath: NSIndexPath) { if indexPath.row == tableView.dataSource!.tableView(tableView, numberOfRowsInSection: indexPath.section) - 1 { paging.next(fetchNextBatch, onFinish: updateDataSource) } } }

Recap

Generators are nice. Allows easily separate and specialise functions to do one thing right.

I have separated and specialised:

Function to fetch data Function to update UI after new data arrive State machine to get next page

Here is a repository with the source code: GeneratedResults-UITableView

PS. Cover photo: https://www.flickr.com/photos/horiavarlan/4332381194