Unlike the Black Box test we previously looked at, where the inner workings of the subject under test are hidden, in a White Box test, we are able to peer into the test using “spies” (I would read this blog if you’re not clear on “Spies” or “Mocks”). In a White Box test we don’t actually care about the input or the output of a particular test subject. What we care about is how some state changed based on an action that was taken.

So the big difference here, compared to previous tests we’ve looked at, is that the test is indirect because we’re actually testing the “side effects” of taking a particular action. Side Effects can be a number of different things, but basically it is anything that happens as a “side effect” of calling a particular function (e.g. logging, posted notifications, async tasks, or changes to state variables).

Abstract Representation Of A White Box Test

Before we get to the open source example pulled straight from a production app, this time, I’d like to start with an extremely simplified test case.

Simplified Test Case

Let’s imagine we’re working on a feature for an application that needs to fetch results from a search engine. We might have a class that defines search actions and one of those actions might be a function that takes a search string and makes a search request to the default search engine. It might look something like this:

class SearchActions { var searchEngine: SearchEngine = .duckDuckGo ... func updateWith(searchTerm: String, requestManager: RequestManageable) { let result = requestManager.performSearch(searchTerm: searchTerm, searchEngine: searchEngine) switch result { case .success(let searchResults): updateUIWithSearchResults(searchResults: searchResults) break case .failure(let error): handleError(error: error) } ... }

Take another look at the requestManager: RequestManageable parameter above. RequestManageable is a protocol that defines the interface to an object that handles the details of the network request to the search engine. Here is the RequestManageable protocol:

protocol RequestManageable { func performSearch(searchTerm: String, searchEngine: SearchEngine) -> Result<SearchResults, Error> }

So now we come to the main question: “How do we test updateWith(:_) ?”

The function doesn’t return anything so a Black Box Test is out of the question. We need to find out what’s happening inside the function as it’s executed. So, updateWith(:_) calls a function on requestManager ( performSearch() ) that we’ve already established will make a network request to a search engine to get a result. One approach to testing might be to deserialize and save that response from the search engine and compare it against future runs of the test.

There are a couple of problems with this approach. The biggest problem is that this test would be dependent on a constant response for a search term from a search engine and there’s no way we can guarantee that the response for a search term will always be the same. Second, this test would go far beyond our goal of unit testing updateWith(:_) and would end up being something more like an integration test between our app and some search engine.

So we need to separate concerns between our subject under test and that function’s dependencies. In Swift this is generally accomplished through Dependency Injection.

Let’s take another look at the parameter list for our function:

updateWith(searchTerm: String, requestManager: RequestManageable)

Since the Type for requestManager has been defined as the protocol RequestManageable it means we can pass any object that conforms to this protocol as the requestManager used by the function. So instead of passing the requestManager , that performs network request operations, we can pass a different object… a spy!

class MockRequestManager: RequestManageable { var invokedPerformSearch = false var invokedPerformSearchParameters: (searchTerm: String, searchEngine: SearchEngine)? var stubbedPerformSearchResult: Result<SearchResults, Error>! func performSearch(searchTerm: String, searchEngine: SearchEngine) -> Result<SearchResults, Error> { invokedPerformSearch = true invokedPerformSearchParameters = (searchTerm, searchEngine) return stubbedPerformSearchResult } }

This time, when performSearch() is called, the values passed in to the function and the status of the invocation of the function is recorded on the MockRequestManager object.

Now we’re finally ready to write our Unit Test!

class SearchTests: XCTestCase { func testSearch() { // Arrange let mockRequestManager = MockRequestManager() mockRequestManager.stubbedPerformSearchResult = Result.success(SearchResults()) // Act SearchActions().updateWith(searchTerm: "test", requestManager: mockRequestManager) // Assert XCTAssertEqual(mockRequestManager.invokedPerformSearch, true) // 1 XCTAssertEqual(mockRequestManager.invokedPerformSearchParameters?.searchEngine, .duckDuckGo) // 3 XCTAssertEqual(mockRequestManager.invokedPerformSearchParameters?.searchTerm, "test") // 3 } }

The important thing we’re testing here is that calling the updateWith(:_) function in turn calls the performSearch() function of whatever RequestManager is injected into the function. Whether it’s a spy or the real thing, this test will confirm that we’re invoking it as expected. We also want to confirm that the parameters passed to the RequestManager were not modified by the previous function and that they were successfully passed. The mockRequestManager object spies on parameters passed and saves them to a variable so we can compare them to the hardcoded values we expected in the test.

So that is a super simple example of a White Box test in Swift. Rather than examining the return value of the function and evaluating whether it makes sense based on the input we passed in to the function, in a White Box test we use dependency injection and mocked objects to spy on the inner workings of the function and we interrogate those spies directly during our assertion phase of the test.

Now it’s time to check out one of the many White Box tests like this that I found across the 10 open source apps I explored. I actually found 2 different flavors of this test. One flavor of the test used Spies to verify side effects during the test. Another flavor of White Box testing did not require a Spy, but instead relied on interrogating the state of delegate objects to confirm that the state of the application was changing as expected. We’ll see examples of both styles next.

Test Case 1 – Brave Browser (With Spies)

In this example we’ll take a look at a feature that the Brave browser implements for saving bookmarks. It turns out that you can actually save any string as a Bookmark in the Brave app, so we’ll be testing that we can save the string: “I Love Chocolate” to our bookmarks repository and retrieve it.

Since we don’t want to actually write anything to a storage repository during this test the first part of the test is to setup our spies for the test. This happens in the setUp() phase of the XCTestCase.

override func setUp() { super.setUp() delegate = MockTopSitesDelegate() dataSource = MockFavoritesDataSource() vc = FavoritesViewController(profile: MockProfile(), dataSource: dataSource) vc.delegate = delegate collectionView = UICollectionView(frame: CGRect.zero, collectionViewLayout: UICollectionViewFlowLayout()) }

You can check out this test in its entirety as well as the MockTopSitesDelegate and MockFavoritesDataSource at the Github repository here. However, if you followed along with the last example then you know what’s going on there fundamentally already.

One interesting thing to note about the MockTopSitesDelegate and MockFavoritesDataSource objects are that they conform to the UICollectionViewDelegate and UICollectionViewDataSource protocols as well. By performing an action on the ViewController's collectionView property we’ll be able to see the resulting state changes reflected on the mock delegate.

func testTopSiteDelegate_ReceivesString() { let index = IndexPath(item: 0, section: 0) // 1 let I_LOVE_CHOCOLATE = "I Love Chocolate" // 1 dataSource.bookmarks[index] = createBookmark(I_LOVE_CHOCOLATE) // 1 // Firing `UICollectionView` delegate method // 2 vc.collectionView(collectionView, didSelectItemAt: index) XCTAssertEqual(delegate.input, I_LOVE_CHOCOLATE, "The Favorites destination is incorrect.") // 3 XCTAssertFalse(delegate.isReturningURL, "Favorites should work for any string NOT just URL's.") // 4 }

Note: I’ve added some additional numbered annotation above, beyond what’s present in the source code so you can more easily follow along

To setup the test, a new bookmark is added to the dataSource’s Bookmarks dictionary. This is the action our test takes. Calling this function doesn’t return anything for us to test, but there are side effects to calling this function that we need to confirm were executed. Here, we assert that one of the side effects of selecting an item at the index where we saved our bookmark is that the delegate’s input property is filled with the value of our original string. Here we assert that the delegate is properly informed that the bookmark is not a URL.

Test Case 2 – Kickstarter (No Spies)

This Kickstarter test is even more simple to follow along with because no spies were used for the test. That means we don’t need to use dependency injection, instead we’ll just verify that the dataSource responds properly to the load(:_) method. This test, is of course verifying a slightly different behavior than either of the previous tests, but because we’re testing modifications to the state of an object rather than the inputs or outputs from a function I still categorize this under the White Box Testing umbrella.

func testSurvey() { let section = ActivitiesDataSource.Section.surveys.rawValue // Test Part 1 self.dataSource.load(surveys: [.template]) XCTAssertEqual(section + 1, self.dataSource.numberOfSections(in: self.tableView)) XCTAssertEqual(1, self.dataSource.tableView(self.tableView, numberOfRowsInSection: section)) // Test Part 2 XCTAssertEqual("ActivitySurveyResponseCell", self.dataSource.reusableId(item: 0, section: section)) self.dataSource.load(surveys: []) XCTAssertEqual(section + 1, self.dataSource.numberOfSections(in: self.tableView)) XCTAssertEqual(0, self.dataSource.tableView(self.tableView, numberOfRowsInSection: section)) }

I’ve added some comments above to the original source the calls out the two different parts to testSurvery() .

In Part 1, the data source is loaded with a data object representing a single template survey. The test then asserts that the number of sections tracked by the data source is equal to 1 ( section = 0) and the numberOfRowsInSection is equal to 1.

Test Part 2 will test that if we load an empty array of surveys that the data source still correctly reports a single section, but this time that 0 rows are displayed in that section.

When To Use This Test?

White Box Testing is very useful for verifying state changes to objects, we saw that with the data source in the test we just looked at. Additionally, these tests are good for verifying the side effects of calling a function (e.g. calling other functions, posting notifications, writing to logs, etc). Finally White Box Tests are good for verifying expected interactions between the subject under test and other objects. We saw this with the “Simplified Test Case” where we tested that the parameters sent to the mockRequestManager were correct.

Unless you are writing a fully “Functional App” with zero side effects, then you will probably need to write some kind of White Box Tests for your app. White Box Testing techniques were the most used technique I saw across the Open Source apps I explored. In face 9 out of 10 Apps used some tests that I thought fit under the White Box Testing umbrella. You can check out all of those apps and their tests in the links at the end of this post.

What Tools Can I Use To Write These Tests?

In days of Objective-C, your objects would inherit from NSObject. This gave a good foothold for mocking libraries to pull some tricks and provide an easy API for making your existing objects perform like Mock objects at Runtime during testing. Some of this is still possible with Swift if you want to inherit from NSObject and annotate your classes with @objc . Read this post for more detailed information on this problem: http://ocmock.org/swift/

There are mocking libraries for Swift (Cuckoo, MockingBird, OCMock), but none of the Apps I explored for this series used a specific library for Mocking. Instead, they opted to write mocks by hand when needed.

There’s nothing wrong with that approach, but I really like to take an in-between approach on apps I work on. I understand the urge to not want to depend on a separate library just for mocking purposes… but I’m also lazy… and I like things to be uniform across the project.

Take a look at the MockRequestManager we used above again:

class MockRequestManager: RequestManageable { var invokedPerformSearch = false var invokedPerformSearchParameters: (searchTerm: String, searchEngine: SearchEngine)? var stubbedPerformSearchResult: Result<SearchResults, Error>! func performSearch(searchTerm: String, searchEngine: SearchEngine) -> Result<SearchResults, Error> { invokedPerformSearch = true invokedPerformSearchParameters = (searchTerm, searchEngine) return stubbedPerformSearchResult } }

This spy was created by creating a few variables and setting them based on the interaction with the MockRequestManager object. Most Spies and Mocks pretty much look the same as this object (give or take certain properties or counters).

Because each mock is entirely built on a protocol definition, we can use that definition to generate a mock directly.

My Favorite way of generating Mock Objects is through the SwiftMockGeneratorForXcode extension that integrates mock code generation options directly into the Xcode Editor menu. Because this generator just creates Swift code it means you don’t need any dependencies installed to use it. So you don’t even need to convince anyone else on your team to use it. But it’s so convenient I’d probably let them in on the secret if I were you 😉

Which Apps Use White Box Testing?

Brave ✅

Wikipedia ✅

Firefox ✅

Kickstarter ✅

WooCommerce ✅

Artsy ✅

Duck Duck Go ✅

WordPress ✅

Magazine Layout ✅