Part I

Developers who have incorporated unit testing into their development process already know its advantages: cleaner code, courage to refactor, and improved productivity. Writing unit tests is easy; it is writing good tests that makes automated unit testing and test-driven development powerful tools. This article provides some basic unit-test patterns that help write good tests, and anti-patterns you should avoid. For beginners, Gil Zilberfeld has also provided a good introduction to unit testing

A great unit test will do the following:

Fail only when a bug has been introduced

Fail Fast, for instant feedback

Easy to understand what scenario is being tested and needs fixing

To do so we follow some of the following unit-test patterns.

Modern tests contain three parts:

[TestMethod] public void GetCount_ItemCountIsZero_NoNewMessages() { //Arrange Mailbox mailbox = new Mailbox(); //Act var result = mailbox.GetCount(0); //Assert Assert.AreEqual("No new messages.", result); } 1 2 3 4 5 6 7 8 9 10 11 12 [ TestMethod ] public void GetCount_ItemCountIsZero_NoNewMessages ( ) { //Arrange Mailbox mailbox = new Mailbox ( ) ; //Act var result = mailbox . GetCount ( 0 ) ; //Assert Assert . AreEqual ( "No new messages." , result ) ; }

Arrange: setup everything needed for the running the tested code. This includes any initialization of dependencies, mocks, and data needed for the test to run.

setup everything needed for the running the tested code. This includes any initialization of dependencies, mocks, and data needed for the test to run. Act: Invoke the code under test.

Invoke the code under test. Assert: Specify the pass criteria for the test, which fails it if not met.

This pattern is very readable and consistent. It’s very easy to identify the different parts of the test and to create tests for other scenarios from a root test. Get more information about this pattern here

In our unit tests, we’d like to test the result of the method execution. Usually, the code results in a state change, which is exposed by the object. We can look at state using:

Values directly returned from the method

Values exposed through the object fields

Values exposed through other methods or properties of the object

Values that come from outside the object, for example, static state or a shared data structure.

Tests that assert these values are called state-based tests. Depending on the test framework of choice, you’ll find Assert APIs that let you compare these values to an expected set.

In this example, we’re using MS-Test framework’s Assert.AreEqual to compare a returned value from method.

[TestMethod] public void GetCount_ItemCountIsZero_NoNewMessages() { //Arrange Mailbox mailbox = new Mailbox(); //Act var result = mailbox.GetCount(0); //Assert Assert.AreEqual("No new messages.", result); } 1 2 3 4 5 6 7 8 9 10 11 12 [ TestMethod ] public void GetCount_ItemCountIsZero_NoNewMessages ( ) { //Arrange Mailbox mailbox = new Mailbox ( ) ; //Act var result = mailbox . GetCount ( 0 ) ; //Assert Assert . AreEqual ( "No new messages." , result ) ; }

State-based tests are usually more robust than their interaction-based counterparts. They don’t increase coupling between the test and code in the Assert part since the assertion is made on data that is exposed through public interfaces. There’s no risk of tests breaking because of refactoring the internal implementation of methods.

This also adds to their readability: The tests describe behavior through public interfaces, without exposing internal mechanisms.

Sometimes, tested methods do not expose their result directly. For example, void methods do not return a value. If they modified some exposed data, we can then check that the data has changed. Yet, if the tested logic calls methods on other dependencies, we still can’t examine actual results.

When this happens, we cannot simply assert on data, but we need to check that the method got called, maybe checking the passed arguments as well.

Our tested method, ReportIfSpam, checks if the message argument is spam, and then calls the static Administrator.SendEmail method. Neither method returns a value.

public void ReportIfSpam(Message message) { if (message.IsSpam) Administrator.SendEmail(message); } 1 2 3 4 5 public void ReportIfSpam ( Message message ) { if ( message . IsSpam ) Administrator . SendEmail ( message ) ; }

We’d like to test what happens in case of a spam message. Our pass criterion is that we called the SendEmail method, with the supplied message as argument.

(In this test, part of the setup is to fake calls to the Administrator class; we don’t want to send a real email every time we run the test).

Most test frameworks don’t support interaction-based tests. We use mocking frameworks, in this case, Typemock Isolator, for this purpose.

[TestMethod] public void ReportIfSpam_MessageIsSpam_SendEmailToAdministrator() { Mailbox mailbox = new Mailbox(); Message spamMessage = new Message(); spamMessage.IsSpam = true; // Set message to be spam Isolate.Fake.StaticMethods<administrator>(); // Fake sending the emails mailbox.ReportIfSpam(spamMessage); //Assert the call was made with the correct argument Isolate.Verify.WasCalledWithExactArguments(() => Administrator.SendEmail(spamMessage)); } </administrator> 1 2 3 4 5 6 7 8 9 10 11 [ TestMethod ] public void ReportIfSpam_MessageIsSpam_SendEmailToAdministrator ( ) { Mailbox mailbox = new Mailbox ( ) ; Message spamMessage = new Message ( ) ; spamMessage . IsSpam = true ; // Set message to be spam Isolate . Fake . StaticMethods < administrator > ( ) ; // Fake sending the emails mailbox.ReportIfSpam(spamMessage); //Assert the call was made with the correct argument Isolate.Verify.WasCalledWithExactArguments(() => Administrator.SendEmail(spamMessage)); } < / administrator >

Interaction-based tests require knowledge about the internal implementation of the method. Since they increase coupling, they are less robust than state-based tests. In addition, to understand the test failure, the reader needs to know more about the implementation, rather than interface, reducing readability.

As a guideline, given the option, we’d prefer state-based tests over interaction-based tests.

Throwing exceptions is part of behavior we’d like to test: making sure exceptions are thrown when they should, along with correct information.

Testing thrown exceptions depends on your test framework, each uses a different sub-pattern. NUnit and xUnit allow us to use APIs, that combine the Act and Assert parts. For example, in NUnit:

[Test] public void ReportSpam_MessageIsSpam_ExceptionIsThrown() { Mailbox mailbox = new Mailbox(); Message spamMessage = new Message(); Assert.Throws<spammessageexception>(()=>mailbox.ReportIfSpam()); } </spammessageexception> 1 2 3 4 5 6 7 [ Test ] public void ReportSpam_MessageIsSpam_ExceptionIsThrown ( ) { Mailbox mailbox = new Mailbox ( ) ; Message spamMessage = new Message ( ) ; Assert . Throws < spammessageexception > ( ( ) = > mailbox . ReportIfSpam ( ) ) ; } < / spammessageexception >

In MSTest (and old versions of NUnit), we test exceptions with an ExpectedException attribute. This version leaves the Arrange and Act parts intact, but breaks the AAA sequence.

[TestMethod] [ExpectedException(typeof(SpamMessageException))] public void ReportSpam_MessageIsSpam_ExceptionIsThrown() { Mailbox mailbox = new Mailbox(); Message spamMessage = new Message(); mailbox.ReportIfSpam(); } 1 2 3 4 5 6 7 8 9 [ TestMethod ] [ ExpectedException ( typeof ( SpamMessageException ) ) ] public void ReportSpam_MessageIsSpam_ExceptionIsThrown ( ) { Mailbox mailbox = new Mailbox ( ) ; Message spamMessage = new Message ( ) ; mailbox . ReportIfSpam ( ) ; }

Comparing the two, the first is more readable and focused. In complex tests, it’s easier to see what we test and if the test fails, it’s easy to see what caused it. When an attribute-based test fails, it’s not apparent where the error occurred and may require debugging to resolve.

It is a good practice to unit test only public interfaces. We usually test for exposed behavior, rather than specific implementation. Increased coupling between the test and the code, may result in the test breaking when the internal implementation has changed, while the exposed behavior hasn’t changed.

That said, there are still cases where we want to test an internal behavior. A classic example is a Balanced Binary Tree. In order to test that the tree is balanced, we will need to access a private interface. They may benefit from testing the algorithms directly, rather than through the object interface. Legacy code frequently includes code that needs testing but is not exposed through a public interface.

There are a couple of solutions that incur changes to the tested code, or collaboration between code and tests.

Making changes to the tested code

Explicitly change private methods to public

Exposing internal methods using the InternalVisibleToAttribute.

Accessing the internal members from the test

Using reflection

Using other tools

The following is an example of how to invoke a private member function in a test using Isolator.

[TestMethod] public void PrivateGetCount_ItemCountIsFive_FiveNewMessages() { //Arrange Mailbox mailbox = new Mailbox(); //Act var result = Isolate.Invoke.Method(mailbox, "GetCount", new object[] { 5 }); //Assert Assert.AreEqual("New messages: 5", result); } 1 2 3 4 5 6 7 8 9 10 11 12 [ TestMethod ] public void PrivateGetCount_ItemCountIsFive_FiveNewMessages ( ) { //Arrange Mailbox mailbox = new Mailbox ( ) ; //Act var result = Isolate . Invoke . Method ( mailbox , "GetCount" , new object [ ] { 5 } ) ; //Assert Assert . AreEqual ( "New messages: 5" , result ) ; }