Taking your unit tests to the next level.

I have been using ScalaCheck testing library for at least 2 years now. It allows you to take your unit tests to the next level.



You can do Property-based testing by generating a lot of tests with random data and asserting properties on your functions. A simple code example is described below.

You can do by generating a lot of tests with random data and asserting properties on your functions. A simple code example is described below. You can do Law testing that is even more powerful and allows you to check mathematical properties on your types.

Property-based testing

Here is our beloved User data type:

case class User(name: String, age: Int)

And a random User generator:

import org.scalacheck.{ Gen, Arbitrary } import Arbitrary.arbitrary implicit val randomUser: Arbitrary[User] = Arbitrary(for { randomName <- Gen.alphaStr randomAge <- Gen.choose(0,80) } yield User(randomName, randomAge))

We can now generate a User like this:

scala> randomUser.arbitrary.sample res0: Option[User] = Some(User(OtwlaaxGbmdhuorlmgvXitbmGfbgetm,22))

Let’s define some functions on the User :

def isAdult: User => Boolean = _.age >= 18 def isAllowedToDrink : User => Boolean = _.age >= 21

Let’s claim that:

All adults are allowed to drink.

Can we somehow prove this? Is this correct for all users?

This is where property testing comes to the rescue. It allows us not to write specific unit-tests. Here they would be:

18-year-olds are not allowed to drink

19-year-olds are not allowed to drink

20-year-olds are not allowed to drink

All of these statements can be replaced by a single property check:

import org.scalacheck.Prop.forAll val allAdultsCanDrink = forAll { u: User => if(isAdult(u)) isAllowedToDrink(u) else true }

Let’s run it:

scala> allAdultsCanDrink.check() ! Falsified after 0 passed tests. > ARG_0: User(,19)

It fails as expected for a 19-year-old.

Property testing is awesome for a few reasons:

Saves time by writing less specific tests

Finds new use cases generated by Scala check that you forgot to handle

Forces you think in a more general way

Gives you more confidence for refactoring than conventional unit tests

Law testing

It gets better: let’s take it to the next level and define an Ordering between Users:

import scala.math.Orderingimplicit val userOrdering: Ordering[User] = Ordering.by(_.age)

We want to make sure that we didn't forget any edge cases and that we defined our order properly. This property has a name, and it’s called a total order. It needs to holds for the following properties:

Totality

Antisymmetry

Transitivity

Can we somehow prove this? Is this correct for all users?

This is possible without writing a single test!

We use cats-laws library to define the laws we want to test on the ordering we defined:

import cats.kernel.laws.discipline.OrderTests import cats._ import org.scalatest.FunSuite import org.typelevel.discipline.scalatest.Discipline import org.scalacheck.ScalacheckShapeless._ class UserOrderSpec extends FunSuite with Discipline { //needed boilerplate to satisfy the dependencies of the framework implicit def eqUser[A: Eq]: Eq[Option[User]] = Eq.fromUniversalEquals //convert our standard ordering to a `cats` order implicit val catsUserOrder: Order[User] = Order.fromOrdering(userOrdering) //check all mathematical properties on our ordering checkAll("User", OrderTests[User].order) }

Let’s run it:

scala> new UserOrderSpec().execute() UserOrderSpec: - User.order.antisymmetry *** FAILED *** GeneratorDrivenPropertyCheckFailedException was thrown during property evaluation. (Discipline.scala:14) Falsified after 1 successful property evaluations. Location: (Discipline.scala:14) Occurred when passed generated values ( arg0 = User(h,17), arg1 = User(edsb,17), arg2 = org.scalacheck.GenArities$$Lambda$2739/1277317528@41d7b4cf ) Label of failing property: Expected: true Received: false - User.order.compare - User.order.gt - User.order.gteqv - User.order.lt - User.order.max - User.order.min - User.order.partialCompare - User.order.pmax - User.order.pmin - User.order.reflexitivity - User.order.reflexitivity gt - User.order.reflexitivity lt - User.order.symmetry - User.order.totality - User.order.transitivity

Sure enough, it fails on the antisymmetry law! Same age and different names are not supposed to be equals. We forgot to use the name in our original Ordering , so let's fix it and rerun the laws:

implicit val userOrdering: Ordering[User] = Ordering.by( u => (u.age, u.name)) scala> new UserOrderSpec().execute() UserOrderSpec: - User.order.antisymmetry - User.order.compare - User.order.gt - User.order.gteqv - User.order.lt - User.order.max - User.order.min - User.order.partialCompare - User.order.pmax - User.order.pmin - User.order.reflexitivity - User.order.reflexitivity gt - User.order.reflexitivity lt - User.order.symmetry - User.order.totality - User.order.transitivity

And now it passes :)

If you are wondering what can you test besides Order s, go check out the docs here: https://typelevel.org/cats/typeclasses/lawtesting.html

Summary