# Custom validity and validity-based testing in Haskell

Date 2016-07-17

Values of custom types usually have invariants imposed upon them. In this post I motivate and announce the validity, genvalidity and genvalidity-hspec libraries that have just come out.

Contrary to what we might like to think, an absence of compilation errors does not imply correctly functioning code. In some cases we expect some invariants to hold about our data, but we don't necessarily check them rigorously.

This is a situation in which the typesystem may be expensive to use to guarantee correctness. That's why we will use testing instead.

### A running example

I will take a contrived example to keep the blogpost short. Assume we have the following context:

-- | [...]
-- INVARIANT >= 2
newtype GreaterThanOne
= GreaterThanOne Int

-- | [...]
-- INVARIANT >= 2
-- INVARIANT must be a prime
newtype Prime
= Prime Int


For now, we just assume that invariants hold. If we now want to write a safe primeFactorisation function, it might have the following type.

primeFactorisation :: GreaterThanOne -> Maybe [Prime]


Now we will go on to how we can actually make this safe and test the validity assumptions.

### Validity

To be able to express the invariants that we would like to enforce on our data, we can write the following function:

myDataIsValid :: MyData -> Bool


We express, in code, exactly what it means for a MyData to be valid. That way we can later check whether our MyData is valid.

This is the validity package comes in. It is really simple and contains not much more than the following type class, but it opens up some possibilities which I will talk about in the rest of this post.

class Validity a where
isValid :: a -> Bool


In the case of the running example, the Validity instances could look as follows.

instance Validity GreaterThanOne where
isValid (GreaterThanOne n) = n >= 2

isPrime :: Int -> Bool

instance Validity Prime where
isValid (Prime n) = n >= 2 && isPrime n


### Genvalidity

Now that we have this concept of Validity, we can start writing tests using it. We will ignore the tests which assert that the output is a valid prime factorisation for now. We can then write the following tests with respect to validity:

describe "primeFactorisation" $do it "fails for invalid GreaterThanOne's"$
forAll ((GreaterThanOne <$> arbitrary) suchThat (not . isValid))$ \i ->
primeFactorisation i shouldBe Nothing

it "produces valid primes if it succeeds" $forAll (GreaterThanOne <$> arbitrary) $\i -> case primeFactorisation i of Nothing -> return () -- Can happen Just ps -> ps shouldSatisfy all isValid  This is quite a mouthful, isn't it? There are also some non-stylistic problems: • someGenerator suchThat (not . isValid) runs someGenerator and retries as long as isValid is satisfied. For types that are mostly valid, this can take a long time and will slow down testing significantly. • Generators like GreaterThanOne <$> arbitrary become quite large (in code) for larger structures. Ideally we would only write them once.

Enter genvalidity. genvalidity provides a typeclass called GenValidity:

class Validity a => GenValidity a where
genUnchecked :: Gen a

genValid :: Gen a
genValid = genUnchecked suchThat isValid

genInvalid :: Gen a
genInvalid = genUnchecked suchThat (not . isValid)


When you instantiatie GenValidity for your custom data type, tests involving isValid become much easier to write:

describe "primeFactorisation" $do it "fails for invalid GreaterThanOne's"$
forAll genInvalid $\i -> primeFactorisation i shouldBe Nothing it "produces valid primes if it succeeds"$
forAll genUnchecked $\i -> case primeFactorisation i of Nothing -> return () -- Can happen Just ps -> ps shouldSatisfy isValid  genInvalid Has a default implementation, but when generating GreaterThanOnes, on average 50% of all generations have to be retried at least once. We can specialize this implementation to run faster by using an absolute value function: class GenValidity GreaterThanOne where genUnchecked = GreaterThanOne <$> arbitrary
genValid = (GreaterThanOne . abs) <$> arbitrary  ### Standard tests We can use these new toys to write tests as described above, but we can also use some of the standard tests that are available via genvalidity-hspec. For example, the above tests can be rewritten as follows: describe "primeFactorisation"$ do
it "fails for invalid GreaterThanOne's" $failsOnInvalidInput primeFactorisation it "produces valid primes if it succeeds"$
validIfSucceeds primeFactorisation


Because we have now written a custom implementation of genValid for GreaterThanOne, we should also add the validitySpec:

validitySpec (Proxy :: Proxy GreaterThanOne)


This will ensure that genValid and genInvalid keep working as intended. However, it cannot check that all possible valid(/invalid) values can still be generated by genValid(/genInvalid), so be careful and check that yourself.

If you liked this blog post, please consider becoming a supporter: