Custom validity and validity-based testing in Haskell
Values of custom types usually have invariants imposed upon them. In this
post I motivate and announce the
genvalidity-hspec libraries that
have just come out.
Contrary to what we might like to think, an absence of compilation errors does not imply correctly functioning code. In some cases we expect some invariants to hold about our data, but we don’t necessarily check them rigorously.
This is a situation in which the typesystem may be expensive to use to guarantee correctness. That’s why we will use testing instead.
A running example
I will take a contrived example to keep the blogpost short. Assume we have the following context:
-- | [...] -- INVARIANT >= 2 newtype GreaterThanOne = GreaterThanOne Int -- | [...] -- INVARIANT >= 2 -- INVARIANT must be a prime newtype Prime = Prime Int
For now, we just assume that invariants hold. If we now want to write a
primeFactorisation function, it might have the following
primeFactorisation :: GreaterThanOne -> Maybe [Prime]
Now we will go on to how we can actually make this safe and test the validity assumptions.
To be able to express the invariants that we would like to enforce on our data, we can write the following function:
myDataIsValid :: MyData -> Bool
We express, in code, exactly what it means for a
MyData to be
valid. That way we can later check whether our
This is the validity package comes in. It is really simple and contains not much more than the following type class, but it opens up some possibilities which I will talk about in the rest of this post.
class Validity a where isValid :: a -> Bool
In the case of the running example, the
could look as follows.
instance Validity GreaterThanOne where isValid (GreaterThanOne n) = n >= 2 isPrime :: Int -> Bool instance Validity Prime where isValid (Prime n) = n >= 2 && isPrime n
Now that we have this concept of
Validity, we can start
writing tests using it. We will ignore the tests which assert that the output
is a valid prime factorisation for now. We can then write the following tests
with respect to validity:
describe "primeFactorisation" $ do it "fails for invalid GreaterThanOne's" $ forAll ((GreaterThanOne <$> arbitrary) `suchThat` (not . isValid)) $ \i -> primeFactorisation i `shouldBe` Nothing it "produces valid primes if it succeeds" $ forAll (GreaterThanOne <$> arbitrary) $ \i -> case primeFactorisation i of Nothing -> return () -- Can happen Just ps -> ps `shouldSatisfy` all isValid
This is quite a mouthful, isn’t it? There are also some non-stylistic problems:
someGenerator `suchThat` (not . isValid)runs
someGeneratorand retries as long as
isValidis satisfied. For types that are mostly valid, this can take a long time and will slow down testing significantly.
GreaterThanOne <$> arbitrarybecome quite large (in code) for larger structures. Ideally we would only write them once.
class Validity a => GenValidity a where genUnchecked :: Gen a genValid :: Gen a genValid = genUnchecked `suchThat` isValid genInvalid :: Gen a genInvalid = genUnchecked `suchThat` (not . isValid)
When you instantiatie
GenValidity for your custom data type,
isValid become much easier to write:
describe "primeFactorisation" $ do it "fails for invalid GreaterThanOne's" $ forAll genInvalid $ \i -> primeFactorisation i `shouldBe` Nothing it "produces valid primes if it succeeds" $ forAll genUnchecked $ \i -> case primeFactorisation i of Nothing -> return () -- Can happen Just ps -> ps `shouldSatisfy` isValid
genInvalid Has a default implementation, but when generating
GreaterThanOnes, on average 50% of all generations have to be
retried at least once. We can specialize this implementation to run faster by
using an absolute value function:
class GenValidity GreaterThanOne where genUnchecked = GreaterThanOne <$> arbitrary genValid = (GreaterThanOne . abs) <$> arbitrary
We can use these new toys to write tests as described above, but we can
also use some of the standard tests that are available via
genvalidity-hspec. For example, the above tests can be rewritten
describe "primeFactorisation" $ do it "fails for invalid GreaterThanOne's" $ failsOnInvalidInput primeFactorisation it "produces valid primes if it succeeds" $ validIfSucceeds primeFactorisation
Because we have now written a custom implementation of
GreaterThanOne, we should also add the
validitySpec (Proxy :: Proxy GreaterThanOne)
This will ensure that
keep working as intended. However, it cannot check that all possible
valid(/invalid) values can still be generated by
genInvalid), so be careful and check that