I have interest in using the quick check library but it seems that it is designed to test properties. What I would like to do is generate random data for my defined data types and test functions I have written. I do not care about what the result is, just if the function produces a runtime error when fed random data. All of the quick check examples I have seen are for testing properties of functions like is the result greater than 5 when fed random data. Is there a way to use quick check in this manner? Something like
data Result = A | B
fun :: Result -> Int
fun A = 5
main = check fun
In the above code I have a custom data type, and a incomplete function. This function will fail if passed B. There are of course more types of runtime errors than just incomplete functions. I would like to have check generate data, and feed it to the function. Not caring what the result is. Is quick check able to do this?
Edit -
I should note that I am not looking for flags that check for incomplete patterns, and what not. I am interested in general purpose runtime error checks.
Rather make sure that functions that don't handle IO
don't throw exceptions if possible.
Exceptions can only get caught in IO
, and they kind of break the things you otherwise expect from a pure function.
It's possible with some helpers from
Control.Exception
and Test.QuickCheck.Monadic
:
> import Control.Exception (Exception, PatternMatchFail,
> SomeException, evaluate,
> fromException, try)
> import Data.Either (either)
> import Test.Hspec (anyException, describe, hspec, it,
> shouldThrow)
> import Test.QuickCheck hiding (Result)
> import Test.QuickCheck.Monadic (assert, monadicIO, run)
For starters, let's write a function that enables us to check that a certain exception is thrown:
> throwsExceptionOr :: Exception e => (e -> Bool) -> (a -> Bool) -> a -> IO Bool
> throwsExceptionOr pe pa = fmap (either pe pa) . try . evaluate
This enables you to write tests like this:
> prop_fun_1 x = monadicIO . run $
> throwsExceptionOr (const True :: SomeException -> Bool)
> (== 5)
> (foo x)
throwsExceptionOr
is very general, so that you can define your own helpers:
> -- | Should always throw an exception.
> throwsException :: Exception e => (e -> Bool) -> a -> IO Bool
> throwsException p = fmap (either p (const False)) . try . evaluate
> -- | Should either pass the test or throw an exception.
> exceptionOr :: a -> (a -> Bool) -> IO Bool
> exceptionOr x p = fmap (either anyException p) . try . evaluate $ x
> where
> anyException :: SomeException -> Bool
> anyException = const True
Now you can write your tests as usual:
> data Result = A | B deriving (Enum, Show, Eq, Ord, Bounded, Read)
>
> instance Arbitrary Result where
> arbitrary = elements [A, B]
>
> foo :: Result -> Int
> foo A = 5
>
> prop_foo x = monadicIO . run $ foo x `exceptionOr` (== 5)
You can go further and move the monadicIO . run
into another helper, but
that's left as an exercise. Furthermore, you can make the functions compatible
with other testing frameworks, such as hspec
or tasty
:
> main :: IO ()
> main = hspec $ do
> describe "foo" $ do
> it "either returns five or throws a pattern match fail" $ propertyIO $ \x ->
> throwsExceptionOr patternMatchFail (==5) (foo x)
>
> it "throws an exception on input B" $
> evaluate (foo B) `shouldThrow` anyException
>
> where
> patternMatchFail :: PatternMatchFail -> Bool
> patternMatchFail _ = True
>
> -- I think there is a better combinator for this
> propertyIO f = property $ \x -> monadicIO . run $ f x
That being said, regardless of the used languages you want to get rid of
possible runtime errors at compile time if possible. This includes getting
rid of partial functions or possible type nonsense. This depends on the actual
use, of course. If you can verify that head
always gets called on a non-empty
list throughout your program, go ahead and use it. If you can't, use pattern
matching (see this discussion
on head
's type).
Either way, given that older versions of GHC don't provide stack calls, you rather want to have errors at compile time than errors without a stack trace at runtime (recent versions of GHC have some nice features for this).