How can I silently catch panics in QuickCheck test

问题:

In the tests of my overflower_support crate, I have found that I get a lot of spurious reports of panics which are already handled using std::panic::catch_unwind(_). This is a bit unfortunate, as it obscures the real errors that may happen. The messages look like:

thread 'safe' panicked at 'arithmetic overflow', src/lib.rs:56

To quell those distracting messages, I introduced the dont_panic(..) function, which hijacks the panic handler, calls a closure and resets the panic handler when done, returning the closures result. It looks like this:

fn dont_panic<F, A, R>(args: A, f: F) -> R
    where F: Fn(A) -> R
{
    let p = panic::take_hook();
    panic::set_hook(Box::new(|_| ()));
    let result = f(args);
    panic::set_hook(p);
    result
}

However, using this function within the function to check somewhat surprisingly not only quells the desired messages, but also quickcheck's error output, which is obviously valuable to me. This occurs even when limiting tests to one thread.

#[test]
fn test_some_panic() {
    fn check(x: usize) -> bool {
        let expected = if x < 256 { Some(x) } else { None };
        let actual = dont_panic(|| panic::catch_unwind(|| { assert!(x < 256); x }).ok());
        expected == actual
    }
    quickcheck(check as fn(usize) -> bool);
}

How can I hide the caught panics from my code while keeping QuickCheck's panics visible?

回答1:

The default panic handler is printing panic information unconditionally on stderr.

You want to register your own handler.

回答2:

I've met the same problem and a few others, and I ended up writing a crate to solve them:

panic-control

With it, your example might be solved by running in a "quiet" thread (assuming you weren't interested in using catch_unwind specifically):

use panic_control::spawn_quiet;

#[test]
fn test_some_panic() {
    fn check(x: usize) -> bool {
        let expected = if x < 256 { Some(x) } else { None };
        let h = spawn_quiet(|| { assert!(x < 256); x });
        let actual = h.join().ok();
        expected == actual
    }
    quickcheck(check as fn(usize) -> bool);
}

回答3:

There were two problems with my approach:

1. The tests run in parallel (and quickcheck appears to add some parallelism of its own, as -j 1 appears ineffective to quell the panic messages).
2. The message gets written (or otherwise suppressed by set_hook(_)) no matter if there's a catch_unwind(_) or not.

However, dpc.pw's idea to distinguish based on files in the panic handler was spot-on. I changed my approach to call an install_handler() function before calling quickcheck(_), which I reproduce here in full:

use std::panic;
use std::sync::{Once, ONCE_INIT};

static HANDLER : Once = ONCE_INIT;

fn install_handler() {
    HANDLER.call_once(|| {
        let p = panic::take_hook();
        panic::set_hook(Box::new(move|info| {
            if info.location().map_or(false, |l| l.file() != "src/lib.rs" &&
                    !l.file().ends_with("/num/mod.rs")) {
                p(info);
            }
        }));
    })
}

This will quell the panic messages if the panic came from src/lib.rs (which is my overflower_support code) or somewhere from /num/mod.rs (because the Rust libcore code may panic, too).

Note that you could omit the Once, but this would add the handler multiple times and increase the size of stack traces considerably while exacerbating test performance.