How can I avoid running some tests in parallel?

I have a collection of tests. There are a few tests that need to access a shared resource (external library/API/hardware device). If any of these tests run in parallel, they fail.

I know I could run everything using --test-threads=1 but I find that inconvenient just for a couple of special tests.

Is there any way to keep running all tests in parallel and have an exception for a few? Ideally, I would like to say do not run X, Y, Z at the same time.

回答1:

As mcarton mentions in the comments, you can use a Mutex to prevent multiple pieces of code from running at the same time:

#[macro_use]
extern crate lazy_static; // 1.0.2

use std::{sync::Mutex, thread::sleep, time::Duration};

lazy_static! {
    static ref THE_RESOURCE: Mutex<()> = Mutex::new(());
}

type TestResult<T = ()> = std::result::Result<T, Box<std::error::Error>>;

#[test]
fn one() -> TestResult {
    let _shared = THE_RESOURCE.lock()?;
    eprintln!("Starting test one");
    sleep(Duration::from_secs(1));
    eprintln!("Finishing test one");
    Ok(())
}

#[test]
fn two() -> TestResult {
    let _shared = THE_RESOURCE.lock()?;
    eprintln!("Starting test two");
    sleep(Duration::from_secs(1));
    eprintln!("Finishing test two");
    Ok(())
}

If you run with cargo test -- --nocapture, you can see the difference in behavior:

No lock

running 2 tests
Starting test one
Starting test two
Finishing test two
Finishing test one
test one ... ok
test two ... ok

With lock

running 2 tests
Starting test one
Finishing test one
Starting test two
test one ... ok
Finishing test two
test two ... ok

Ideally, you'd put the external resource itself in the Mutex to make the code represent the fact that it's a singleton and remove the need to remember to lock the otherwise-unused Mutex.

This does have the massive downside that a panic in a test (a.k.a an assert! failure) will cause the Mutex to become poisoned. This will then cause subsequent tests to fail to acquire the lock. If you need to avoid that and you know the locked resource is in a good state (and () should be fine...) you can handle the poisoning:

let _shared = THE_RESOURCE.lock().unwrap_or_else(|e| e.into_inner());

If you need the ability to run a limited set of threads in parallel, you can use a semaphore. Here, I've built a poor one using Condvar with a Mutex:

use std::{
    sync::{Condvar, Mutex},
    thread::sleep,
    time::Duration,
};

#[derive(Debug)]
struct Semaphore {
    mutex: Mutex<usize>,
    condvar: Condvar,
}

impl Semaphore {
    fn new(count: usize) -> Self {
        Semaphore {
            mutex: Mutex::new(count),
            condvar: Condvar::new(),
        }
    }

    fn wait(&self) -> TestResult {
        let mut count = self.mutex.lock().map_err(|_| "unable to lock")?;
        while *count == 0 {
            count = self.condvar.wait(count).map_err(|_| "unable to lock")?;
        }
        *count -= 1;
        Ok(())
    }

    fn signal(&self) -> TestResult {
        let mut count = self.mutex.lock().map_err(|_| "unable to lock")?;
        *count += 1;
        self.condvar.notify_one();
        Ok(())
    }

    fn guarded(&self, f: impl FnOnce() -> TestResult) -> TestResult {
        // Not panic-safe!
        self.wait()?;
        let x = f();
        self.signal()?;
        x
    }
}

lazy_static! {
    static ref THE_COUNT: Semaphore = Semaphore::new(4);
}

THE_COUNT.guarded(|| {
    eprintln!("Starting test {}", id);
    sleep(Duration::from_secs(1));
    eprintln!("Finishing test {}", id);
    Ok(())
})