Error with XCTestExpectation: API violation - mult

Question

I'm using XCTestExpectations in Xcode 6 (Beta 5) for asynchronous testing. All my asynchronous tests pass individually every time I run them. However, when I try to run my entire suite, some tests do not pass, and the app crashes.

The error I get is says API violation - multiple calls made to -[XCTestExpectation fulfill]. Indeed, this is not true within a single method; my general format for my tests is shown below:

- (void) someTest {
    /* Declare Expectation */
    XCTestExpectation *expectation = [self expectationWithDescription:@"My Expectation"];
    [MyClass loginOnServerWithEmail:@"[e-mail]" andPassword:@"asdfasdf" onSuccess:^void(User *user) {
        /* Make some assertions here about the object that was given. */

        /* Fulfill the expectation */
        [expectation fulfill];
    }];

    [self waitForExpectationsWithTimeout:5.0 handler:^(NSError *error) {
        /* Error handling here */
    }];
}

Again, these tests do pass when run individually, and they are actually making network requests (working exactly as intended), but together, the collection of tests fail to run.

I was able to have a look at this post here, but was unable to get the solution to work for me.

Additionally, I'm running OSX Mavericks and using Xcode 6 (Beta 5).

Answer 1

Here is probably the answer you are looking for :

XCTestExpectation: how to avoid calling the fulfill method after the wait context has ended?

It at least fixed the issue for me.

Answer 2

I don't think using __weak or __block is a good approach. I have written many unit tests using XCTestExpectation for awhile and never had this problem until now. I finally found out that real cause of the problem which potentially may cause bugs in my app. The root cause of my problem is that startAsynchronousTaskWithDuration calls the completionHandler multiple time. After I fix it the API violation went away!

[self startAsynchronousTaskWithDuration:4 completionHandler:^(id result, NSError *error) {
    XCTAssertNotNil(result);
    XCTAssertNil(error);
    [expectation fulfill];
}];

Although it took me a few hours to fix my unit tests but I came to appreciate the API violation error which will help me avoid future runtime problem in my app.

Answer 3

Try declaring your expectationWithDescription as weak and unwrap the optional "expected" variable.

 weak var asyncExpectation = expectationWithDescription("expectation")

 check for options in the block.
 if let asyncExpectation = asyncExpectation{
    asyncExpectation.fulfill()
 }

This avoids deallocation of asyncExpectation variable and calling your expectation on nil.

Answer 4

I think it's likely that you have a retain cycle issue somewhere preventing to release your object which is calling the block in which the expectation is fulfill multiple time.

Otherwise, if it's an expected behavior that your expectation is called multiple time, I've write a little extension allowing to specify an expectation count:

import XCTest

extension XCTestExpectation {
    private class IntWrapper {
        let value :Int!
        init?(value:Int?) {
            self.value = value
            if (value == nil) {
                return nil
            }
        }
    }

    private struct AssociatedKey {
        static var expectationCountKey = ""
    }

    var expectationCount:Int? {
        get {
            return objc_getAssociatedObject(self, &AssociatedKey.expectationCountKey) as? Int
        }
        set {
            objc_setAssociatedObject(self, &AssociatedKey.expectationCountKey, newValue, .OBJC_ASSOCIATION_RETAIN)
        }
    }

    func decrementalFulfill() {
        guard let expectationCount = self.expectationCount else {
            fulfill()
            return
        }
        self.expectationCount = expectationCount - 1
        if self.expectationCount <= 0 {
            fulfill()
        }
    }
}

full code (with test :) here: https://gist.github.com/huguesbr/7d110bffd043e4d11f2886693c680b06