Cannot run Xcode (7.3) UI tests in Jenkins

2019-08-13 19:03发布

问题:

I am trying to run Xcode UI tests from command line on a Jenkins machine that work on my local computer. However, when I try to run the UI tests in Jenkins, they appear to fail to launch the app in the simulator. What happens is the device loads in the simulator, the UI test target opens, and the screen goes black. Then nothing seems to happen for 30 seconds until I get an error as follows (I expect the app to launch and the automation to begin):

2016-04-22 11:44:54.549 XCTRunner[33303:299557] Running tests...
11:44:54.707 XCTRunner[33303:299590] _XCT_testBundleReadyWithProtocolVersion:minimumVersion: reply received
11:44:54.711 XCTRunner[33303:299587] _IDE_startExecutingTestPlanWithProtocolVersion:16
2016-04-22 11:45:24.730 XCTRunner[33303:299557] *** Assertion failure in void _XCTFailInCurrentTest(NSString *, ...)(), /Library/Caches/com.apple.xbs/Sources/XCTest_Sim/XCTest-10112/XCTestFramework/Classes/XCTestCase.m:63
2016-04-22 11:45:24.732 XCTRunner[33303:299557] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: '_XCTFailInCurrentTest should only be called while a test is running.
Failure description: Failed to receive completion for <XCDeviceEvent:0x7f88c85972c0 page 12 usage 64 duration 0.01s within 30.0s'
*** First throw call stack:
(
    0   CoreFoundation                      0x0000000102369d85 __exceptionPreprocess + 165
    1   libobjc.A.dylib                     0x0000000101ddddeb objc_exception_throw + 48
    2   CoreFoundation                      0x0000000102369bea +[NSException raise:format:arguments:] + 106
    3   Foundation                          0x0000000101a27e1e -[NSAssertionHandler handleFailureInFunction:file:lineNumber:description:] + 169
    4   XCTest                              0x0000000101875bbf _XCTFailInCurrentTest + 375
    5   XCTest                              0x000000010189717c -[XCUIDevice _dispatchEventWithPage:usage:duration:] + 847
    6   XCTest                              0x00000001018c2612 XCInitializeForUITesting + 575
    7   XCTest                              0x000000010186191d -[XCTestDriver _runSuite] + 141
    8   XCTest                              0x00000001018627d1 -[XCTestDriver _checkForTestManager] + 259
    9   XCTest                              0x00000001018aca9a _XCTestMain + 628
    10  CoreFoundation                      0x000000010228f2ec __CFRUNLOOP_IS_CALLING_OUT_TO_A_BLOCK__ + 12
    11  CoreFoundation                      0x0000000102284f75 __CFRunLoopDoBlocks + 341
    12  CoreFoundation                      0x00000001022846d2 __CFRunLoopRun + 850
    13  CoreFoundation                      0x00000001022840f8 CFRunLoopRunSpecific + 488
    14  GraphicsServices                    0x0000000104424ad2 GSEventRunModal + 161
    15  UIKit                               0x000000010271ff09 UIApplicationMain + 171
    16  XCTRunner                           0x00000001017e68ad XCTRunner + 6317
    17  libdyld.dylib                       0x0000000104c4692d start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException
Build step 'Execute shell' marked build as failure
Finished: FAILURE

Could the app be failed to start because my Jenkins machine can't open it fast enough before Xcode implicitly fails the test? Or could there be a notification that is blocking the app from launching?

回答1:

What happens is that XCTest tries to send Home button press event to the device, but does not receive notification for completion for 30 seconds.

I don't think it has anything to do with a notification UIAlertView blocking the Home button from sending the testing target to the background, since in my experience such problem raises a different error message ("Failed to background test runner within ...").

I am not familiar with the Jenkins solution, but if you suspect the timeout to be too short, you can try changing it as follows (some hacky way I found).

Add the following lines to a code that will run earlier then [XCTestDriver _runSuite] (for instance, add your own __attribute__((constructor)) function and put this code inside):

extern __attribute__((weak)) void* _XCTSetEventConfirmationTimeout(NSTimeInterval);
// Change the default 30 seconds timeout.
long MY_NEW_EC_TIMEOUT = 120;
NSLog(@"Trying to call _XCTSetEventConfirmationTimeout(%ld)", MY_NEW_EC_TIMEOUT);
if (&_XCTSetEventConfirmationTimeout) {
    NSLog(@"_XCTSetEventConfirmationTimeout address found");
    _XCTSetEventConfirmationTimeout(MY_NEW_EC_TIMEOUT);
    NSLog(@"_XCTSetEventConfirmationTimeout successfully called");
} else {
    NSLog(@"_XCTSetEventConfirmationTimeout address not found");
}