The WordPress for iOS project had a number of unit tests using Core Data and a custom asynchronous test helper.  The helper used a semaphore in a global scope and a bit of method swizzling to give a wait/notify mechanism.  The problem with this solution was the global semaphore and poorly written tests causing a conflict.  Tests would call the ending wait and previous tests running Core Data would fire off notifies causing a mismatch between the original test and the recipient of the message to pass by the current semaphore.

The solution was to eliminate the global scope and have a semaphore object/instance that could be passed around by the single test.  The more I started designing the solution I realized we should just start using the new asynchronous testing capabilities of Xcode 6.  Xcode 6 / iOS 8 / Yosemite brings us a wealth of new tools to play with including Swift, the newest language developed by Apple.

XCTestExpectation

The newest member to the XCTest framework is XCTestExpectation.  This class encapsulates a single semaphore (of sorts) allowing your test to wait for 1..n things before the test is considered over.  This helps with asynchronous things like AFNetworking, Core Data, and Grand Central Dispatch.  If you’re using a Core Data stack with multiple NSManagedObjectContext instances (background & main for example) then you’re going to almost certainly want to know how to test asychronously.

Creating a XCTestExpectation is quite simple:

[code language=”objc”]
– (void)testObjectPermanence {
XCTestExpectation *saveExpectation = [self expectationWithDescription:@"Context save expectation"];
// ..
}
[/code]

The method expectationWithDescription: is part of XCTestCase and it has a companion method waitForExpectationsWithTimeout:handler: which you call when you’re reading to block the test execution until [saveExpectation fulfill] is called.  Fulfilling an expectation means the expectation has been met and the flag should be flipped for the test that is waiting.  If your test execution never reaches a fulfillment, then your test will fail when your timeout threshold is met.

Some important things to note about XCTest asynchronous testing:

  • A XCTestExpectation instance can only be fulfilled a single time.  Use multiple instances if you have more than one expectation.
  • Only a single waitForExpectationsWithTimeout:handler: can be active at any one time.
  • Unit tests, by default, run one at a time and not in parallel.  If you modify the test suite running to parallelize, I can see things breaking fairly quickly.
  • There are convenience methods to create expectations on a KVO property or NSNotification

Core Data Secret Sauce

There are a bunch of waits you could fire off a fulfillment on an XCTestExpectation when a Core Data context saves.  In the case of WordPress-iOS we use a central class, ContextManager, to help manage all of the Core Data levers and knobs.  There is a companion class called CoreDataTestHelper that does a bunch of useful things like overriding our persistent store to be in memory rather than on disk and also swizzle a method on ContextManager to fire fulfillments if an expectation is recorded.  Here’s an example of a test using this expectation helper:

[code language=”objc”]
– (void)testObjectPermanence {
XCTestExpectation *saveExpectation = [self expectationWithDescription:@"Context save expectation"];
[CoreDataTestHelper sharedHelper].testExpectation = saveExpectation;

NSManagedObjectContext *derivedContext = [[ContextManager sharedInstance] newDerivedContext];
Blog *blog = [self createTestBlogWithContext:derivedContext];
[[ContextManager sharedInstance] saveDerivedContext:derivedContext];

// Wait on the merge to be completed
[self waitForExpectationsWithTimeout:2.0 handler:nil];

XCTAssertFalse(blog.objectID.isTemporaryID, @"Object ID should be permanent");
}
[/code]

You can see we store the single expectation for the next NSManagedObjectContext save on the CoreDataTestHelper sharedHelper instance. Later on when the context is saved, the following swizzled code executes and fulfills the expectation:

[code language=”objc”]
+ (void)load {
Method originalSaveContext = class_getInstanceMethod([ContextManager class], @selector(saveContext:));
Method testSaveContext = class_getInstanceMethod([ContextManager class], @selector(testSaveContext:));
method_exchangeImplementations(originalSaveContext, testSaveContext);
}

– (void)testSaveContext:(NSManagedObjectContext *)context {
[self saveContext:context withCompletionBlock:^() {
if ([CoreDataTestHelper sharedHelper].testExpectation) {
[[CoreDataTestHelper sharedHelper].testExpectation fulfill];
[CoreDataTestHelper sharedHelper].testExpectation = nil;
} else {
NSLog(@"No test expectation present for context save");
}
}];
}
[/code]

Next Steps

Our asynchronous tests are definitely not finished and the Core Data testing stack will most likely continue to change.  Our service layer doesn’t enforce a single save paradigm – we tell contributors that they should probably save often and at least one time before the service method completes.  The problem with the design we’re using above is that there is no good way to let a number of saves happen and to wait until the very last save.  In order do that, we’d have to offer up a special saveContext method on ContextManager that could get an instance of an XCTestExpectation.  That method would exist only as a class extension in our CoreDataTestHelper.  It’s messy but then again so is Core Data at times.  🙂

Pull Request

The changes I made to the WordPress for iOS project are up for review in this GitHub Pull Request.