CouchDB Logo

Rediscovering CouchDB

I’m currently writing the materials for my presentation on Core Data & Synchronization of data for RWDevCon 2016. One of the requirements for the demonstration app is a web service that provides a REST API to sync with. One of the requirements of the talk is that I cannot rely upon an Internet connection. Every person going through the tutorial needs to be able to bring up a local web service to following along with while coding the iOS app on their machine.

I started looking into a lot of wrappers for web apps, including Electron. The issue is I have a limited amount of time to come up with a solution that is super simple for conference attendees to install and bring up. I haven’t developed a web app for a couple years – and I feel so rusty. I do not have a trusty fallback programming language outside of Swift and Objective-C. I really feel stupid sometimes not knowing Ruby, Python or any other language as well as I feel like I should.

I ended up digging for two full nights and got super frustrated. There is just so much to learn about a new language, their web frameworks, persistence layers, and on and on. I still need to find time to write the iOS demo application – so I ended up time-boxing the search. Suddenly I had a moment of clarity. I remembered CouchDB – a document store/database that has a JSON REST API built in!

I first learned about the Apache CouchDB project from a presentation given by Jens Alfke at SecondConf in Chicago, 2011.

Attendees will get a single ZIP file with the CouchDB binary. I’ll then have a small script they can execute to seed the data into the store. Super super simple. So far I’m in love with CouchDB again. 🙂

Natural Language Search with Core Data

Here’s a super interesting write up on how to implement a natural language search using Core Data in a Mac or iOS app. Black Pixel needed to find a way to bring a more natural way for users of the Inspirato app to search for the right vacation. Pretty cool stuff here and something I was very excited to read about with my previous experience with search technologies.

Developing Inspirato’s Search Tool

Core Data Object IDs can change

I thought I knew a lot about Core Data with having used it a lot over the past years.  Today, I learned something new that I feel like I should have known for a long time.  NSManagedObjectIDs can change.  Seriously.

Identity in Core Data is really a tuple of (file aka store UUID, entity name, primary key). This can be captured in the -URIRepresentation from NSManagedObjectID. However, that requires using the same file, and not resetting the primary keys (e.g. changing the store UUID, or delete the file and creating a new one, or performing a not-light-weight migration)

If you want a different notion of identity, you can just add an UUID string as an attribute to your entity. A separate mapping table is not recommend. The 2 apps should agree to use the same UUIDs for the same identities.

– Ben

Ref: https://devforums.apple.com/message/480640#480640

Ben is one of the core developers of Core Data at Apple.  The last sentence describing ways primary keys can change blew my mind when mentioning “not-light-weight migration”.  In WordPress for iOS we sometimes have to perform heavyweight migrations – especially in the case of removing objects or combining object ancestries.  In a number of cases we use NSManagedObjectID’s URIRepresentation and throw it in NSUserDefaults for retrieval later.  Turns out, that ID isn’t stable enough between migrations.

The solution, as Ben points out, is to create a UUID and make the ID yourself upon initial persistence.

Mind blown.

Asynchronous unit testing Core Data with Xcode 6

The WordPress for iOS project had a number of unit tests using Core Data and a custom asynchronous test helper.  The helper used a semaphore in a global scope and a bit of method swizzling to give a wait/notify mechanism.  The problem with this solution was the global semaphore and poorly written tests causing a conflict.  Tests would call the ending wait and previous tests running Core Data would fire off notifies causing a mismatch between the original test and the recipient of the message to pass by the current semaphore.

The solution was to eliminate the global scope and have a semaphore object/instance that could be passed around by the single test.  The more I started designing the solution I realized we should just start using the new asynchronous testing capabilities of Xcode 6.  Xcode 6 / iOS 8 / Yosemite brings us a wealth of new tools to play with including Swift, the newest language developed by Apple.

XCTestExpectation

The newest member to the XCTest framework is XCTestExpectation.  This class encapsulates a single semaphore (of sorts) allowing your test to wait for 1..n things before the test is considered over.  This helps with asynchronous things like AFNetworking, Core Data, and Grand Central Dispatch.  If you’re using a Core Data stack with multiple NSManagedObjectContext instances (background & main for example) then you’re going to almost certainly want to know how to test asychronously.

Creating a XCTestExpectation is quite simple:

- (void)testObjectPermanence {
    XCTestExpectation *saveExpectation = [self expectationWithDescription:@"Context save expectation"];
    // ..
}

The method expectationWithDescription: is part of XCTestCase and it has a companion method waitForExpectationsWithTimeout:handler: which you call when you’re reading to block the test execution until [saveExpectation fulfill] is called.  Fulfilling an expectation means the expectation has been met and the flag should be flipped for the test that is waiting.  If your test execution never reaches a fulfillment, then your test will fail when your timeout threshold is met.

Some important things to note about XCTest asynchronous testing:

  • A XCTestExpectation instance can only be fulfilled a single time.  Use multiple instances if you have more than one expectation.
  • Only a single waitForExpectationsWithTimeout:handler: can be active at any one time.
  • Unit tests, by default, run one at a time and not in parallel.  If you modify the test suite running to parallelize, I can see things breaking fairly quickly.
  • There are convenience methods to create expectations on a KVO property or NSNotification

Core Data Secret Sauce

There are a bunch of waits you could fire off a fulfillment on an XCTestExpectation when a Core Data context saves.  In the case of WordPress-iOS we use a central class, ContextManager, to help manage all of the Core Data levers and knobs.  There is a companion class called CoreDataTestHelper that does a bunch of useful things like overriding our persistent store to be in memory rather than on disk and also swizzle a method on ContextManager to fire fulfillments if an expectation is recorded.  Here’s an example of a test using this expectation helper:

- (void)testObjectPermanence {
    XCTestExpectation *saveExpectation = [self expectationWithDescription:@"Context save expectation"];
    [CoreDataTestHelper sharedHelper].testExpectation = saveExpectation;

    NSManagedObjectContext *derivedContext = [[ContextManager sharedInstance] newDerivedContext];
    Blog *blog = [self createTestBlogWithContext:derivedContext];
    [[ContextManager sharedInstance] saveDerivedContext:derivedContext];

    // Wait on the merge to be completed
    [self waitForExpectationsWithTimeout:2.0 handler:nil];

    XCTAssertFalse(blog.objectID.isTemporaryID, @"Object ID should be permanent");
}

You can see we store the single expectation for the next NSManagedObjectContext save on the CoreDataTestHelper sharedHelper instance. Later on when the context is saved, the following swizzled code executes and fulfills the expectation:

+ (void)load {
    Method originalSaveContext = class_getInstanceMethod([ContextManager class], @selector(saveContext:));
    Method testSaveContext = class_getInstanceMethod([ContextManager class], @selector(testSaveContext:));
    method_exchangeImplementations(originalSaveContext, testSaveContext);
}

- (void)testSaveContext:(NSManagedObjectContext *)context {
	[self saveContext:context withCompletionBlock:^() {
        if ([CoreDataTestHelper sharedHelper].testExpectation) {
            [[CoreDataTestHelper sharedHelper].testExpectation fulfill];
            [CoreDataTestHelper sharedHelper].testExpectation = nil;
        } else {
            NSLog(@"No test expectation present for context save");
        }
    }];
}

Next Steps

Our asynchronous tests are definitely not finished and the Core Data testing stack will most likely continue to change.  Our service layer doesn’t enforce a single save paradigm – we tell contributors that they should probably save often and at least one time before the service method completes.  The problem with the design we’re using above is that there is no good way to let a number of saves happen and to wait until the very last save.  In order do that, we’d have to offer up a special saveContext method on ContextManager that could get an instance of an XCTestExpectation.  That method would exist only as a class extension in our CoreDataTestHelper.  It’s messy but then again so is Core Data at times.  🙂

Pull Request

The changes I made to the WordPress for iOS project are up for review in this GitHub Pull Request.

On the Importance of Glue

Glue is the thing that … well … glues everything together.  You don’t see glue (if the product is made right) but it plays an important role in the overall satisfaction with the product.  If you buy a bird house that has bad gluing technique or not enough glue, you will be upset when the first bird flattens the house.

The same hold trues for mobile apps as well as web and desktop applications.  For sake of this discussion, I’m limiting myself to mobile apps and mentioning some specific iOS technologies.

So much time is seemingly spent on visual design and user experience that occasionally the glue doesn’t receive the attention it should.  Glue is everything that the user doesn’t see outright in your app:

  • Persistence / State Restoration
    • Core Data
    • Data Synchronization (iCloud, Dropbox, Simperium, etc.)
    • UIStateRestoration
  • Performance
    • Power
    • CPU
    • Memory
  • Integration & Networking
  • Internationalization
  • Dates & Times
  • Accessibility
  • Unit Testing

It’s easy to get hung up on the visual details and leave some of these incredibly important things to the side.  Users may not see the problems outright but they will come to associate a bad feel with your app if things don’t work quite right.

Example: The Nike+ Fuelband App

I’ve been an avid Fuelband user for almost two years now.  I’ve come to treat the device as something that I use every day and pay attention to how much activity I’ve done for the day.  Getting to green is my goal and it irritates me when things don’t work.  Some of the glue that has been broken lately includes funky sync over Bluetooth LE and bad dates/times when syncing with their web app.  The web app itself has also been flaky and slow.  This is a good example of when a beautifully designed app fails the user and causes distrust of the entire brand.

The Recourse

Admit your mistakes and start fixing them.  The WordPress iOS app has been around for quite some time but had been unstable for a while.  The nature of open source software means there are so many hands touching it.  We started focusing on the glue to make the experience better overall.  Once we got some of the stability back into the project, we moved forward with the UI changes for iOS 7.  Things are improving with every release – especially when we are able to spend the time to replace aging code.  Core Data has a big role in the application and stabilizing the stack there with multithreading has made a huge improvement.  We have big plans for further UI improvements!  Get involved at http://make.wordpress.org/mobile if you’re interested. 🙂

The Balance

A balance has to be maintained between visual and the non-visual elements of an application.  If you’re in charge of running a mobile project, make sure the people paying the bill know upfront what’s important to keep in mind when developing an application.  If they don’t want to pay for what you’re estimating, don’t take the project.  This may be a little hard at an ad agency or working with an internal department, but the outcome is the same.  Make sure they know if all they want to focus on is the user experience and visual design, the output will be of prototype quality.  It’s your job to educate your customer and product owner of the importance of the things that can’t be seen.

 

CocoaConf Chicago – Advanced Core Data

Aaron Douglas - Advanced Core Data

I was lucky enough to be able to speak at CocoaConf Chicago 2014 about some more advanced Core Data topics.  The bulk of the talk surrounded concurrency and data model migrations but I did touch on a number of other things.  Sadly the session wasn’t recorded, but I am considering recording a screencast if there is enough interest.