How to Sync iPhone Core Data with web server, and

Question

I have been working on a method to sync core data stored in an iPhone application between multiple devices, such as an iPad or a Mac. There are not many (if any at all) sync frameworks for use with Core Data on iOS. However, I have been thinking about the following concept:

1. A change is made to the local core data store, and the change is saved. (a) If the device is online, it tries to send the changeset to the server, including the device ID of the device which sent the changeset. (b) If the changeset does not reach the server, or if the device is not online, the app will add the change set to a queue to send when it does come online.
2. The server, sitting in the cloud, merges the specific change sets it receives with its master database.
3. After a change set (or a queue of change sets) is merged on the cloud server, the server pushes all of those change sets to the other devices registered with the server using some sort of polling system. (I thought to use Apple's Push services, but apparently according to the comments this is not a workable system.)

Is there anything fancy that I need to be thinking about? I have looked at REST frameworks such as ObjectiveResource, Core Resource, and RestfulCoreData. Of course, these are all working with Ruby on Rails, which I am not tied to, but it's a place to start. The main requirements I have for my solution are:

1. Any changes should be sent in the background without pausing the main thread.
2. It should use as little bandwidth as possible.

I have thought about a number of the challenges:

1. Making sure that the object IDs for the different data stores on different devices are attached on the server. That is to say, I will have a table of object IDs and device IDs, which are tied via a reference to the object stored in the database. I will have a record (DatabaseId [unique to this table], ObjectId [unique to the item in the whole database], Datafield1, Datafield2), the ObjectId field will reference another table, AllObjects: (ObjectId, DeviceId, DeviceObjectId). Then, when the device pushes up a change set, it will pass along the device Id and the objectId from the core data object in the local data store. Then my cloud server will check against the objectId and device Id in the AllObjects table, and find the record to change in the initial table.
2. All changes should be timestamped, so that they can be merged.
3. The device will have to poll the server, without using up too much battery.
4. The local devices will also need to update anything held in memory if/when changes are received from the server.

Is there anything else I am missing here? What kinds of frameworks should I look at to make this possible?

Answer 1

I have just posted the first version of my new Core Data Cloud Syncing API, known as SynCloud. SynCloud has a lot of differences with iCloud because it allows for Multi-user sync interface. It is also different from other syncing api's because it allows for multi-table, relational data.

Please find out more at http://www.syncloudapi.com

Build with iOS 6 SDK, it is very up to date as of 9/27/2012.

Answer 2

2017

Regarding this incredibly old question.

It would be very much like asking

"I want to buy a device which is a phone that I can carry with me - but also use for many computing tasks, even browsing the WWW!"

Obviously, the answer to that one is if you've been on Mars, one of the main technologies realized on this planet recently was "smart phones", buy one.

These days, creating an OCC system from scratch would be as insane as creating an SQL database from scratch.

Obviously, for OCC, which is the base paradigm of all non-trivial apps now, you use

• Firebase
• PubNub
• Couchbase

and so on, which are quite simply, the major advance in human technology of the last few years.

Today, you would no more create OCC from scratch than you would

• write your own operating system from scratch

• write your own SQL database from scratch

• write your own font-rendering from scratch

Note that indeed, in a professional sense you can't be "an ios programmer" or "an android programmer" any more.

Who cares about knowing how to layout tables and buttons?

You're a Firebase/whatever expert, and as an incidental side issue you know how to layout buttons, etc on ios or android.

The only issue is which BAAS to use - for example, maybe PlayFab if it is game oriented, maybe PubNub if it is really message driven, maybe ably.io, maybe kinvey if you're corporate - whatever.

Answer 3

First you should rethink how many data, tables and relations you will have. In my solution I’ve implemented syncing through Dropbox files. I observe changes in main MOC and save these data to files (each row is saved as gzipped json). If there is an internet connection working, I check if there are any changes on Dropbox (Dropbox gives me delta changes), download them and merge (latest wins), and finally put changed files. Before sync I put lock file on Dropbox to prevent other clients syncing incomplete data. When downloading changes it’s safe that only partial data is downloaded (eg lost internet connection). When downloading is finished (fully or partial) it starts to load files into Core Data. When there are unresolved relations (not all files are downloaded) it stops loading files and tries to finish downloading later. Relations are stored only as GUID, so I can easly check which files to load to have full data integrity. Syncing is starting after changes to core data are made. If there are no changes, than it checks for changes on Dropbox every few minutes and on app startup. Additionaly when changes are sent to server I send a broadcast to other devices to inform them about changes, so they can sync faster. Each synced entity has GUID property (guid is used also as a filename for exchange files). I have also Sync database where I store Dropbox revision of each file (I can compare it when Dropbox delta resets it’s state). Files also contain entity name, state (deleted/not deleted), guid (same as filename), database revision (to detect data migrations or to avoid syncing with never app versions) and of course the data (if row is not deleted).

This solution is working for thousands of files and about 30 entities. Instead of Dropbox I could use key/value store as REST web service which I want to do later, but have no time for this :) For now, in my opinion, my solution is more reliable than iCloud and, which is very important, I have full control on how it’s working (mainly because it’s my own code).

Another solution is to save MOC changes as transactions - there will be much less files exchanged with server, but it’s harder to do initial load in proper order into empty core data. iCloud is working this way, and also other syncing solutions have similar approach, eg TICoreDataSync.

-- UPDATE

After a while, I migrated to Ensembles - I recommend this solution over reinventing the wheel.