Synchronize IOS core data with web service?

core-data, ios5, web-services

Solution

My solution to this was to store two copies of the data set in a CoreData database. One represents the last-known server state and is immutable. The other is edited by the user.

When it is time to sync changes, the app creates a diff between the edited and immutable copies of the data. The app sends the diff to a web service which applies the diff to its own copy of the data. It replies with a full copy of the data set, which the app overwrites onto both of its copies of the data.

The advantages are:

- If there is no network connectivity, no changes are lost: the diff is calculated each time the data set needs to be sent, and the immutable copy is only changed on a successful sync.

- Only the minimum amount of information that needs to be sent is transmitted.

- Multiple people can edit the same data at the same time without using locking strategies with a minimum opportunity for data loss via overwrites.

The disadvantages are:

- Writing the diffing code is complex.

- Writing the merging service is complex.

- Unless you are a metaprogramming guru, you'll find that your diff/merge code is brittle and has to change whenever you change your object model.

Here are some of the considerations I had when coming up with the strategy:

- If you allow changes to be made offline, checkin/checkout locking won't work (how can you establish a lock with no connection?).

- What happens if two people edit the same data at the same time?

- What happens if one person edits data on one iOS device when connectionless, switches it off, edits on another device and then turns the original device back on?

- Multithreading with CoreData is an entire problem class in itself.

The closest thing I've heard of to out-of-the-box support to do anything remotely like this is the new iCloud/CoreData syncing system in iOS6, which automatically transmits entities from a CoreData database to iCloud when they change. However, that means you have to use iCloud.

EDIT: This is very late, I know, but here's a class that is capable of producing a diff between two NSManagedObject instances.

// SZManagedObjectDiff.h
@interface SZManagedObjectDiff

- (NSDictionary *)diffNewObject:(NSManagedObject *)newObject withOldObject:(NSManagedObject *)oldObject

@end

// SZManagedObjectDiff.m
#import "SZManagedObjectDiff.h"

@implementation SZManagedObjectDiff

- (NSDictionary *)diffNewObject:(NSManagedObject *)newObject withOldObject:(NSManagedObject *)oldObject {

    NSDictionary *attributeDiff = [self diffAttributesOfNewObject:newObject withOldObject:oldObject];

    NSDictionary *relationshipsDiff = [self diffRelationshipsOfNewObject:newObject withOldObject:oldObject];

    NSMutableDictionary *diff = [NSMutableDictionary dictionary];

    if (attributeDiff.count > 0) {
        diff[@"attributes"] = attributeDiff;
    }

    if (relationshipsDiff.count > 0) {
        diff[@"relationships"] = relationshipsDiff;
    }

    if (diff.count > 0) {
        diff[@"entityName"] = newObject ? newObject.entity.name : oldObject.entity.name;

        NSString *idAttributeName = newObject ? newObject.entity.userInfo[@"id"] : oldObject.entity.userInfo[@"id"];

        if (idAttributeName) {
            id itemId = newObject ? [newObject valueForKey:idAttributeName] : [oldObject valueForKey:idAttributeName];

            if (itemId) {
                diff[idAttributeName] = itemId;
            }
        }
    }

    return diff;
}

- (NSDictionary *)diffRelationshipsOfNewObject:(NSManagedObject *)newObject withOldObject:(NSManagedObject *)oldObject {

    NSMutableDictionary *diff = [NSMutableDictionary dictionary];

    NSDictionary *relationships = newObject == nil ? [[oldObject entity] relationshipsByName] : [[newObject entity] relationshipsByName];

    for (NSString *name in relationships) {

        NSRelationshipDescription *relationship = relationships[name];

        if (relationship.deleteRule != NSCascadeDeleteRule) continue;

        SEL selector = NSSelectorFromString(name);

        id newValue = nil;
        id oldValue = nil;

        if (newObject != nil && [newObject respondsToSelector:selector]) newValue = [newObject performSelector:selector];
        if (oldObject != nil && [oldObject respondsToSelector:selector]) oldValue = [oldObject performSelector:selector];

        if (relationship.isToMany) {

            NSArray *changes = [self diffNewSet:newValue withOldSet:oldValue];

            if (changes.count > 0) {
                diff[name] = changes;
            }

        } else {

            NSDictionary *relationshipDiff = [self diffNewObject:newValue withOldObject:oldValue];

            if (relationshipDiff.count > 0) {
                diff[name] = relationshipDiff;
            }
        }
    }

    return diff;
}

- (NSDictionary *)diffAttributesOfNewObject:(NSManagedObject *)newObject withOldObject:(NSManagedObject *)oldObject {

    NSMutableDictionary *diff = [NSMutableDictionary dictionary];

    NSArray *attributeNames = newObject == nil ? [[[oldObject entity] attributesByName] allKeys] : [[[newObject entity] attributesByName] allKeys];

    for (NSString *name in attributeNames) {

        SEL selector = NSSelectorFromString(name);

        id newValue = nil;
        id oldValue = nil;

        if (newObject != nil && [newObject respondsToSelector:selector]) newValue = [newObject performSelector:selector];
        if (oldObject != nil && [oldObject respondsToSelector:selector]) oldValue = [oldObject performSelector:selector];

        newValue = newValue ? newValue : [NSNull null];
        oldValue = oldValue ? oldValue : [NSNull null];

        if (![newValue isEqual:oldValue]) {
            diff[name] = @{ @"new": newValue, @"old": oldValue };
        }
    }

    return diff;
}

- (NSArray *)diffNewSet:(NSSet *)newSet withOldSet:(NSSet *)oldSet {

    NSMutableArray *changes = [NSMutableArray array];

    // Find all items that have been newly created or updated.
    for (NSManagedObject *newItem in newSet) {

        NSString *idAttributeName = newItem.entity.userInfo[@"id"];

        NSAssert(idAttributeName, @"Entities must have an id property set in their user info.");

        id newItemId = [newItem valueForKey:idAttributeName];

        NSManagedObject *oldItem = nil;

        for (NSManagedObject *setItem in oldSet) {
            id setItemId = [setItem valueForKey:idAttributeName];

            if ([setItemId isEqual:newItemId]) {
                oldItem = setItem;
                break;
            }
        }

        NSDictionary *diff = [self diffNewObject:newItem withOldObject:oldItem];

        if (diff.count > 0) {
            [changes addObject:diff];
        }
    }

    // Find all items that have been deleted.
    for (NSManagedObject *oldItem in oldSet) {

        NSString *idAttributeName = oldItem.entity.userInfo[@"id"];

        NSAssert(idAttributeName, @"Entities must have an id property set in their user info.");

        id oldItemId = [oldItem valueForKey:idAttributeName];

        NSManagedObject *newItem = nil;

        for (NSManagedObject *setItem in newSet) {
            id setItemId = [setItem valueForKey:idAttributeName];

            if ([setItemId isEqual:oldItemId]) {
                newItem = setItem;
                break;
            }
        }

        if (!newItem) {
            NSDictionary *diff = [self diffNewObject:newItem withOldObject:oldItem];

            if (diff.count > 0) {
                [changes addObject:diff];
            }
        }
    }

    return changes;
}

@end

There's more information about what it does, how it does it and its limitations/assumptions here:

http://simianzombie.com/?p=2379

Problem

Original source