Part 7: Managing the Share and Background Maintenance

This is the seventh in an eight-part series on implementing data sharing in Shopping UK using CloudKit.

Shopping UK is a smart shopping list for UK shoppers. It knows almost every product in the supermarket and will arrange them by aisle. Lists can be shared with family or friends.

Last week, we looked at how data is synchronised between devices and some of the challenges to consider. Today, we’ll look how to change or stop a share, what happens when a list is deleted and background maintenance activities.

Managing the Share as the Owner

The “owner” of the list is the person that originally shared it.

You’ll remember from Part 4 how a UICloudSharingController is used to create the initial share. Well, the same controller is also used for viewing and managing the share after it has been created.

As an owner of the list, you can:

• View the list of participants.
• Invite new participants.
• Remove participants.
• Stop sharing the list.

Inviting new participants

Only the owner of the list can invite new participants.

You can invite up to 100 people. This limit is imposed by CloudKit, but it is more than adequate for our purposes. The UICloudSharingController has options for controlling who can accept an invite (anyone with the link or named accounts), and their access permissions (read-write or read-only).

Lists in Shopping UK can only be shared with named accounts and read-write permissions — the user doesn’t get to choose. This may change in the future, but I honestly couldn’t think of a use case for allowing a read-only shopping list or anonymous access at present (perhaps you have one?).

The app doesn’t need to do anything special when a user adds a new participant. The UICloudSharingController takes care of it all:

• Generating and sending invites
• Updating the cloudkit.share record with the new participant’s details
• Updating the cloudkit.share acceptance status

When the cloudkit.share status changes to accepted, the CKRecordZone for the shared list will automatically appear in the participant’s shared CKDatabase.

Removing participants

Only the owner of the list can remove a participant, but anyone can remove themselves.

As an owner, you remove a participant using the “Remove Access” button:

Note: If there is only one participant in the list, the list will stop sharing.

The app doesn’t need to do anything special to support this action. The UICloudSharingController takes care of everything — i.e. updating the cloudkit.share record to remove the participant’s details, which will hide the list’s CKRecordZone from the participant’s shared CKDatabase.

The removed participant will see this message before the list is deleted from their device:

This is handled by subscriptions and the CKFetchDatabaseChangesOperation described in Part 3 and Part 6. The fetch operation will return a Deleted Record Zone response and the app responds by showing this message.

Note: The list data is still in iCloud, in the owner’s Private database, and on the other participants’ devices.

Stop Sharing

The “Stop Sharing” button will remove sharing for all participants and remove the data from iCloud.

When this option is used, the app needs to do some housekeeping of its own to reset the local list to be a local-only list as it was before the list was first shared.

The app must do three things:

1. Remove the locally stored CKServerChangeTokens
2. Delete the CKRecordZone that contains the list data from iCloud
3. Remove the local reference to the iCloud list.

The app is notified of the “Stop Sharing” action by implementing the cloudSharingControllerDidStopSharing method in the UICloudSharingControllerDelegate

The zone is deleted using deleteRecordZoneWithID on the CKDatabase

When the zone is deleted, the participants will each receive a message like the one seen when a single participant is removed.

Managing the Share as a Participant

As a participant, the UICloudSharingController shows a different view. There are no options for inviting others, stopping the share or removing other participants. The only option is to remove yourself from the list:

The UICloudSharingController takes care of the mechanics of this by updating the cloudkit.share record.

What happens when a list is deleted

A shared list can be deleted by either the owner or a participant. The behaviour is different for each.

Deleting a list as an owner

When the owner deletes the list, everything is removed:

1. The list data from the owner’s device.
2. The list’s CKRecordZone from the owner’s private CKDatabase in iCloud.
3. The list’s CKRecordZone from all participants’ shared CKDatabase in iCloud.
4. The list data from the devices of all participants.

The owner’s app handles tasks 1 and 2, using the “Stop Sharing” logic to remove the zone.

Task 3 happens automatically because when the owner’s private zone is removed, the shared zones disappear too.

The participant’s app handles task 4, the same way as for “Stop Sharing”.

Deleting a list as a participant

When a participant deletes a list, things are simpler, because the original list is not deleted — it remains in iCloud, and on the owner’s device, and on the devices of other participants.

Only two things need to be removed:

1. The list data from the participant’s device.
2. The cloudkit.share CKRecord from the participant’s shared CKDatabase in iCloud.

The app handles both tasks itself. The cloudkit.share is deleted like any other CKRecord, using deleteRecordWithID, which provides a simple wrapper around CKModifyRecordsOperation.

Note: after deleting the list, a participant can re-join later using the same invitation they originally received.

What happens when a user signs into or out of iCloud?

It is important to understand that when a list is first shared, the shopping list is no longer considered to be a locally hosted list that lives on the device. It is now an iCloud-hosted list and the data on the local device is just a locally cached copy.

This means when a user signs out of iCloud, all iCloud-hosted lists must be deleted from the local device. If the app didn’t do this, it would have no way to guarantee their integrity because the user no longer has access to the iCloud database that contains list changes. Furthermore, permitting continued access to the list would violate the list’s security because the user is logged out of iCloud and no longer authorised to view the list’s contents.

When a user signs-in to iCloud, the opposite happens. The app will restore a copy of all iCloud-hosted lists from the Private and Shared databases of the newly available iCloud account.

The app detects a change of account status by observing the CkAccountChanged notification.

When this notification is triggered, the app sends an accountStatus message to CloudKit to check the account status.

If the response indicates a status change from NoAccount to Available, all iCloud-hosted lists are restored to the device.

If the response indicates a status change from Available to NoAccount, all iCloud-hosted lists are removed from the device.

The account change notifications have been reliable so far. But I like to have a manual option if things go wrong. That’s why I added an option to force a re-fetch of all iCloud-hosted lists in the “Troubleshooting” section of the Help menu.

Without this option, the user would have to sign-out then sign back into iCloud to refresh, and this can be time consuming.

Background Maintenance

Once a share is established, the user need not do anything. The app will happily synchronise changes all day long.

But, behind the scenes, nothing stands still.

If you’ve been following the previous posts, you’ll recall how Shopping UK doesn’t synchronise the state of the list. Instead, it synchronises the Journal Entry records that represent the history of changes made to the list (see Part 2 and Part 6 for a refresher). And from these changes, the current list can be built.

But, you’re probably wondering what will happen after a few months, when there are thousands of JournalEntry records?

Firstly, for devices already sharing the list, it doesn’t matter how many JournalEntry records there are because, as we saw in Part 3 and Part 6, the app uses CKServerChangeTokens to only fetch new records.

But this doesn’t let us off the hook entirely.

What would happen if the journal keeps growing?

Two problems:

1. Storage and Quotas.
2. New Participants joining the list.

Every CKRecord takes up space (albeit just a few bytes), and this counts towards the share owner’s iCloud storage quota. If the number of JournalEntry records were allowed to grow indefinitely, the owner’s iCloud storage would fill up.

Secondly, if a new participant joins the list later, the size of the journal becomes relevant. Remember how, in Part 5, a CKServerChangeToken is not used when retrieving the list initially. With a large journal, it could take a considerable time to fetch data from iCloud, and to apply the changes to the view in the user interface.

Neither of these situations is good.

What’s the answer?

The approach I chose for Shopping UK was to compress the journal by removing old JournalEntry records that aren’t necessary for reconstructing the current list.

For example, the two lists below are identical but the second was built using fewer JournalEntry records:

Admittedly, by removing rows 1 and 3, we have lost some of the list’s history, but the final state of both lists is the same.

Let’s look at another example. Are these two lists equivalent?

No, not quite. The first list includes “bread” in a marked-off state, but the second doesn’t show “bread” at all.

Does this matter? Maybe not.

If “bread” was bought a long time ago, does it need to be on the list? After all, the list’s purpose is to tell us what we need to buy not what was bought.

Then again, if my wife has just bought “bread”, I’d find it useful to see it crossed-off the list, so I know it was bought and not just deleted.

From these examples, we can infer some general principles:

1. Recent items should be left alone, and
2. The full history of changes should be preserved in an Activity view.

And this is how journal compression works in Shopping UK.

Compression is never applied to the full set of JournalEntry changes, only to those created before a cut-off time.

And every change made to the list is also recorded in an Activity view that won’t be affected when the journal is compressed.

When are JournalEntry records compressed?

Compression occurs in two places:

1. Before updating the user interface
2. In iCloud, at regular intervals

Before updating the user interface

When a device requests new changes from iCloud, many JournalEntry records may be returned. Updating the user interface rapidly with so many changes can make interaction sluggish and distract the user. To prevent this, the app will compress JournalEntry records older than three hours and only apply the remaining changes to the user interface.

Practically, this means if a user on another device added and marked-off “milk” more than 3 hours ago it won’t be shown in the list (because the “add” and the “mark-off” both happened more than 3 hours ago). But if the user added and marked-off “milk” within 3 hours the item would be shown in the list like this in a marked-off state, like this: milk

In iCloud

The journal in iCloud will also be regularly compressed.

Every device that has access to a shared list is responsible for compressing the JournalEntry records in iCloud. After each iCloud upload (see Part 6), the app will check if the journal can be compressed. But, to save battery and bandwidth, this won’t happen more often than every 30 minutes.

Compression is a three-stage process:

1. Fetch eligible JournalEntry records from iCloud.
2. Attempt to compress them on the app.
3. Delete the discarded JournalEntry records from iCloud and update the CompressedUntil date on the ShoppingList record

JournalEntry records older than 14 days are considered eligible for compression.

The app uses a CKQueryOperation to fetch the eligible JournalEntry records.

If the compression logic identifies records to be deleted, a CKModifyRecordsOperation is used to remove the records from iCloud and update the CompressedUntil date.

The default SavePolicy of IfServerRecordUnchanged is used, which means the entire operation will succeed or fail atomically.

Consequences of Compression

Journal compression is a housekeeping activity, and users should mostly be oblivious to its existence. The only time the effect of compression will be surfaced to users is if their device has not been used for more than 14 days. If this happens, the app will perform a full re-fetch of the shared lists from CloudKit to ensure they are fully up-to-date.

Incidentally, if you used the app before version 3.3 was released, you may have experienced an issue that caused me no end of stress trying to reproduce and diagnose. The old, home-grown sharing mechanism used a primitive form of journalling, and didn’t have a way to detect if a device hadn’t been updated recently. Frustrated users would send me messages asking why the sync stopped working after the app hadn’t been used in a while. If you were one of the users affected by this issue, I’m truly sorry it took me so long to find and fix.

Alternatives to compression

Journal compression is a convenient way of reducing the size of the app’s write-ahead journal. It works well for a shopping list because the state of items on a list follows a predictable pattern: they are Added then Marked-Off or Deleted. And each pair of start-end states (Add then Mark-Off) and (Add then Delete) can be neatly purged without affecting other items. For other types of app, a more appropriate clean-up strategy for a write-ahead journal may involve a low-water mark of some sort.

Next week, in the final part of the series, we’ll look at what happens when things go wrong: error handling, merging data, and diagnosing problems.

---

If you get a chance, please try Shopping UK and let me know what you think at @wheelies

---