Firebase-ios-sdk: FR: Local only documents in Firestore

[paulharter]

Feature proposal

• Firebase Component: Firestore

It would be very useful to be able to have some documents that are not synced to the cloud but can be persisted locally and will not be cleared from local cache. Maybe a flag set when a doc is created.

[wilhuff]

Hi @paulharter, thanks for writing in!

This is something we've discussed before, though historically we've gotten stuck on how to specify the configuration for this, and whether or not that configuration should be durable. We've primarily thought about this in terms of specifying that certain collections would be exempt from synchronization, rather than specific documents, but I can see the appeal of making this something you do on a per-write basis. What do you think of specifying local-only collection-wide?

In the meantime note that you can partially simulate this already. The way to do it is:

• Create a secondary Firebase App with a different name than the default
• Create a secondary Firestore instance associated with that app.
• Call disableNetwork on that Firestore instance as the first thing you do with it.

I'd consider this viable only in limited circumstances, and in every circumstance where I can see this being useful, having a per-user set of documents that are synced works just as well and is safer. The reason for this reservation is that the SDK will still track all the pending writes for this secondary instance and this queue will grow without bound. For a small few documents that are updated rarely this would work though.

[paulharter]

@wilhuff

Yes - having a collection specific behaviour for this sounds like a good plan.

I had two very separate use cases I was thinking about and they probably actually have different solutions.

The first, and more important, is simply to have some kinds of documents that will never sync to the cloud but can be persisted locally. At the moment I'm using Core Data in tandem with Firestore in the same app to do this. It would be nice to only use one db for both. Your suggestion of local only collections would work perfectly for this.

The other use case is quite different. I am writing a collaborative video editing app which syncs quite a lot of docs to the cloud. I was thinking that it would be nice to keep the same application but disable all sync, so I could offer an offline only version as a free/demo. The disableNetwork approach would work better for this if there wasn't the write queue issue. Would it be possible to have a DB that will never sync at all that could then be deleted and replaced with a sync one?

Slightly separately I was also wondering about managing locally cached docs. Is it possible to flush a set of docs out of the cache, maybe with a predicate?

Thanks

Paul

[wilhuff]

@wilhuff

Yes - having a collection specific behaviour for this sounds like a good plan.

I had two very separate use cases I was thinking about and they probably actually have different solutions.

The first, and more important, is simply to have some kinds of documents that will never sync to the cloud but can be persisted locally. At the moment I'm using Core Data in tandem with Firestore in the same app to do this. It would be nice to only use one db for both. Your suggestion of local only collections would work perfectly for this.

Glad to hear it. I can't promise when we'd work on this, but the feature is definitely on our backlog.

The other use case is quite different. I am writing a collaborative video editing app which syncs quite a lot of docs to the cloud. I was thinking that it would be nice to keep the same application but disable all sync, so I could offer an offline only version as a free/demo. The disableNetwork approach would work better for this if there wasn't the write queue issue. Would it be possible to have a DB that will never sync at all that could then be deleted and replaced with a sync one?

You can already delete the local data associated with an instance using the clearPersistence(completion:) call. If the instance was already running you'll need to call terminate(completion:) first to shut it down. So, if we added support for running an instance completely offline, replacing the instance as you described would be possible.

The more interesting question is whether or not a completely offline instance (or collection) could be upgraded. Auto-uploading everything in a previously offline-only instance seems fraught with peril, especially once we add support for pre-loaded data bundles, so this would require some care. If we did nothing to explicitly support this, the result is safe but somewhat clunky (to upload you'd have to read the local documents you want to upload and then write them).

Slightly separately I was also wondering about managing locally cached docs. Is it possible to flush a set of docs out of the cache, maybe with a predicate?

This isn't possible today. The cache is currently bounded with an approximate maximum size and documents are deleted based on the least recently used query or write that has touched them.

One approach to having differing cache policies is (again) to use multiple Firestore instances, with different maximum cache sizes or even with persistence disabled. This would give you the ability to age documents out differently, but it would still be the same size-based LRU mechanism doing the removal. We don't have plans to refine this, though it does seem like adding a primitive that explicitly removed a locally cached document would make this possible.