Read and Cache Real-time Data with Firebase Firestore

Photo by Joshua Hoehne on Unsplash

Firestore is a NoSQL Document Database for storing you app’s data. If you are new to Firestore, you can read about it here. The pricing for database is done based on number of documents reads. Which means, if you can reduce the repetitive reads. You can save some money and app will load faster because we read less data.

Firestore already has offline caching support but we don’t have much control over it.

As the saying goes. There is no 100% right or 100% wrong way of doing something. This might not be the best way to cache and read data from firestore. But it does help.

From now on we’re going to solve this problem keeping in mind one of the hard computer science problem.

Cache invalidation

So, that out of the way. Let’s get started.

Use case

Let’s look at simple use case where this actually can be useful. Let’s imagine we have a collection with 100 documents. Every time we read the collection we read all the 100 documents. Even-though the documents have not changed.

We can avoid the documents reads by reading only documents that have changed.

Here is how

First, we maintain a property in every document called updatedAt. Which will tells us the document's updated time in milliseconds.

On the client-side. While query the data, we’ll add a where condition to only query documents which have updatedAt value greater than maxUpdatedAt value.

Where maxUpdatedAt is max of all the updatedAt values present in local cache on the client-side. If there is no cache, so we can pass 0 as maxUpdatedAt.

Example

Let’s say we’ve have 3 documents in collection call note on the server.

{
  "1qtBx0ikpxNi6fxIqMX1": {
    "text": "Meditate",
    "updatedAt": 1585486383381
  },
  "479tvFCMH5BL2xsKf90r": {
    "text": "Be mindful",
    "updatedAt": 1585486383381
  },
  "6eC2FtZrGAvF3AGdqcM5": {
    "text": "Stay present",
    "updatedAt": 1585486383381
  }
}

On the client-side we can write our query like this.

firestore()
  .collection('note')
  .where('updatedAt', '>', maxUpdatedAt)
  // add real-time listener or get the data

After the read will can upsert our local cache with the changes and keep listening for new changes as well.

Updating the updatedAt

For this method to work. We must make sure that we update the updatedAt property of the document. Whenever the document has changed.

We can do these by manually passing the updatedAt to be firestore.Timestamp.now().toMillis()

Or we can write Cloud Function Trigger to listen to document changes. And update the updatedAt property of the document.

exports.onNoteUpdated = functions.firestore
  .document(“note/{noteId}”)
  .onUpdate(change => {
    if (change.before.data().updatedAt === change.after.data().updatedAt) {
      return Promise.reject()
    }
    return change.after.ref.update({
      updatedAt: firestore.Timestamp.now().toMillis()
    })
});

If you are not sure why we’re doing the before and after updatedAt check. It’s because whenever the document changes, this code is ran. In the code we are changing the document’s updatedAt value. That means this code will run again and again until you shut it down, by that time you bill will be skyrocketed.

So, the check makes sure that we don’t update the updatedAt if before and after updatedAt are same values.

Handling deletes

So far so good. Let’s come to the hard part. Cache invalidation. How in the world will client-side know that a document was deleted and should be remove from the cache.

To solve this problem we can use soft deletes. Instead of actually deleting the document. We add a property called deletedAt to the document to tell that a particular document is deleted. Like below.

{
  "1qtBx0ikpxNi6fxIqMX1": {
    "text": "Meditate",
    "updatedAt": 1585491466274,
    "deletedAt": 1585491466274
  },
  ...
}

We must make sure to update the updatedAt as well. So the client-side can read the document change and remove it from cache if deletedAt is set.

I hope you learnt something new with this.

Thanks for reading.