An Introduction to Firebase’s Cloud Firestore: Part II

by Reese Williams

TL;DR

• Firestore's indexing rules aim to protect you from slow queries.
• These indexing rules sometimes require you to either filter data yourself or to restructure your database.

Background

If you missed Part I, you can find it here.
We won't be continuing the same project from Part I, but this post does assume knowledge of the topics covered in Part I, such as the different fundamental data types (like Snapshots and References).

Querying

Firestore's Query API will be largely familiar to anyone with a background in relational databases, as it provides a SQL-like syntax for filtering and sorting data.
However, Firestore has a much more restricted set of tools compared to a relational database system like Postgres or MySQL.
Firestore keeps things to the bare minimum, which requires you to think ahead when structuring and querying your data.

Firestore exposes a Query API with some methods that might look pretty familiar.
For example, the following query returns all users who are 30 years old:

const users = database.collection('users').where('age', '==', 30)

Query methods return a Query object, so you can chain together multiple methods to create more complex queries.

const users = database
  .collection('users')
  .where('age', '>', 18)
  .where('age', '<', 45)

Firestore also supports several other SQL-like methods.

const users = database
  .collection('users')
  .where('age', '>', 18)
  .select('name', 'birthday')
  .limit(10)

A list of all available methods can be found in the reference documentation.
Some methods that are particularly interesting are stream, which lets you stream snapshots as they are retrieved, and the startAt and startAfter methods, which allow you to paginate your data.

Much like a Reference, calling get on a Query object executes the query and returns a Snapshot, which you can then manipulate as usual.

const userQuery = database.collection('users').where('age', '<=', 25)
const userSnapshots = await userQuery.get()
userSnapshots.forEach(snapshot => console.log(snapshot.data()))

Firestore's API is intentionally compact, so it requires a bit of planning ahead of time to make sure that your database structure allows for quick queries.
Many data organization strategies from relational databases, such as using highly normalized tables with a lot of references, won't work well in Firestore, and the API reflects that.

Limitations

Once you start filtering by multiple fields, you might run into some errors.
But don't worry, Firestore will point you in the right direction!

const users = database
  .collection('users')
  .where('age', '>', 30)
  .where('name', '==', 'SomeName')
// => Error: The query requires an index. You can create it here: <:long-url:>

In Part I of this series, I mentioned that Firestore uses indexes for everything.
By default, Firestore creates indexes for every subfield in all of your documents, but it doesn't create indexes for multiple subfields.
You'll have to do that yourself, but if you follow the link in the error message, Firestore will walk you through the whole process.

Sometimes, however, Firestore is not nearly as helpful.
For example, take a look at this query:

const users = database
  .collection('users')
  .where('age', '>', 30)
  .where('name', '>=', 'SomeName')
// => Error: Cannot have inequality filters on multiple properties

We get an error, but what do we do about it?

Again, this is because of Firestore's index rules.
Firestore enforces rules to keep queries fast, which it accomplishes by only allowing queries where records will be adjacent to each other within a given index.
However, this query doesn't guarantee that the records are adjacent, meaning that Firestore would have to scan the entire index.

In instances like this, Firestore doesn't have an easy fix, and its error message, while somewhat confusing, reflects that.
Getting around this constraint is a bit trickier.
Unlike our earlier query, we can't simply create a new index for this particular query.

One option is to use only one of the where filters and do the other filter in your application, although this can quickly slow down your application on large projects since you may be loading a lot of data into memory.
If filtering in your application is problematic, you may need to reorganize your database structure to allow for more efficient queries.

Recap

Firestore is a great tool because it aims to be scalable by default.
By making restrictions to limit slow queries, Firestore enforces index usage for all queries, and while this does require some extra planning on the developer's part, these limitations exist to help us in the long run.