An Introduction to Firebase’s Cloud Firestore: Part I

TL;DR

• Taking the time to properly structure your data will make querying faster and easier.
• Firestore has some powerful, though sometimes unintuitive, features to help scale your data.

Overview

Google Firebase's Cloud Firestore is a NoSQL cloud database optimized for web and mobile development.
It has many powerful features optimized for application development, such as built-in event listeners, offline support, and integration with other Firebase and Google Cloud products.

In this post we'll dive into some of Firestore's core features by building a RESTful back-end for a social networking app with Node, Express, and TypeScript, which will give us a practical reference for Firebase best practices.
Since we're only building the back-end, a tool like Postman is highly recommended.

We're going to keep things simple: for now, we'll only be able to create users, create groups, and add users to groups.

To get started, clone the starter repo by running the following commands:

git clone -b starter https://github.com/TheGnarCo/firebase-tutorial.git
cd firebase-tutorial
npm ci

If you just want to see the finished project, you can find the repo here.

Firebase Setup

Before we start writing any code, we need to get a database up and running.
To do this, you'll need a Google account to use the Firebase console.
If you don't have a Google account already, you can create one here.

Next, we'll create our new database.
Go to the Firebase website and click on "Go to console" in the top right corner of the page.
If this is your first Firebase project, you can click "Create a project". If you already have a Firebase project, you can click "Add a project".
You'll be prompted to choose a name and whether or not to opt in to Google Analytics.
You can choose whatever you'd like for either of those; they won't affect your database setup.

Once your app has been initialized, you should see a "Database" option on the left side of the page.
Click the link, followed by "Create Database" on the following page.

You'll want to start in "test mode."
Note that this is not safe for production use, but will be fine for the purposes of this tutorial.
After that, you'll be prompted to pick a location, but the default should be fine.

Now we need to add our Firebase credentials to the Express app cloned from the repo.
To do this, we need our Firebase service key.
This can be found by clicking the settings icon on the left side of the Firebase console (next to "Project Overview").
Then, under the "Service accounts" tab, click "Generate new private key".
This will download a JSON file, which you should then move into the root directory of your project and name serviceKey.json.

On the same page where you generated your private key, there should be some sample code.
Near the bottom of that code sample there should be a url next to databaseURL.
Create another file called .env at the root of your project with the following contents (and be sure to use the url you just copied):

FIREBASE_DATABASE_URL=https://url-you-just-copied.com

At this point, if you run npm run start from your project directory and navigate to http://localhost:3000 (or a different port if you set the PORT environment variable), you should see {hello: "world"}!

Data Structure

Now that we have our database set up, let's think about how we want to structure our data.
Of course, we'll need collections for users and groups, but we also need to know which users are in which groups.
For this, we'll also create a members table. This will act similar to a join table in a relational database, but with a slightly different setup.

Before we dive into the code, go into your Firebase console and create these three collections.
You can do this by going to the database page and clicking "Start collection".
You don't need to create any documents yet (although Firebase will make an empty one automatically), since we'll do that next.

Creating Documents

Phew, enough setup.
We actually get to write some routes now!

Let's get rid of our Hello World route and create a new one.
We can start with a POST endpoint to add a new group.

We'll also be adding some type annotations throughout this project.
Since this project is pretty small, they'll be quite basic, but if you choose to add more functionality, they'll certainly come in handy.

// types.ts
export type Group = {
  title: string
}

By default, Firestore doesn't do any validation of the documents we're inserting, so we'll take care of the type checking on our end.
For a larger project, it might be best to take a look at setting up some of Firestore's data validation settings, but for now we'll rely on our types to guide us.

// index.ts
import database from './database'

app.post('/groups/create', async (request: Request, response: Response) => {
  const { title }: Group = request.body
  const newGroupReference = await database.collection('groups').add({ title })

  response.send({ id: newGroupReference.id })
})

Note that the database object is imported from database.ts, which handles initializing the database with the serviceKey.json object and your database URL that we set up earlier.
If you're following along without using the starter repo, take a look here to see how to initialize your database object.

This route introduces the first key concept to understanding how Firestore handles documents: the Reference type.

References

References refer to document locations within a given database and are used to read, write, and listen for changes to a location.
Many common methods return Reference types, such as collection(), doc(), and add().
We'll use these methods later once we start working with Snapshots and DocumentData.

Adding Users

Now that we have the fundamentals of Firestore objects under our belts, we can add some more routes.

Our next route is much like the first.
This time, we'll be creating users, but it'll look pretty familiar.

// types.ts
export type Group = {
  title: string
}

export type User = {  name: string}

// index.ts
app.post('/groups/create', async (request: Request, response: Response) => {
  const { title }: Group = request.body
  const newGroupReference = await database.collection('groups').add({ title })

  response.send({ id: newGroupReference.id })
})

app.post('/users/create', async (request: Request, response: Response) => {  const { name }: User = request.body  const newUserReference = await database.collection('users').add({ name })  response.send({ id: newUserReference.id })})

Adding Users to Groups

Let's add one last route before we end the first part of this series.
This route introduces several new concepts, such as Snapshots, DocumentData, and Firestore indexes.

// index.ts
app.post('/groups/create', async (request: Request, response: Response) => {
  const { title }: Group = request.body
  const newGroupReference = await database.collection('groups').add({ title })
  await database    .collection('members')    .doc(newGroupReference.id)    .set({})
  response.send({ id: newGroupReference.id })
})

app.post('/users/create', async (request: Request, response: Response) => {
  const { name }: User = request.body
  const newUserReference = await database.collection('users').add({ name })

  response.send({ id: newUserReference.id })
})

app.post(  '/groups/:group_id/add/:user_id',  async (request: Request, response: Response) => {    const { group_id, user_id } = request.params    const membersReference = await database.collection('members').doc(group_id)    const membersSnapshot = await membersReference.get()    const members: MemberList = await membersSnapshot.data()    members[user_id] = true    membersReference.set(members)    response.send(members)  },)

Note the added lines in our first route:

await database
  .collection('members')
  .doc(newGroupReference.id)
  .set({})

This is to initialize a collection of members associated with the new group. If we didn't do this, we'd run into problems later when we try to add a member to the group:

const membersReference = await database.collection('members').doc(group_id)

Here, we'd receive undefined instead of an empty collection to add to.

Document Indexes

It might seem strange at first to store members in an object where the user's id is the key and the value is true.
What's the purpose of using true?
Why not just use an array?

The answer: indexes.
Unlike other databases, Firestore uses indexes for all queries.
It manages this by automatically indexing subfields of every document.

Although arrays do have an "array-contains" index on them by default, using map subfields (even simple ones like { user_id: true }) creates an index with every document containing that subfield.
This means that even with millions of users, we can still query those subfields very quickly.

Besides this one quirk, there are a few other new concepts here.
We used the doc method to retrieve a specific document, but just like collection, doc returns a Reference.
This time though, instead of working with the reference directly, we need to handle the underlying data.
This brings us to the other two types that Firestore uses to represent our data: Snapshots and DocumentData.

Snapshot

Snapshots are immutable representations of a given document at a specific time.
Snapshots are retrieved when calling get on a reference to a document or collection.

DocumentData

DocumentData is exactly what its name implies: it's an object holding all of a document's data.

This data gets retrieved from a Snapshot object by calling either snapshot.data() or snapshot.get(field).
data returns all attributes for a given record, while get returns only the field that you pass to it.

Once we've retrieved the members object and added our new member, we then use set to update the original members document.

Recap

That covers the basics of Firestore for this part of our project.
In this post, we discussed the Snapshot, Reference, and DocumentData types, creating data, and retrieving data.
In later posts, we'll go over some other common tools in Firebase, such as more complex querying and validation.