firebase database: scale with multiple databases

Scale with Multiple Databases

The best way to optimize performance and scale your data in Firebase Realtime Database is to split your data across multiple Realtime Database instances, also known as database sharding. Sharding gives you the flexibility to scale beyond the limits that apply to individual database instances, in addition to load balancing and performance optimization.

When to shard your data

You might want to shard your data across multiple databases if you're using Realtime Database and fit into any of the following scenarios:You want to scale beyond the limit of 100,000 simultaneous connections, 1,000 write operations/second, or any of the other limits for a single database instance.
You have multiple, discrete data sets and want to optimize performance (for example, a chat app that serves separate, independent groups of users).
You want to balance load across multiple databases to improve uptime and reduce the risk of overloading a single database instance.

How to shard your data

To shard your data, follow these steps (described in greater detail below):Map your data to multiple databases according to your app's specific needs.
Create multiple database instances.
Configure your app so it connects to the Realtime Database instance necessary for each data set.

Map your data

When you're mapping your data to multiple databases, try to satisfy the following conditions:Each query only runs against a single database instance. Realtime Database doesn't support queries across database instances.
No sharing or duplication of data across database instances (or minimal sharing or duplication).
Each app instance only connects to one database at any given moment.
As you're mapping your data, consider applying the following strategies:

Create a "master shard"

Store a map of how your data is stored across database instances. This way, you can programmatically look up which database instance corresponds to the connecting client. Keep in mind that this might have more overhead than directly connecting to the particular database instance you need, when you need it.

Bucket data by categories or by customer

Store data in siloed database instances, grouped by user or data type. For example, if you build a chat application that serves multiple organizations, you can create a database instance for each organization and store all the chat data in unique database instances.In this case, organization A and organization B don't share data, there isn't any duplicate data in your databases, and you only perform queries against a single database instance. Additionally, users in each organization only connect to their organization's database when they use the chat app.You can then create several database instances in advance and use the organization's ID to map a team to its database instance. For example, organization A maps to Realtime Database A.The way you map data for your app depends on your particular use case, but the conditions and strategies outlined above can help you define what works for your data.

Create multiple Realtime Database instances

If you're on the Blaze pricing plan, you can create multiple database instances in the same Firebase project.

In the Firebase console, go to the Data tab in the Develop > Database section.
Select Create new database from the menu in the Databases section.
Customize your Database reference andSecurity rules, then click Got it.
Repeat the process to create as many database instances as you need. Each database instance has its own set of Firebase Realtime Database Rules, so you can fine-tune access to your data.Note: If you downgrade from the Blaze plan to the Flame or Spark plans, your secondary database instances will be disabled, but not deleted. You can enable them again by upgrading to the Blaze plan.

Edit and deploy Firebase Database Rules for each instance

Make sure you've set up your Firebase Database Rules to grant appropriate access to each database instance in your project. Each database has its own set of rules, which you can edit and deploy from the Firebase console, or using the Firebase CLI to deploy targets.To edit and deploy rules from the Firebase console, go to the Rules tab in the Develop > Databasesection. Select the database you want to edit and modify the rules.To edit and deploy rules from the Firebase CLI, follow these steps:Set up deploy targets for each database Associate all databases that use the same rules file with the same target. For example:

firebase target:apply database main my-db-1 my-db-2
firebase target:apply database other my-other-db-3
Update 

firebase.json

 with the deploy targets:

{
"database": [
{"target": "main", "rules", "foo.rules.json"},
{"target": "other", "rules": "bar.rules.json"}
]
}
Run the deploy command:

firebase deploy
Make sure you consistently edit and deploy rules from the same place. Deploying rules from the Firebase CLI overrides any edits you've made in the Firebase console, and editing rules directly in the Firebase console overrides any recent changes you've deployed through the Firebase CLI.

Connect your app to multiple database instances

Use the database reference to access data stored in secondary database instances. You can get the reference for a specific database instance by URL or app. If you don't specify a URL, you'll get the reference for the app's default database instance.WEBSWIFTOBJECTIVE-CANDROID

// Get the default database instance for an app

@property (strong, nonatomic) FIRDatabaseReference *ref;

self.ref = [[FIRDatabase database] reference];

// Get a secondary database instance by URL
@property (strong, nonatomic) FIRDatabaseReference *ref;

self.ref = [[FIRDatabase databaseWithURL:@"https://testapp-1234.firebaseio.com"] reference];
 

Optimize the connections on each database

If each client needs to connect to multiple databases during a session, you can reduce the number of simultaneous connections to each database instance by connecting to each database instance for only as long as is necessary.

Get more advice

If you need more help sharding your data across multiple database instances, reach out to the Firebase experts on our Slack channel or on Stack Overflow.