Organizing your Firebase Cloud Functions
Organize your Cloud Functions for Firebase with scale in mind. Handle a small or huge amount of Cloud Functions without a big index file. Keep a clear overview over all of your triggers.
It’s not long ago that Firebase released Cloud Functions for Firebase. If you never heard of them please take a look here and catch up after you feel ready for a deeper experience or watch the video below.
About Firebase Cloud Functions
The shortest explanation of Firebase Cloud Functions in a node environment I can write is: The Firebase Cloud Functions are just a folder in your project with it’s own node modules and a index file where you run your backend code. Your code can be triggered by url requests or realtime-database, storage, and authentication triggers.
Here is an example how a small index file could look like
'use strict'
const functions = require('firebase-functions')exports.helloWorld= functions.database.ref('/users').onWrite( (eventSnap, context)=> { console.log('Hello World') return true}
);
It sounds great and believe me it is. They take away from us the need to manage a “real” server and focus just on coding. Still we have the freedom to organize our Cloud Functions (backend code) however we want. This freedom can lead to some weird situations like a huge index file or lots of files with strange names to keep in track witch one is for what. No matter how you organize your cloud functions they have to end in the index file. This can be treated as a problem or used as solution.
Our experience
Like everyone else when we started working with the Cloud Functions there where just few of them. All fit easily into the index file. Why bother? With time more and more functions came to the file so it become pretty large.
Separating the functions in different files was the next step. It was a good solution for a while. You can imagine what happened next. Even this was hard to maintain. We made our first huge mistake. We named the files on what they are doing and not what is triggering them. This made it very hard to find something. This is how our Cloud Functions folder was organised:
functions\
authorisation.js
chat.js
containerTaskRequestAPI.js
containerTaskRequests.js
containerTaks.js
counting.js
createCMR.js
ctsTerminal.js
diverNotifications.js
driverSync.js
index.js
notifications.js
terminalCheck.js
...and so on...Even if you write the functions on your own. After time you forget what they where for or what you want to say with the name. With more developers working on the same project this becomes more confusing. We tried to use some conventions but they can’t be easily predicted to cover all use-cases so they changed over time and old names remained. This resulted in a huge number of files where some where named with older and others with newer conventions.
At that moment we realized that we had to move all the files into separate folders according to different scopes or functionality. Even if we find a way to move them to different folders we would have to make a huge refactoring over all our functions. We realized that as the number of functions raised we need to refactor all of them because our concept of organisation changed. Therefore I started to search for a solution that should remain the same from zero functions to hundreds+ functions. Prepared for easy scaling.
The idea
While searching the right way to just put the functions in separate folders I cam over a interesting GitHub discussion. There I found the solution to our problem with organizing the Firebase Cloud Functions. We found there how to make the separation into folders but also how to automate the addition of all functions from those folders to the index file. Without that we would have to add each function from every folder and file to the index file and still end with a huge index file on large projects.
The solution for our problem was the comment from David King in the GitHub discussion mentioned above and a combination of all other comments.
This is the first of four parts of our solution. The second one also comes from David King. It allows us to give the functions camel case names from the path where they are saved.
From the comment we can see that David is probably not using this way to automatically name his functions. At this point I got a clear plan how to organize our Cloud Functions for Firebase.
There we can also see from Nicolas Garnier the right way to write functions outside the index file without having trouble with the admin initialization:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
try {admin.initializeApp(functions.config().firebase);} catch(e) {} // You do that because the admin SDK can only be initialized once.
const mkdirp = require('mkdirp-promise');
// etc... my imports ...
/**
* Blur offensive images uploaded on Cloud Storage.
*/
exports = module.exports = functions.storage.object().onChange(event => {
// Function's code...
});The trick with the admin initialization is to putt it into a try catch .
At this point I wan’t to thank David King and Nicolas Garnier for their comments and help on that GitHub issue and give them credit. They deserve a huge credit for this! Without them I would still have a painful work on handling all my Cloud Functions.
Our solution for handling Firebase Cloud Functions
After the initial idea from the comment above I had a very clear plan on how to organize my Cloud Functions. There are four main parts of the solution:
- automatically load all functions in the index file
- automatically name all functions from their location
- save your functions in a folder structure that is like your realtime-database structure
- name your files according to the trigger you use ( onWrite, onCreate, onUpdate , onChange and onDelete)
The main concept is that:
The naming of functions should indicate the trigger and not what they are doing.
We already saw the code we can use to automatically load all functions in our index file and name your functions from their location. Here is the whole code of our Firebase Cloud Functions index file:
'use strict';
/** EXPORT ALL FUNCTIONS
*
* Loads all `.f.js` files
* Exports a cloud function matching the file name
* Author: David King
* Edited: Tarik Huber
* Based on this thread:
* https://github.com/firebase/functions-samples/issues/170
*/
const glob = require("glob");
const camelCase = require("camelcase");const files = glob.sync('./**/*.f.js', { cwd: __dirname, ignore: './node_modules/**'});
for(let f=0,fl=files.length; f<fl; f++){
const file = files[f];
const functionName = camelCase(file.slice(0, -5).split('/').join('_')); // Strip off '.f.js'
if (!process.env.FUNCTION_NAME || process.env.FUNCTION_NAME === functionName) {
exports[functionName] = require(file);
}
}
Don’t forget to install the modules with npm i -S glob, camelcase .
The small changes I made was to add the ignore option to the glob initialization, the modified naming call and the modified file names from functions.js to f.js. The functions are renamed to f just to reduce the typing amount. This is the whole code you need to have in your index file.
To separate the main Cloud Function triggers all realtime database functions should be in a folder called db and the folder structure in it should have exactly the same structure as your Firebase realtime database. The auth and storage triggers should have also their own folders auth and storage .
The filenames in all of the folders should be named as the trigger they are watching. This is how the folder and files should be named if we use the functions.auth.user().onCreate and functions.auth.user().onCreate triggers:
functions/
auth/
onCreate.f.js
onDelete.f.js
db/
... database functions ...
storage/
... storage functions ...
index.jsIf we watch only the auth functions they would be deployed with the names authOnCreate and authOnDelete .
Let us now deep down to the database functions. Here is the structure if we add some functions to the db folder:
functions/
auth/
onCreate.f.js
onDelete.f.js
db/
users/
onCreate.f.js
onDelete.f.js
roles/
onWrite.f.js
chats/
onCreate.f.js
tasks/
onCreate.f.js
completed/
onWrite.f.js
storage/
... storage functions ...
index.jsIf we deploy this kind of folder structure we would have this Cloud Functions deployed to Firebase:
authOnCreate
authOnDelete
dbUsersOnCreate
dbUsersOnDelete
dbRolesOnWrite
dbChatsOnCreate
dbTasksOnCreate
dbTasksCompletedOnWriteBecause the storage triggers can’t be triggered on a specific path we can name them according to the conditions on witch their code will run. A complete implementation of the mentioned concepts would look like this:
functions/
auth/
onCreate.f.js
onDelete.f.js
db/
users/
onCreate.f.js
onDelete.f.js
roles/
onWrite.f.js
chats/
onCreate.f.js
tasks/
onCreate.f.js
completed/
onWrite.f.js
storage/
userPhotos/
onFinalize.f.js
taskFiles/
onFinalize.f.js
utils/
counting.js
notifications.js
email.js
index.jsThey would deployed look like this:
authOnCreate
authOnDelete
dbUsersOnCreate
dbUsersOnDelete
dbRolesOnWrite
dbChatsOnCreate
dbTasksOnCreate
dbTasksCompletedOnWrite
storageUserPhotosOnFinalize
storageTaskFilesOnFinalizeWe can see that just files that end with .f.js will be deployed as functions. That allows us to have helper functions in a separate file in the root of the functions folder or even between the triggers itself. You are also not forced to use only this kind of naming. If you really need specific names you can use them
Upsides and downsides
If you start using a solution like this there should be a clear list of upsides and downsides you have by using it. The upsides are:
- very small index file (about 20 lines of code without the comments)
- clear naming conventions
- easy overview over all functions/triggers
- scalability
- easy separation of usual code and functions/triggers
- similar structures between database and functions
- you don’t have to manually add the functions to the index file
- all functions in the Firebase dashboard are sorted exactly as they are sorted in your functions folder
- working in teams is much easier with no need to handle merge conflicts in a single index file
The downsides would be:
- you have to use a specific naming for every function file. Add to every file
.f.jsat the end - the folder structure can get very large with large projects and more usage of nesting
That are all upsides and downsides I found for now.
Demo
If you want to see all this in action we provide a full functional open source demo application called React Most Wanted. It doesn’t have a lot of functions but the best thing about this solution is that it’s the same for small and large projects.
Update
The code from this instruction was working for years now in all of our projects. Now we have a new way to load our functions and while doing that we also enable es6 import and export syntax in all of your functions without breaking the old ones using the old syntax.
The new solution is basicaly the same but now under a npm package called firebase-function-tools . That makes bugfixes and updates to the function loading process much easier.
Before we can use the new method we need to install our package with this command: npm i -S firebase-function-tools
Now the code for loading our firebase functions would look like this:
const loadFunctions = require('firebase-function-tools')
const functions = require('firebase-functions')
const admin = require('firebase-admin')
const config = functions.config().firebaseadmin.initializeApp(config)loadFunctions(__dirname, exports)
As you can see now we have just 6 lines of code and the best part is that all of that is enough to use the es6 import and export syntax in every of your functions. We can stey with the naming as we defined it above but now we can have much much less code:
import { database } from 'firebase-functions'export default database.ref('/triggers/{uid}').onWrite(snap => { return snap.after.ref.set(null)
})
The code above shows a firebase cloud function that deletes data created under a specific path.
For more complex function calls don’t forget that the functions import looks different in es6 :
import * as functions from 'firebase-functions'Ohhh and one more thing ;) The same package will have some helper functions to avoid writing the same code over and over again in different projects. It already supports a thumbnail generator, users maper, counter and I will add more and more usefull tools to it.
If you think that you have a great solution for a common problem with firebase functions feel free to send a PR. They are always welcome :)
Update 2
Because the latest Firebase version have now a limit for the number of functions you can deploy at once and some of our producton apps have more than allowed we had to find s smooth solution for this. The great people at Firebase already made great solution for this by allowing grouped functions. We just need to implement it in our library.
If you already us hits library it’s just a little parameter away to have grouped functions. Just add a true to your loadFunctions call like here:
const loadFunctions = require('firebase-function-tools')
const functions = require('firebase-functions')
const admin = require('firebase-admin')
const config = functions.config().firebaseadmin.initializeApp(config)loadFunctions(__dirname, exports, true)
That’s it :)
Your functions will be gruped by the first folder. If you follow the instructions in this article you probably have as first level folders db , auth , storage , firestore , https , etc.. That means that all functions in auth will be in the groups “auth” , all in the folder db will be in the group “db” and so on. You can now deploy only (for example) the db functions by calling:
firebase deploy ---only functions:dbWARNING: If you already deployed all your functions without grouping and enable it later on. Be avare that all your current functions will get renamed because the grouped ones hava a “-” after the group name!!! For example the function authOnCreate will be renamed to auth-onCreate . If you rename a function it will first get deleted and then the new one deployed. I would recommend to do this migration on a time when your application is not used at all or has the minimum usage. Also don’t forget to adopt the names for the callable functions and those use setup in the firebase.json file.
If you still have to much functions in a group I would recommend just to split the group into multiple parts like db1 , db2 , db3 .
Conclusion
While Cloud Functions for Firebase provide a huge flexibility on how to use them, they allow all of us to get messy with the way we organize and store our code. Using the above guidelines you should be able to organize your Cloud Functions code in a scalable and clear way so you can find everything you search for and just forget about the index file. It is very important to name your functions by the triggers and not by their purpose.
I hope that this can help others to organize their Firebase Cloud Functions. It would be great to hear how you organize your Cloud Functions and what you think about this solution. Any comment is welcome.
If you want to keep up with my new articles on Firebase you can follow me on Twitter here. The next one would be probably about authorization with roles and grants.
