Category: Software Development

One issue my team had for the Glamis Recovery project was sharing code and logic between our frontend (Firebase Hosting) and our backend (Firebase Functions). If you’re not familiar with firebase, Hosting and Functions have certain assumptions and requirements of your directory structure for deploying to these services:

• Your frontend code will most likely be bundled and placed in a distribution folder for the Firebase CLI to see. Each page of your web app needs an html file with its Javascript dependencies linked to it.
• All of the backend code needs to be in the functions directory to be deployed. The index.js file will export all of the functions for Firebase to deploy.

Depending on how you setup your project’s directory structure and devops processes, which new web developers and users of Firebase may find challenging (because I certainly did), you may run into the problem that it’s difficult to share code between Hosting and Functions.

This blog post is about the dangers of keeping your frontend and backend logic separate from one another and what to do about it to improve the maintainability of your software project.

Divergence of the Codebase

Our codebases diverged for the following reasons. Don’t let these happen to your project.

• The frontend was using ES6 modules but the backend was using CommonJS modules. Though the Firebase Functions documentation features examples with CommonJS require statements, you can just as easily use ES import statements as well. VS Code will even give you a hint about it and do it for you.
• Letting the two different directories be a “knowledge gap” for your codebase. There are ways around this that we’ll talk about below.
• Poor adherence to software engineering principles. Focus on modularity, Single Responsibility, and testability from the beginning. Refactor your code when is out of compliance with these principles.

Keeping the Codebase DRY (Don’t Repeat Yourself)

Our codebase truly wasn’t very big, so we decided that we would just duplicate any logic that was needed for both the frontend and backend. Maybe that meant writing duplicate functions or maybe that meant copy-pasting entire directories of class files.

We thought, “if we change it in one place, we’ll just make sure to change it in the other.” BIG MISTAKE FOLKS. BIG MISTAKE.

This was just a two person team. Sure, maybe this would work out because we both knew everything about the codebase…at the time. Maybe we could keep each other accountable…for a while. Maybe we could be careful and diligent…until that one time.

Just think back to a time when you stepped away from a piece of code for a month. After coming back to it, it’s basically like we weren’t the original authors at all. Our brains are the epitome of an LRU Cache. And attempting to manage the continuous upkeep of this much duplicated code went south pretty fast.

It wasn’t maintainable. The code diverged. There were bugs.

Solutions that Didn’t Really Work

There were a couple of things that we tried that didn’t pan out as solutions. Keep in mind the following directory tree. This is how our project is generally structured. Most of the code we want shared originated in the src directory with all our frontend code.

project-root/
├─ functions/
│  ├─ node_modules/
│  ├─ .gitignore
│  ├─ index.js
│  ├─ package.json/
├─ node_modules/
├─ hosting_distr/
│  ├─ bundled_frontend_code
├─ package.json/
├─ src/
│  ├─ shared_resources/
│  ├─ frontend_code.js

Symlinks

In Linux, Symlinks are special files that exist in one directory and “point to” a file somewhere else. We thought we could keep all of our frontend code in the src directory and have symlinks in the functions directory pointing to the shared files in src. This works when you’re running the firebase emulators locally on your computer, but as soon as you got to deploy said functions directory, expect to get errors because ALL of the code needed for the functions needs to exist in that directory and the CLI can’t resolve the symlinks for you.

Local Node.js Modules

The Functions docs say that you can use local Node.js modules as part of your function. The local module is actually just another pointer, this time managed by npm instead of the OS; npm copies the files over to where they get used. You need to run npm install (from within the functions directory) to actually get these local modules from src copied put into the functions/node_modules directory.

This seemed promising, until you read the starred note: “Note: The Firebase CLI ignores the local node_modules folder when deploying your function.” In essence, this means that Firebase will call npm install for you on their backend and if that local module isn’t inside of the functions directory when they do that, npm will be very unhappy.

Private Modules

What about taking local modules a step further and putting the shared code into a private package on the npm servers. That way Functions can have the code when it calls npm install.

For $7 a month to have a paid npm organization, be my guest. I pay $0.22 to Firebase each month for the entirety of my backend services. I wasn’t going to pay $7 to share code with myself.

Current Solution

Identify and Organize Shared Resources

Instead of actively maintaining two copies of the shared resources with man-power and version control, we keep the authoritative copy in the src directory and we deleted all duplicate code out of the functions directory. Anything that may need to be shared between the frontend and backend is organized in the shared_resources directory:

shared_resources/
├─ classes/
├─ constants/
├─ enums/
├─ utils/

Have Backend Code Import from Shared Directory

Let the backend code import from the shared directory as if that shared directory existed inside of the functions folder. I promise that it will be there when the code runs, even though I just said that we removed it all.

import Vehicle from "./shared/classes/Vehicle.js";


import PERMISSIONS from "./shared/enums/Permissions.js";
import { MIN_COVERAGE_LENGTH } 
from "./shared/constants/coverage.js";

Using npm scripts to Copy the Shared Directory

We identified only a few moments when we needed to copy the shared resources over to the the functions directory, and we automated that copy operation into our existing development and build practices.

• Before we launch the firebase emulator suite, the shared directory should exist in the functions directory.
• Before we deploy any code, we always do a build. The build step is really for the frontend code, but now we just get the backend up-to-date at the same time.

We use npm Pre Scripts to automatically do the copy operation for us so we don’t have to think about it at all. Here are the relevant scripts:

"scripts": {

    "watch": "webpack --watch --config ./webpack.dev.cjs",
    "prefirebase:emulators": "npm run copy-shared",
    "firebase:emulators": "firebase emulators:start",



    "prebuild:prod": "npm run copy-shared",
    "build:prod": "webpack --config ./webpack.prod.cjs",
    "copy-shared": "rm -rf ./functions/shared && cp -R ./src/shared ./functions/shared"
  },

Have Git Ignore the Duplicate Directory

Because we only want a single authoritative location for these shared resources, we have git ignore the functions/shared_resources directory. No reason to be worrying about including it into version control.

Result

All of these work together to make sure the functions directory has its own copy of the shared resources when its needed. In reality, that copy is ephemeral: it’s not being tracked by version control and it’s getting deleted and recopied frequently by our npm scripts.

We no longer have to worry about our frontend and backend diverging because there is only one authoritative copy.

Shortcomings and Future Work

Two things come to mind when I think about this setup.

First, the entire time I was writing this post, all I could think about was how we hadn’t tried just putting the authoritative shared directory into the functions directory instead of src. src isn’t needed by Firebase Hosting; we only care about the distribution folder that gets made after everything is bundled together. Webpack could certainly handle reaching into the functions directory for any dependencies. I think we discounted this idea because of how much more code there is for the frontend. There was almost a psychological barrier there for us to write code for the frontend that looked like this:

import { SharedClass } from "../functions/shared/class.js";

Maybe it’s worth trying out instead.

Deploying Functions

Second, and more importantly, depending on how/when you deploy your functions, they may be utilizing stale code instead of the newest source code. Every time you modify the shared directory, there is a chance that code your functions depend on has changed. This should trigger a new deployment of your functions.

With Firebase Functions, you have the ability to deploy single functions, categories of functions, or all of the functions. We leaned on deploying singles for a while, but we recently reorganized so that we deploy categories. But we’ve been doing so manually instead of automatically in a CI pipeline.

Our hubris is begging for another important lesson. There’s an implicit assumption here in our setup that:

1. We can remember all of the dependencies inside of our functions code.
2. We will remember to deploy those functions when the dependencies get updated.

I’ll be working on this problem soon.

References

1. Feature image: Photo by Chester Ho on Unsplash