Monorepos By Example: Part 1

Through practical example we explore monorepos.

First, a monorepo is:

Definitions vary, but we define a monorepo as follows:
The repository contains more than one logical project (e.g. an iOS client and a web-application)
These projects are most likely unrelated, loosely connected or can be connected by other means (e.g via dependency management tools)
The repository is large in many ways:
- Number of commits
- Number of branches and/or tags
- Number of files tracked
- Size of content tracked (as measured by looking at the .git directory of the repository)

Atlassian — Monorepos in Git

Second, I am going to defer the arguments for / against monorepo projects:

Monorepos in the Wild

Instead we are going to explore several practical issues through example. In particular, we are going to use the Lerna tool to manage a monorepo project (a JavaScript project).

The final monorepo for this series is available for download.

Scaffolding

Before we get into the problems that monorepo projects (and particular with Lerna) solve, we need to scaffold a Lerna monorepo project; the good news is that this is super easy.

Install Lerna globally with:

sudo npm install --global lerna

note: This tutorial was written with Node.js v8.9.4 and Lerna v2.9.0.

Create a new folder and run the following commands in it to turn it into a Lerna monorepo project.

git init
lerna init

The resulting folder structure is:

Create Packages

Now that we have a Lerna monorepo project, we can start to create packages. In the context of deploying to the npm Registry, the packages would be the individual npm packages. More generally, packages allow for more loose coupling and independent dependency management (more on these later).

We create packages by creating folders in the packages folder, e.g., apple, banana, and grocery, and run the following command in each:

npm init -y

At this point the folder structure is:

Third-Party Dependencies

Say we wanted all three projects to depend on the npm package sillyname@0.0.3 (a particular version); we run the command (Lerna commands can be executed from any folder in the project):

note: If you are following along, skip this command and rather run the next one down (with hoist).

lerna add sillyname@0.0.3

The resulting folder structure is:

Observations:

• As we want the packages to be independent, each gets an updated package.json (with the sillyname dependency).
• Notice that it does generate the new package-lock.json (much like Yarn, npm now locks down secondary dependencies). Also, the Lerna documentation mentions that it supports Yarn.
• With Lerna, we can add dependencies to multiple packages with a single command.
• Each package has its own copy of the sillyname package; wasting disk and slow to install.
• One way to be more efficient is to manually update each of the packages’s package.json files and install the sillyname package once in a parent folder’s node_modules folder (remember Node.js will look in parent folders for packages). But this is tedious.

Lena has a solution; the hoist option:

lerna add sillyname@0.0.3 --hoist

The resulting folder structure (this screenshot does not show the large node_modules folder in the project’s root) is:

Observations:

• Each of the package.json files lists sillyname as a dependency.
• The sillyname package is installed in the node_modules folder in the project’s root).
• The rest of the packages in the node_modules are because the lerna package is a development dependency of the project.

Selective Upgrade

Say we want to upgrade the version of the sillyname package for only the grocery package; we run the following command:

lerna add sillyname@0.1.0 --scope=grocery

The resulting folder structure (this screenshot does not show the large node_modules folder in the project’s root) is:

Observations:

• The grocery’s package.json file is updated to depend on v0.1.0 of sillyname.
• The grocery folder has a node_modules folder with the updated version of sillyname in it.

Internal Dependencies

Now say we want the grocery package to depend on both the apple and banana packages. We run:

lerna add apple banana --scope=grocery

The resulting folder structure (this screenshot does not show the large node_modules folder in the project’s root) is:

Observations:

• The grocery’s package.json file is updated to depend on both the apple and banana packages.
• Lerna creates symbolic links (light blue) in the grocery’s node_modules folder; allows grocery to transparently access the files in the apple and banana packages.

Code

Now that we have all of our dependencies set, we can write some code.

packages/apple/index.js

const sillyname = require('sillyname');
module.exports = `apple and ${sillyname()}`;

packages/banana/index.js

const sillyname = require('sillyname');
module.exports = `banana and ${sillyname()}`;

packages/apple/grocery.js

const sillyname = require('sillyname');
const apple = require('apple');
const banana = require('banana');
console.log(`grocery and ${sillyname()}`);
console.log(apple);
console.log(banana);

Running the following command from the grocery package folder:

node index.js

outputs something like this:

grocery and Linenhiss Butterfly
apple and Trailspeaker Scribe
banana and Translucentpuma Kangaroo

Having completed coding, we add the following file as we do not want to store the node_modules folders in source control.

.gitignore

**/node_modules

And then add files, commit, and push the project to a remote repository.

Bootstrap

Now say another member on the team wants to work on the project. They first globally install Lerna.

sudo npm install --global lerna

They then clone the repository and from the project’s root folder they run the following command to install all the dependencies (including the symbolic links).

lerna bootstrap --hoist

That member is now ready to code; obviously following their normal workflow patterns (like creating a feature branch, etc.)

Next Steps

In the next article, Monorepos By Example: Part 2, we continue to explore some more practical issues.