JavaScript Starter Architecture For the General Website

I’ve worked in many different languages over the years I’ve been a computer nut. To-date, my use of JavaScript has been been mostly to bend web browsers to my will for some minimal purpose. Lately I have spent some time thinking about how poor the JavaScript architecture is for most websites. I’m talking about your average website — not the new style Single Page Application. On your average website you will typically find a lot of:

• Just-In-Time Snippets of code dropped into HTML on an as-needed basis.
• Inconsistent coding — it’s hard for multiple developers to keep consistent coding practices when their code is scattered throughout the site.
• Bad web browser loading behavior — these bits of code exist everywhere and <script src=…> tags are loading different things at different times. Maybe someone put a defer or async tag out there, but do they really know what that means?
• Non-existent testing — when the code is spread throughout your HTML or even jumbled up into unstructured JS files, you can’t hope to test it effectively. Instead you type up really good testing notes for your friend in QA, right?

Rethink Website JavaScript Architecture — For All Websites!

Something has to change in how JavaScript is managed, even for the general website. The wild-west coding style has died long ago in most areas of software development, and I declare it’s time for it to die in JavaScript land as well. The language itself has matured so much in the last two years, so there’s almost no excuse to not have the rest of our practices grow along with it.

First, I want to be up front and honest with everyone reading this — I am not expert JavaScript developer. I am an expert, generalist software software developer with extensive expertise in architecture. It doesn’t take me more than a couple of days to pick up the ins and outs of most languages these days. With that stage set, this adventure into JavaScript Architecture is with fresh eyes and from someone who really doesn’t have an extensive background in the JavaScript tools.

These are some goals that I set out to achieve:

• Package management system to easily integrate 3rd party packages; I’ve heard this is well done in the JavaScript world.
• Ability to build all site-code and 3rd party dependencies together into one JS file.
• Ability to use the latest JavaScript standards — browsers aren’t all caught up, but there’s trivial ways to use the latest by transpiling.
• Minification of all code.
• Testability.

As I learn and figure out what I’m doing, I’m going to document everything here step by step so you can follow along and try every step with me.

Package Management System — npm

For this, I just went with npm. I have previous experience with it on some older projects, and it easily suited my needs. A simple npm init starts you off with your starter package.json, and then npm install to install standard packages. Using -D or -P you can tell it to save the package to your package.json file for development or production use; that is a nice, clean separation of concerns in your tool space.

It also supports configuring scripts to do common things like building and packaging. I will show various things that I configure in my demo build as I describe it below. To get started, create a new, empty directory, and execute npm init and accept all of the defaults — those are things you can explore later.

Aside: This is an architecture article and not a development environment article, but I must say that I really enjoy Visual Studio Code along with its integrated Terminal for doing all of my npm work.

Sample Code & Our First 3rd Party Package

For demonstration purposes, we’re going to make a small bit of code that turns a query parameter into a cookie — a relatively common thing on many website for marketing tracking. To parse the query string, we’re going to use the query-string npm package: npm install -P query-string

Then grab these files and store them in the source/folder:

• source/referral-cookie.js
• source/site.js

There’s nothing to test at this point because web browsers don’t like some of the JavaScript code that I’ve put in these files. We have to wait until the next step to test it.

Code for this step on Bitbucket.org

Build System — webpack

The goal of this step is to package up all of our JavaScript into one file. This makes the loading into the browser faster and more efficient. I tried both webpack and rollup — webpack immediately worked to my expectations while rollup seemed to want me to do a bunch of extra rigging to make it work. So, webpack won this spot.

Webpack works for more than just JavaScript, and someday I might come back and learn the extra functionality, but for the purpose of taking an entry point (source/site.js) and producing a single JavaScript file with everything necessary to run… It just works.

First, install it: npm install -D webpack

Then, configure it by creating your webpack.config.js with this content:

const path = require('path');
module.exports = {
  entry: './source/site.js',
  output: {
    filename: 'site.js',
    path: path.resolve(__dirname, 'dist')
  }
};

Finally, add this to the ‘scripts’ dictionary in your package.json file so we can run webpack with npm:

"scripts": {
 "pack": "webpack --config webpack.config.js"
}

Now you can run it easily with npm run pack and you should see output similar to this:

> js2@1.0.0 pack /home/dev/js2
> webpack --config webpack.config.js
Hash: 98554658518c146e9d3d
Version: webpack 3.10.0
Time: 86ms
  Asset   Size  Chunks             Chunk Names
site.js  13 kB       0  [emitted]  main
   [0] ./source/site.js 87 bytes {0} [built]
   [1] ./source/referral-cookie.js 808 bytes {0} [built]
    + 4 hidden modules

Take a look at dist/site.js and you will see that your code along with other code is now packaged all together in one unit. You can add --display-modules to your webpack command if you want to see the 3rd party modules that it baked into your file.

To test this, create dist/index.html with the following contents (I know, this should be source/index.html with a build step, we’ll do that later!):

<html>
    <head>
        <script src="./site.js" type="text/javascript"></script>
        <title>Testing title...</title>
    </head>
    <body>
        <div>Test Content</div>
    </body>
</html>

Aside: Running Local Tiny Webserver

We want to test what we just did in a browser. You could load the file from the filesystem, but I find it easier and more complete to load it from a network server. There’s a small project called serve that we can install — I do it globally because I use it everywhere: npm install -g serve.

Now run serve dist and open a web browser to the URL it provides on your console. You will not see anything spectacular on the page because our code only does behind the scenes work. To see behind the scenes, in Chrome, open the developer tools and open Application — Cookies. Then add ?ref=13245 to the URL. When you load the page two things should happen:

1. A cookie named ref should appear with the value 12345__XXXXXXXXX where the Xs are replaced with the current UNIX timestamp.
2. The URL should be replaced to not include the ref= portion.

If you access the URL with ref=54321 you should see the cookie update and the ref= be removed from the URL once again.

Code for this step on Bitbucket.org.

Modern JavaScript — babel

The test code we’ve used here has been very simple and hasn’t stretched the abilities newer JavaScript functionality. Let’s add a Class to push these boundaries — oh, and we will need to use Internet Explorer to make this break since Chrome already supports Class. Create source/Cookies.js with this contents:

"use strict";
export default class Cookies {
    set(name, value) {
        document.cookie = name + '=' + value;
    }
}

Aside: I know this is a poor example of a class, and set should be a static method and not an instance method… This cheesy little class that you shouldn’t use for anything is only here for demonstration purposes for babel.

Next, add the import line to the top of referral-cookie.js and change start of the ReferralCookie function as follows (new parts in bold):

import Cookies from './Cookies';
export default function ReferralCookie() {
    const QueryString = require('query-string');
    const parsed = QueryString.parse(location.search);
    const cookies = new Cookies();
if ("ref" in parsed) {
        cookies.set("ref", parsed["ref"] + "__" + Math.round((new Date()).getTime() / 1000));
....

At this point, npm run pack, and load the page in Internet Explorer. You should get an error: Syntax Error: site.js (120,1)

Let’s install babel now: npm install -D babel-core babel-preset-env babel-loader

Create .babelrc:

{
  "presets": ["env"]
}

Add this to your webpack.config.js:

module: {
    rules: [
        {
            test: /\.js$/,
            exclude: /node_modules/,
            loader: 'babel-loader'
        }
    ]
  }

Now re-pack it: npm run pack then refresh IE. The console error is gone, and if you type document.cookie you will see your cookie sitting there! (Aside: I’m not a fan of the Internet Explorer developer tools!)

Code for this step on Bitbucket.org.

Minification — UglifyJS

Developers are the only ones that need to be able to read JavaScript code. Web browsers do NOT need to be able to read long, descriptive variable names or paragraphs of comments around elegant code. Humans need that, not machines. As website owners, we also want to greatly minimize the load time of our sites, so minification of our code is beneficial all around.

Since webpack is building everything, we want to have it integrate our minification solution. So let’s start by installing the uglifyjs-webpack-plugin: npm install -D uglifyjs-webpack-plugin. This will download uglifyjs as part of the dependency tree.

Add the following to our webpack.config.js file:

const UglifyJsPlugin = require('uglifyjs-webpack-plugin')
module.exports = {
  // ...Existing content here...
  plugins: [
    new UglifyJsPlugin()
  ]
}

Now run npm run pack once again and look at the change in dist/site.js. On my test, the resulting site.js went from 14.3KB to 5.5KB.

Code for this step on Bitbucket.org.

Testability

I think this is long enough — we’re going to save testing for another day. Hopefully you can see how putting your JavaScript into a framework like this sets you up to be able to do unit and other types of testing. I’ll probably cover this in another, future article.

Recap Development Packages

# Package, Build, and Packaging System
npm init
npm install -D webpack
# Modern JavaScript Coding
npm install -D babel-core babel-loader babel-preset-env
# Minification
npm install -D uglifyjs-webpack-plugin

Where’s the Architecture?

You are absolutely right to ask this question. We haven’t really setup an architecture in the normal sense of software architecture. What we have done so far is to setup a framework to work with JavaScript for our website with just a hint of architecture. This basic structure is absolutely required before you can attempt to enforce an architecture.

Again, I’m pretty green in the land of JavaScript, but I’m far from green in software architecture. I’m still in the beginning stages of getting this in place for a site I oversee, but my thoughts on the architecture which will build upon this includes:

• Keeping everything in modular components.
• One entry script site.js for the JavaScript that is always run on the site. The base templates for the site will include this. This executes immediately, and will have a structure to specify immediate or after onload execution for specific bits.
• One entry script embed.js for JavaScript that is always run on the site and is expected to be directly embedded into the page inside of a <script> tag — for really important things that don’t want to wait for a load of a separate JavaScript file. This executes immediately, and will have a structure to specify immediate or after onload execution for specific bits.
• Few entry scripts page-specific-functionality.js for larger bits of JavaScript that are needed for specific subsets of pages — like signup/checkout page functionality. These may not execute functionality right away, but may hook themselves into elements on the page.

As I continue on this adventure myself, I will write more articles and detail out more findings and recommendations. So far, I am pleased with this setup for organizing the code for my site.

JavaScript masters, please let me know if there’s something better or something I missed! I’m definitely interested in more information about alternatives that I did not use or try — or better operations with the ones I did try.