The only NodeJs introduction you’ll ever need.

A step by step introduction to writing web apps with the JavaScript environment.

Photo by Farzad Nazifi on Unsplash

If you write front end code, it shouldn’t be news that you could easily write web apps with NodeJs since both rely heavily on JavaScript.

It’s very important to understand that Node isn’t a silver bullet, it’s not always the best solution for every project. Anybody can start up a server in Node, but it takes a deeper understanding of the language to write web apps that scale.

Recently, I’ve been having a lot of fun with Node and I think I’ve learned enough to share, get feedback and learn more. Generally to cement my knowledge.

So lets get started.

Before Node.js.

Web applications were written in a client/server model where the client would demand resources from the server and the server would respond with the resources. The server only responded when the client requested and would close the connection after each response.

This pattern is efficient because every request to the server takes time and resources(memory, CPU etc). So, it’s wiser to close a connection after serving the requested asset so the server could respond to other requests too.

So how do servers like these respond to millions of requests coming in at the same time? If you asked this question, you understand that it wouldn’t be nice if requests to servers would be delayed till all other requests were responded to.

Imagine visiting Facebook and you were told to wait for 5 minutes, for thousands of people requesting before you. There has to be a way to run thousands or at least hundred of requests at once. Good news!!. There’s a thing as Threads.

Threads are a way for systems to run multiple operations concurrently. So every request would open a new thread, every thread had all it required to run the same code to completion.

If it sounds strange?. Let’s use this analogy.

Imagine a restaurant with just one person serving food. When the demand for food increases, things would crazily go wrong. People would have to wait till all preceding orders were delivered. A way to solve this problem would be to employ more people to serve food right? This way, more customers would be served at once.

Each thread is a new employee and the browsers, well, hungry people. I’m sure you can get the point now.

But this system has a downside. It would get to a point where there’s a lot of requests and starting up new threads would consume a whole lot of memory and system resources. Just like in our example, employing more and more people to serve food would cost more money and space.

But it’s also a good thing that immediately the server responds to the browser, the connection is closed and all resources (memory etc) are retrieved.

A benefit of this system is that it excels at CPU intensive applications. CPU intensive operations require a lot of logic and take more time to compute. Since every new request is handled by a new thread, it takes serious work off the main thread therefore making the system faster and able to perform important computations.

It’s quite efficient because each operation executes code off the main thread. But could things get better?

Here comes NodeJs.

Image by gareth at unsplash.

Imagine we have this multi-threaded server (running Ruby on rails) that reads a file saved on the server and sends it to the requesting browser. It’s important to understand that Ruby won’t read the file for us directly. Ruby will tell the file system to read the file and return the contents . The file system is a program used to store and retrieve data on a computer.

The point is, Ruby just sits there doing nothing till the file system is done. And then Ruby collects the content and sends the contents to the browser.

Here’s where NodeJs comes in. In Node, while the file system is reading the file, Node uses the idle time to handle other requests. When the file system is done, it’s smart enough to tell Node to come take the resource and send to the browser. This is possible because of Node’s event loop.

Node is primarily made up of JavaScript and the event loop.

The event loop is basically a program that waits for events and dispatches them when they happens. One other important fact you might know is that JavaScript is single thread and so is Node.

Remember our restaurant example? Node would have just one person to serve food, no matter the crowd.

Unlike other languages that require a new thread or process for every single request, NodeJs takes all requests and then delegates most of the work to other system workers. There’s something called a Libuv library which handles this work effectively with help from the OS kernel. When the background workers are done doing their work, they emit events to NodeJs callbacks registered on that event.

This brings us tocallbacks. They are basically functions passed into other functions as arguments and are called when certain conditions occur.

So what NodeJs developers basically do is write event handlers that get called when certain Node events happen.

NodeJs is extreamly faster than mullti-threaded systems. Yea even with a single thread.

This is because programs usually don’t only consist of numeric, arithmetic and logic computations that take much time. In fact, a lot of times they merely write something to the file system, do network requests or access peripheries such as the console or an external device. Node excels in situations like this. When it experiences them, it quickly delegates the work to someone else and tackles other incoming requests.

If you’ve been following along strictly, you might also have speculated that NodeJs doesn’t excel at operations that consume CPU. CPU intensive operations overload the main thread (Only thread). When using single threaded systems, it’s ideal to avoid such operations so the main thread can do other things.

It’s also important to know that everything in JavaScript runs parallel except your code. Yea your code executes one thing at a time even when other workers are doing their jobs concurrently.

Let’s us an analogy in case you didn’t quite grasp 100%.

Imagine a king that has thousands of other servants, the king writes a list of things he wants the servant to do (a long list), a servant takes it and delegates work to all other servants. When one is done, he takes his result to the king. The king is always busy making other lists while the servants were working, so he collects his result and gives him another work to do.

The point is, the king does one thing at a time even if alot of things are running parallel. In this anology, the king is your code and the servants are NodeJs background workers. Everything is happening parallel except your code.

Let’s keep moving!!!

Don’t do that.

Writing web apps with NodeJs.

Writing web apps with Node involves writing event handlers that get called when certain Node events occur. Let’s see an example.

1. Create a new folder, cd into the folder with your terminal or command prompt.
2. Do an npm init, hit enter till you successfully create a package.json file in the root of the folder.
3. Create a server.js file in the root folder, copy and paste the code below.

4. In the command prompt, type in node server.js and hit enter. You’ll see something similar to this.

node server.js
//Node server started at port 3000

Open your browser and hit localhost:3000. You should see a Hello worldmessage.

On the first line, we required the httpmodule. The module provides as interface to deal with http operations, we call the createServer() method which well, creates a server.

Secondly, we attach an event handler for requestson the server object. The on method takes a second argument which is a callback. The callback takes two objects as arguments. The requestand responseobjects which have information about incoming request and outgoing responses.

We could also let Node do some of the work for us.

Here, we just pass a callback to the createServer(), Node adds the requestevent handler for us. We are only concerned with the requestand response object at this point.

We use the response.writeHead()to attach some headers to the server’s response (status code and content type), response.write()writes to the web page. Then we use the response.end()to close the response to the server.

Lastly, we tell the server to listen at a port (3000). So we can at least see a demo of our web app while developing on our local machines. (localhost:3000). The listen method takes a callback as second argument. This callback fires immediately the server is started.

Getting used to callbacks in Node.

Node is a single threaded evented environment, meaning everything responds to events. Callbacks are functions that are passed to other functions as arguments and are called when an event happens.

Our example can futher be written as.

makeServer here is a callback, since functions are first class objects in JavaScript, they can be passed into variables and then other functions.

If you’re not used to JavaScript, you might need some time to get used to evented programming.

If you start writing some serious JavaScript code, you might get into trouble with callback hell. Where your code becomes hard to read because there’re a lot of functions nested deeply. When this happens, you might want to find more advanced and efficient ways to replace callbacks. Look into promises, Eric Elliott wrote a guide to Mastering the JavaScript interview: What is a promise. It’s a nice place to get started.

Routing in NodeJs.

A server stores lots of files, when browsers request, they tell the server what they are looking for. The server responds accordingly by giving them files they ask for. This is called Routing.

In NodeJs, we need to manually define our routes. It’s no big deal, here’s how to do a basic one.

Paste this code in your server.js file, navigate to it, run it with node server.js head to your browser hit localhost:3000 and localhost:3000/about,. Try doing it with something like localhost:3000/somethingelse. Our error page right?.

While this may satisfy our immediate need of starting a server, it’s crazy to do this for every web page on your server. Nobody does this actually, but it’s just to give you the idea of how things work.

If you observed, we required another module; url. It provides us with an easy way to work with urls.

The .parse()method takes a url as argument and breaks it into protocol , host path and querystringetc. If you don’t get that part, it’s alright.

Let’s take for example the url.

So when we do url.parse(request.url).pathname , it gives us the pathname or the url which is primarily what we use to route requests. But things can get easier though.

Routing with Express.

If you’ve done some underground research, you must have heard about Express. It’s a NodeJs framework for easily building web apps and APIs. Since you want to write NodeJs apps, you’ll need Express too. It makes everything easier. You’ll see what I mean.

In your terminal or command line, navigate to your root folder, punch npm install express --save. This installs the express package. To make it available in our program, we require it.

const express = require('express');

Hurray, life’s easier.

Now, let’s do some basic routes with express.

Now this looks clean right? I believe you might be getting tuned already. After importing the express module, we called it as a function. This begins our server journey.

Next, we try to set the port with server.set(). process.env.PORTgets the environment port the app is running on and somehow, if it’s not available, we default it to 3000.

If you observed the code above, routing in Express follows a pattern.

server.VERB('route',callback);

VERB here is any of the GET, POST etc, pathname is a string that gets appended to the domain .And the callback is any function we want to fire when the requests come in.

There’s one more thing.

server.use(callback);

Whenever you see a .use()method in express, it’s called a middleware. Middlewares are functions that do some http heavy lifting for us. In the code above, the middleware is an error handling one.

Finally, we do our normal server.listen()remember?

This is what routing looks like in NodeJs applications. Next we delve into databases in Node.

Databases in NodeJs applications.

Many people love the idea of using JavaScript for everything. There’re some databases that can fit right in. Like MongoDb, CouchDb etc. These databases are NoSQL databases.

A NoSQL database is structured in a key/value format. It is document based and data is not stored in a tabular form.

We’ll be looking at mongoDB which is a document based NoSQL database. If you’ve used a relational database like MySQL, SQL server etc. You should be conversant with the concept of databases, tables, rows and columns. It’s not so much of a different game from MongoDB.

Here’s a post from the Mongo official site that compares Mongo and MySQL

MongoDB and MySQL Compared

To make things more organized, you’ll want to use Mongoose which primarily enforces a Schema, basic type checks and validation to our documents before saving them to Mongo. It acts as a middleman between Mongo and Node.

To get started with Mongo, head to the official Mongo site, hit the green download button, navigate to community server and select your operating system and spec.

This is a long post, to keep things short and separated, be sure to check the official mongoDB installation guide to get started.

Chris Sevilleja also wrote Easily Develop Node.js and MongoDB Apps with Mongoose and I think it should start as a basic guide to getting started with Mongo.

Building RESTful APIs with Node and Express.

APIs are a way for web applications to send data to one another. Have you ever been to a site where you make use of your facebook account to sign in or sign up. Facebook gives out a part fo their functionality for other applications to use. That’s what an API looks like.

A RESTful API is one that the server or client has no idea of the state of each other. By using a REST interface, different clients hit the same REST endpoints, perform the same actions, and receive the same responses without minding the state of each other.

An API end point is a single function in an API that returns data.

Creating a RESTful API involves sending data in JSON or XML format. Let’s try to make one in NodeJs. We’ll make one that returns dummy json data when a client requests via Ajax. It’s not a fancy API, but it’ll help us understand how things work in Node. So…

1. Create a folder called node-api
2. Navigate into the folder from the command line, hit npm init . This creates an file that contain all our dependencies.
3. Type in npm install --save express to install express.
4. Create 3 files; server.js, index.html, and users.js in the root directory.
5. Copy the following code into the respective files.

This is the data we want to share to other applications. We export it so every program can easily make use of it. That’s the idea. We store the users array in the modules.exports object.

Here, we require('express) and create our server with express(). If you watch closely, you’ll also see that we’re requiring something else. That’s our users.js, remember we stored the data we’re sharing? We need it for the program to work.

Express has some methods that help us send certain content to the browser. response.sendFile() searches for a file and sends it to the browser. Here, we use a __dirname to get the root folder where our server is running from, then we add + 'index.js' to make sure we target the right file.

The response.json() sends json content to requesting websites. We pass it an argument, users array which is what we’re actually sharing. The rest should look familiar to you I guess.

Right in the root folder, Hit node server.js. Now, open your browser. Hit localhost:3000, press the button and open your browser console.

btn.addEventListent('click',getData); The getData makes a ‘GET’ ajax request. It contains a $.ajax({properties}) function that sets certain parameters likeurl, success and error etc.

In real world systems, you won’t just be reading JSON files. You would want to Create, Read, Update and Delete data. Express allows these these through htttp verbs. POST, GET, PUT and Delete respectively.

To go deeper into making APIs with Express, Chris Sevilleja wrote Build a RESTful API with Express 4. I think it’s helpful.

Networking with NodeJs sockets.

A computer network is a connection of computers to share and receive information. To do networking in NodeJs, we require the net module.

const net = require('net');

In Transmission Control Protocol (TCP) , there must be two endpoints; one endpoint(computer) binds to a numbered port while the other connects to the port.

If you don’t understand it, take this analogy.

Imagine your cell phone. Once you buy a sim, you get assigned a number and you bind to that number. When your friends want to call you, they connect to your number. You are one endpoint and your caller is another.

I’m sure you understand now.

To cement this knowledge, we’re going to make a program that watches a file and informs connected clients when the file changes. Let’s dive right in.

1. Create a folder, call it node-network .
2. Create three files, filewatcher.js , subject.txt , and client.js . Put this into filewatcher.js .

3. Next we should provide a file to watch. Put this into subject.txt and be sure to save.

Hello world, I'm gonna change

4. Next, lets create our client. Put this into client.js .

const net = require('net');
let client = net.connect({port:3000});
client.on('data',(data)=>{
 console.log(data.toString());
});

5. We’ll need two terminals. In the first one, we run the filename.jswith the name of the file to watch like so.

Note: The lines that begin with // are not to be typed, they are additional information.

//the subject.txt will get stored in our filename variable
node filewatcher.js subject.txt
//listening for subscribers

On the other terminal, we’ll run client.js .

node client.js
//watching subject.txt for file changes.

Now go ahead and change subject.txt . Make sure to save the changes. Take a look at the client.js terminal and notice the additional line.

//subject.txt has changed.

One major characteristics of a network is that many clients can connect at the same time. Start up another command line, navigate to client.js run node client.js and change the subject.txt file again (don’t forget to save).

What are we doing?

Don’t worry if you don’t understand, let’s go over it together.

Our filewatcher.js basically does three things

• creates a server to send messages to many clients. net.createServer()
• Tell the server that a client connected, also tells the client that there’s a file being watched. console.log() and connection.write() .
• Finally watches the file with thewatcher variable and closes it when the client disconnects. Here’s filewatcher.js again.

We require two modules, fs and net for reading and writing files and doing network connections relatively. If you watch closely, you’ll notice the process.argv[2] , process is a global variable that provides vital information about the NodeJs code running. argv[] is an array of arguments. When we say argv[2] , we refer to the third argument used to run the NodeJs code. In the command line, we would enter the name of the file we want to watch as third argument argv[2] . Remember this?

node filewatcher.js subject.txt

We also have something pretty familiar; the net.createServer() this function fires whenever a client connects to the port, it accepts a callback. The callback takes only one parameter which is the object that communicates to connecting clients.

The connection.write() method sends data to any client connecting to the port 3000. In this case, our connection object. In this case, we tell the client that a file is being watched.

The watcher contains a function that sends information to the client that the watched file has changed. And when the client closes the connection by exiting the process, the onclose event fires and the event handler passes message to the server and closes the watcher.

//client.js
const net = require('net'),
      client = net.connect({port:3000});
client.on('data',(data)=>{
  console.log(data.toString());
});

client.js is pretty straightforward, we require the net module and connect it to port 3000. We then listen for the data event and logs it to the console.

You should understand that whenever our filewatcher.js does a connection.write() , our client gets a data event.

This is just a scratch of the surface of how networks work. Primarily, one endpoint broadcasts messages when certain events occur and every client connected receives it as a data event.

If you would love to go deeper into networking with Node, Here’s a post from the official NodeJs documentation; The net module. You might also want to read Building a Tcp service using Node.

Phew!, now that was a lot.

If you wish to write scalable web apps with NodeJs, you need to know more than just writing servers and routing with express.

Here are some books I recommend.

NodeJs the right way.

Web Development With Node and Express.

Have anything I’d love to know, drop it in the comments below. If you think this was valuable, follow me on Twitter for more stuff.