Node.js Introduction

What is Node.js? Before we get into that, let’s back up a minute and start by talking about web servers. You know about web servers, right? They’re computers that have software running on them that are able to respond to HTTP requests from clients (which most of the time means browsers like Chrome, Firefox, Safari, and Internet Explorer… either desktop or mobile). They’re written in languages like Perl, C, and C++. Then they have other code like Python or PHP that runs on top of them so that when a client connects to a server, all the code runs, works it’s magic (hopefully in a semi-functional way) and returns a web page of HTML to the browser… or if the endpoint is a web service: XML or a JSON payload. This is a pretty standard interaction that occurs billions of times each day across the world. If you do anything on the Internet, you participate in this process most often without even thinking about what is going on behind the scenes in the client-server interaction.

So where does Node.js come into the picture? Basically Node.js is an implementation where, instead of responding to HTTP requests, processing them, and returning web pages using the aforementioned languages, you use JavaScript instead! Node.js becomes a web server written in JavaScript. And it doesn’t just stop there. You can also run Node.js right in your own desktop environment and use utilities to perform file processes and other tasks just as you could do if you were using an installed application that was written in Java or something else. So Node.js could perhaps be better described as “JavaScript running outside of the browser.” Local applications are definitely a very useful component of Node.js, but we’re going to specifically focus on the aspect of using Node.js as a web server. But I’d definitely recommend looking at a number of different utilities available for installation to see if you can use Node.js in your development processes or just as something that makes your life easier in some way.

So, if the real interesting thing about Node.js is that JavaScript runs on the server and returns web pages and you can program your own web server in JavaScript… why would you want to do that, you ask? After all, a lot of these web servers written in these other languages are really well established, have been around for years, and have been optimized for performance and reliability by a community of super-smart programmers who have been hammering away on these projects for what seems like forever. Why do I have to reinvent the wheel and write my own server software where I’ll likely make all kinds of mistakes and leave security holes open? These are definitely all valid concerns. The real trade-off is that, depending on the type of application you are developing, in some cases the performance possibilities and simplicity that Node.js offers are too good to pass up. And don’t fret. Fortunately there are modules out there — like Express, a web-application framework that runs on top of Node.js — that are fully functioning web servers that have great communities and lots of testing behind them! You don’t even have to write the web server yourself. You can just install it and beyond your way!

But again you may still be wondering why are people going to all this trouble. As it turns out there are some aspects of the JavaScript language that have certain advantages that other languages don’t. One of the big challenges for websites living on servers is handling many different requests coming into it at once. As traffic increases, performance often suffers resulting in slow loading web pages and a diminished user-experience (UX). You may have experienced this in your development career or just your everyday browsing experience (though high-traffic is not always the sole cause of slow websites). To “scale” with the increase in traffic a website’s server has to do something to handle it. One approach is to make use of multithreading which can be challenging to develop for. Another approach companies with high-traffic websites use is just to add a bunch more servers (which can also carry an increased cost of installation and maintenance). Node.js, however, because of the way that it is structured can handle many different concurrent connections and requests from clients because of JavaScript’s event-driven architecture and the ability to make use of callback functions that execute when a resource is ready. Thus, Node.js can get a request, say “Here’s the callback to call once everything ready” and then move on to handling the next request. As is summarized in the introduction to the book Node.js: Up and Running

JavaScript is an event-driven language, and Node uses this to its advantage to produce highly scalable servers. Using an architecture called an event loop, Node makes programming highly scalable servers both easy and safe. There are various strategies that are used to make servers performant. Node has chosen an architecture that performs very well but also reduces the complexity for the application developer. This is an extremely important feature. Programming concurrency is hard and fraught with danger. Node sidesteps this challenge while still offering impressive performance. As always, any approach still has trade-offs…

So Node.js is definitely not a silver-bullet, but it does offer some advantages that other environments may not have. Without getting too bogged down in the technical details, just know that Node.js is just something you can use to run your website like any other piece of software you might choose for various reasons. Hopefully, this will become a bit clearer as we move along. Having never programmed a web server myself, I sometimes don’t really know what the all the hype for Node.js is all about. But people much smarter than I say that it’s worth taking a look at.

Installing Node.js

To install Node.js on your machine it might be a bit different depending on what platform you are using. Head on over to the Node.js website to follow the instructions there for your platform. One thing I’d definitely recommend, you’d definitely want to have Node.js avaialble to be run from the command line globally so that you can run Node from the command-line anywhere. In Windows, this means adding it to your environment PATH (which is one of the options when using the installer).

After you have node installed, do a quick test to see that everything is working. Create a file called test.js and write the following code in the file…

console.log("Hello! Node.js is running...");

Then open your favorite terminal (I like BASH) in the same directory and type the following…

node test.js

If you see the text “Hello! Node.js is running…” displayed, Node.js is installed and you are good to go!

Obligatory “Hello World” in Node.js

Below is the obligatory “Hello World” example in Node.js. Recall that we said that that the interesting thing about Node.js is that we are running JavaScript on the server. Well, below is all the code that you need to create a functioning web server written in JavaScript using Node.js

var http = require('http');

http.createServer(function (request, response) {
  response.writeHead(200, {'Content-Type': 'text/plain'});
  response.end('Hello World\n');
}).listen(8124);

console.log('Server running at https://127.0.0.1:8124/');

Now if you open a browser and type in https://127.0.0.1:8124/ you should see the text “Hello World” appear. Of course, this is not a very interesting server and it only has one route, but it shows how little code in JavaScript is needed to up and running with Node.js.

Installing Modules with NPM

Node.js uses the module pattern to load dependencies. This post on RequireJS and this post on JavaScript modules talk a bit more about what that is if you are not entirely familiar with the module pattern in JavaScript. In the previous section we loaded Node.js’s HTTP module with the following line of code…

var http = require('http');

You aren’t just limited to the modules that ship with Node. You can also use a utility called “npm” that is part of the Node.js installation (so you already have it installed because you have Node installed) to install other modules directly from your client using the command line. That’s another thing that people really like about Node.js: the community. There are many different helpful utilities that 3rd party developers have written for Node.js and posted them on npm where they can be installed and used freely by everybody. Sometimes people call the npm utility “Node Package Manager” or “Node Packaged Modules” though the npm website FAQ insists that this is not the case. Whatever you call it, it’s a useful tool. For example, to install Express — a 3rd party Node.js module that you can use to easily create web applications just as you would using something like Django, CodeIgniter, Laravel in Python or PHP — we just have to type the following into our command line…

npm install express

This will download all of the files that express requires and will place them in a folder called “node_modules”.

Then to use the express module in our Node.js files we just have to include it just as we did with our HTTP module. For example, the “Hello World” in Node.js example above couple be rewitten using Express to produce the same result…

var app = require('express');

app.get('/', function(request, response) {
   response.send("Hello World");
});

app.listen(8124);

If you need to install more than one module at once you can separate them by spaces…

npm install express socket.io grunt

Another way that you can install modules is to create a package.json file and put it in your project root. Then add the following…

{
    "name": "Node Primer",
    "description": "Node Primer by 9bit Studios...",
    "version": "0.0.1",
    "dependencies": {
        "express": "3.x"
    }
}

then all you have to do is type the following…

npm install

Node.js will look for the package.json file in the same directory and pull down all the dependencies you have specified and place them all in the node_modules folder!

The first few properties of the package.json file define some metadata, but the most important component is the dependencies. You can add more and more to this object (comma-separated), specify version numbers and by typing the above command to install everything that you need. For example, to add the module mongoose, you could edit the package.json as follows…

{
    "name": "Node Primer",
    "description": "Node Primer by 9bit Studios...",
    "version": "0.0.1",
    "dependencies": {
        "express": "3.x"
        "mongoose": "~3.5.5"
    }
}

And then just run npm install again. It’s that easy!

As mentioned previously, there are lots and lots and lots of modules available on the Node Package Manager Registry. Feel free to browse around to see if you can find anything that might be useful. Some of the more popular modules that people install include Grunt which is a JavaScript task-runner, socket.io which helps developers utilize web-sockets in JavaScript, and Mongoose which is a utility that focuses on easily integrating the NoSQL database MongoDB into your application. More info on Grunt can be found on the Grunt website. There many other modules available on npm for all sorts of different uses. You’ll just need to do a search for whatever is in line with your particular needs.

Obviously, we’ve just scratched the surface on some of the exciting and interesting aspects of Node.js. We’ve looked installation, installing modules, and how Node.js it can be used to create web applications on the server. That’s just the beginning. In upcoming discussions, we’ll look at some of the more advanced features of Node and how you can leverage some of the some of the more popular modules to use Node in whatever capacity you need it. Until next time…

That being said, the use of modules can be a very flexible and commonly used pattern for many different types applications — from games, to web apps, to script loaders, to just about everything else. So it is definitely something worth knowing if you’re in the business of creating large-scale JavaScript applications.

So let’s take a look at a module in native JavaScript…

var moduleNamespace = (function () {

    var privateVariable;
    var privateMethod;
    privateVariable = 0;

    privateMethod = function( foo ) {
        console.log( foo );

    };

    return {
        
        publicVariable: "foo", 
        
        publicFunction: function( bar ) {
            privateVariable++;
            privateMethod(bar);
        }
    };
    
})();

In the code above we have a variable named “moduleNamespace” that is assigned to an anonymous Immediately-invoked Function Expression (IIFE) — also sometimes refereed to as a self-executing function. It’s basically a function that will automatically run in your application when it’s declared. The end result is that there’s no need to write moduleNameSpace(); elsewhere in our program.

There are a lot of benefits to doing things this way. For one, this gives us a way to mimic public and private properties and methods that you find in class-based languages like C++ and Java. The private variables and functions above would not be accessible outside the scope of the anonymous self-executing function. Another benefit is that we have a lot of functionality (and we can add much more) but the only component in the global namespace is the name of the module (in this case “moduleNamespace”).

We can access the private variables and methods of the module above through calling the public methods…

alert(moduleNamespance.publicVariable);
moduleNamespance.publicMethod("Testing");

There are other variations on this module pattern that a lot of developers use. Often times, developers will move all of the functions and methods out of the return object…

var moduleNamespace = (function () {

    // Private
    var privateVariable;
    var privateMethod;
    privateVariable = 0;

    privateMethod = function( foo ) {
        console.log( foo );

    };

    // Public 
    var publicVariable =  "foo";
        
    publicFunction = function( bar ) {
        privateVariable++;
        privateMethod(bar);
    };

    return {
        publicVariable: publicVariable,
        publicFunction: publicFunction
    };
    
})();

And then the return object will return a list of key properties matched to value methods and variables. A lot of times this effectively achieves the same thing. However, if we wanted to have a different “name” of a function mapped to the one inside our module we could do it by changing the return object to something like the following…

 
return {
    a: publicVariable,
    b: publicFunction
};

Then we’d call the module variables and methods like this…

alert(moduleNamespance.a);
moduleNamespance.b("Testing");

Sometimes that gives you a bit more flexibility in how you want your application to call your module methods and access your module’s variables.

Just a quick note about namespacing for modules. “moduleNamespace” is the only variable that’s exposed to the namespace above it (which in this case would be “window”). A lot of times applications will be contained inside of an “app” namespace (or whatever variable name is chosen) because it’s not really considered good-form to dump a bunch of global variables into the window object. When this approach is used at the top of every JavaScript file in the application you may see something like…

 
var app = app || {};

Note that the name of “app” is often different (sometimes it’s the name of the application). Basically this is saying that if app is already present in the window object make the variable “app” that you’re going to be using in the JavaScript file that contains your module or modules equal to the global app variable in window or make a new empty object called app if there is not a variable called app in window. This way, you can load your JavaScript files in any order that you want, synchronously or asynchronously, and you never have to worry about dependencies of the app object being present in the global window object in your individual JavaScript files.

Then you would have to add your module to the “app” namespace instead of the global namespace…

var app = app || {};

app.moduleNamespace = (function() {

    ...

})();

That in a nutshell is how to work with modules in JavaScript.

To accomplish this in my workflow I use something called UglifyJS. It’s a simple package that runs in a Node.js environment. Node.js is a platform that allows you to run JavaScript system level applications on a server environment (or in this case, on our local machine). We can open our terminal and type something like “node myJavaScriptFile.js” and run some JavaScript we have written to do all kinds of neat stuff… all outside of the browser. So first things first, go and download and install Node.js if you haven’t already.

After that navigate to the directory where node is installed and run…

nmp (which stands for Node Package Manager, is a way of quickly installing the latest version of Node applications). You may have seen this elsewhere with Python’s pip command or Ruby’s gems. The -g parameter stands for global, which will enable you to run UglifyJS from anywhere on your machine.

Now what you can do is simply run the command according to the following format in the directory where your JavaScript files are…

In the example above, javascript-file1.js, javascript-file2.js, and javascript-file3.js will all be concatenated and compressed into a file named scripts.js file in the order in which they were set. So following this convention, you can load your dependencies where they are important in your JavaScript projects. Backbone.js, for example, has a hard dependency on Underscore. So you’d want to add Underscore *before* you’d add Backbone if you were doing a Backbone project. Also, a lot of times developers like to have jQuery available to them in their Backbone models, collections and views. So we could add jQuery into our list of files as well resulting in something like the following…

** Note that all of these files (underscore.js, jQuery.js, backbone.js) have to exist within the directory that you’re running the command from. scripts.js will be created if it doesn’t exist and overwritten if it does.

But whatever JavaScript you decide you need to include, all you have to do once the UglifyJS build process is complete is load the scripts.js file in your HTML. You can name this file whatever you want (it doesn’t have to be scripts.js). What I usually do in my workflow is set up a .bat file (and call it build-js.bat or something) that has all of my files and their dependencies set in the order that I need them. Then when I need to deploy my JavaScript to production I just run the .bat file.

So whichever way you choose to do it (either using the asynchronous module definition (AMD) pattern using something like RequireJS or using a tool like UglifyJS or using another similar solution) there are definitely good ways to load your handle your project’s JavaScript JavaScript in an efficient and effective manner.

…a JavaScript file and module loader. It is optimized for in-browser use, but it can be used in other JavaScript environments, like Rhino and Node. Using a modular script loader like RequireJS will improve the speed and quality of your code.

In this section we will walk through some basics on how and when you can use RequireJS and discuss some of the benefits you may get from doing so. Personally, I’ve always thought that RequireJS can be a somewhat confusing concept to wrap one’s head around at first. I think this may have mainly been because a lot of times RequireJS is demonstrated in the context of a big web application project and it can sometimes be difficult to follow what its responsibilities are in the midst of numerous files doing all sorts of different things. So we are going to start off very very slowly (we’re just going to use it to load a couple JavaScript files) and we will look at doing more in depth things later on.

You can grab the demo files below from this tutorial if you want to follow along…

Why Use RequireJS?

Why would you want to use RequireJS? Well there can be a number of reasons but at the bottom of it all the main purposes are to keep your JavaScript nice and organized, fast-loading, and easily extendable. If you are just doing a little bit of jQuery for a bit of enhancement on the front end of a website, then maybe RequireJS is a bit overkill for your purposes. However, the real value of RequireJS comes in when you start to develop larger client-side web applications that have numerous JavaScript files, with different dependencies, all interacting with one another..

Often times, if you look in an HTML document of you’ll see something like the following…

<!DOCTYPE html>
<html>
<head>
<title>Loading Scripts</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<script src="https://code.jquery.com/jquery-latest.js" type="text/javascript"></script>
<script src="js/underscore.js" type="text/javascript"></script> 
<script src="js/plugins.js" type="text/javascript"></script>  
<script src="js/scripts.js" type="text/javascript"></script>  
</head>
<body>

<h1>JavaScript Loading...</h1>
<div id="output"></div>

</body>
</html>

As we can see here, we’re loading jQuery, we loading some other libraries, we’re loading some jQuery plugins and we’re loading other scripts from other places. There might even be some more scripts that we’d load before the closing </body> tag. Just lots and lots and lots of script tags in the document. What RequireJS can do for us is take that out of the DOM and use JavaScript to load other JavaScript. Take a look at the HTML document below..

<!DOCTYPE html>
<html>
<head>
<title>Require JS</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<script data-main="app/main" src="require.js"></script> 
</head>
<body>

<h1>RequireJS Loading...</h1>
<div id="output"></div>

</body>
</html>

You can see we have just the one tag there. That is what loads and RequireJS and that’s really all we’re going to need as far as script definitions in the DOM go (looks nice and clean, eh?). RequireJS is going to handle the loading of all the other JavaScript files and dependencies we may have (e.g. jQuery, Underscore, etc). We’re going to define how these files get loaded, where they get loaded from, and what happens after they are loaded and we don’t have to go mucking up to the DOM with massive numbers of script tags. That’s a big benefit of RequireJS. Also, in the case above where we’re loading multiple JavaScript tags in the DOM, we’re doing so sequentially so that the browser has to wait for each script and load and before it will continue loading the rest of them. This can cause things to slow down quite a bit. But with RequireJS scripts are loaded asynchronously, so that scripts are loaded in parallel and things are not slowed down by say a much larger script on a much slower connection loading faster than a smaller script on a quicker connection. What’s even more beneficial, is that we can still tell RequireJS not to run code until the required dependency is loaded. So. for example, we can make sure that wouldn’t attempt to run any jQuery before jQuery is loaded. RequireJS will handle that for us.

So to review the benefits…

1. Single point of entry, no mucking up the DOM with numerous script tags.
2. Asynchronous loading of scripts allows for faster load times where we can still tell require not to run code before the dependency is loaded.

Using RequireJS

Now let’s talk about the RequireJS script definition…

<script data-main="app/main" src="require.js"></script> 

The src attribute should be pretty familiar if you’ve ever loaded JavaScript before. We’re basically just specifying the path to the require.js file (which is in the root directory in this case). What about that data-main=”app/main”? Well, RequireJS loads file paths in pretty much the same way as normal only it drops the “.js” What this data/main attribute is saying is “Hey RequireJS, load the file main.js found in the “app” directory and use that as the starting point for loading and running all the rest of our JavaScript.” The important thing is notice how the .js is dropped off of main.js in the path and remember that this is how RequireJS specifies paths to JavaScript files. We’ll create the main.js file and write the code for it next.

Now let’s create a file called main.js and put it in a directory called ‘app’ RequireJS is in the root of the project (where the HTML file is).

require.config({
  paths: {
    jquery: 'libs/jquery/jquery',
    underscore: 'libs/underscore/underscore',
  }

});

What this is doing is defining the paths to our JavaScript files from where this main.js file is located. And like with the data-main attribute above, the convention is to drop the “.js” from the end of each file. So the full path to each of these files from the root of our application so far would be app/libs/jquery/jquery.js and app/libs/underscore/underscore.js respectively.

What’s nice about having these other files defined is that we can easily set paths over HTTP if we wanted to as well. So if we wanted to we could change the code above to…

require.config({
  paths: {
    jquery: 'https://code.jquery.com/jquery-latest',
    underscore: 'libs/underscore/underscore',
  }

});

Where we are loading jQuery from the jQuery’s home site (just remember that the .js is dropped here as well so the full path to the file is actually https://code.jquery.com/jquery-latest.js).

Now below this let’s add some code below this config definition so that our file looks like the following…

require.config({
  paths: {
    jquery: 'libs/jquery/jquery',
    underscore: 'libs/underscore/underscore',
  }

});
require(['jquery'], function($){ 
    $('#output').html('We have now loaded jQuery...');
});

What are we doing here? Well here is where we are executing our jQuery after it has been loaded. Notice that jQuery uses the $ so what RequireJS is saying is load the first dependency in our config (jQuery) and pass in the $ as the marker that maps to jQuery. Remember in the HTML we had a <div> with the id of “output”. Here is a demonstration of how we can load jQuery through RequireJS and use it! We wouldn’t be able to use jQuery if we hadn’t loaded it successfully with RequireJS. Congratulations! You’ve just completed your first load.

What if we wanted to load something else? Let’s add UnderscoreJS to our list of items we are loading.

require.config({
  paths: {
    jquery: 'libs/jquery/jquery',
    underscore: 'libs/underscore/underscore',
  }

});
require(['jquery', 'underscore'], function($, _){ 
    $('#output').html('We have now loaded jQuery...');
    alert(window._.isEmpty({ name: 'Joe'}));
 
});

As you can see we have loaded Underscore.js and we get an alert of ‘false’ because the object we are passing in is not empty. This means that we know that we have loaded Underscore successfully because we would get an error otherwise.

So as we can see we are using JavaScript to load other JavaScript and really we could load as many libraries as we want! We just have to keep adding to our config paths and the array of properties that RequireJS takes in.

What About Loading Our Own JavaScript?

To load our own JavaScript files we do something similar. We can just include the path how we did it for the others. Let’s create an app.js file that is in the same directory as main.js. We can specify the path as being just the file name because it is in the same directory (or we wanted to we could even omit it and RequireJS will still look for it here). But in case you wanted to put it somewhere else for whatever reason, it’s good practice to have the path definned here.

require.config({
  paths: {
    jquery: 'libs/jquery/jquery',
    underscore: 'libs/underscore/underscore',
    app: 'app'
  }

});
require(['jquery', 'underscore', 'app'], function($, _, App){ 
    $('#output').html('We have now loaded jQuery...');
    alert(window._.isEmpty({ name: 'Joe'}));
    //do something with App here....
});

Now in App we can just start typing out any old JavaScript that we want but we’re most likely going to want to load the same dependencies such as jQuery and UnderscoreJS so that we can use them in our app.js file. So let’s use RequireJS to load them…

define(['jquery','underscore'], function($, _){

    console.log('Hello from App!');
	
    var initialize = function(){
       console.log('Initialize function has been called from main.js...')
    };

});

And if we can even have our app code return something so we can run it inside of main if we want.

define(['jquery','underscore'], function($, _){

    console.log('Hello from App!');
	
    var initialize = function(){
       console.log('Initialize function has been called from main.js...')
    };

    return { 
        init: initialize
    };
});

Now we can see that when we call App.init() in main.js, that function in app.js will run…

require.config({
  paths: {
    jquery: 'libs/jquery/jquery',
    underscore: 'libs/underscore/underscore',
    app: 'app'
  }

});
require(['jquery', 'underscore', 'app'], function($, _, App){ 
    $('#output').html('We have now loaded jQuery...');
    alert(window._.isEmpty({ name: 'Joe'}));
    App.init();
});

Shims

We should have a very brief discussion of shims and the use of the shim object in RequireJS. If you noticed up above when we used Underscore we had to say “window._” as opposed to just “_”. We could do this with jQuery using the ($) but we couldn’t with Underscore. Why is that? RequireJS implements the asynchronous module definition (AMD) pattern. Basically that just means that we load small little chunks of code where we can know what dependencies have already been loaded and run them, returning an object that other modules can use. It’s what we’ve been doing all along in using RequireJS. RequireJS website discusses the reasons for why AMD is important here…

The AMD format comes from wanting a module format that was better than today’s write a bunch of script tags with implicit dependencies that you have to manually order and something that was easy to use directly in the browser. Something with good debugging characteristics that did not require server-specific tooling to get started.

Remember, as we discussed above, one of the main reasons we wanted to use RequireJS is that we wanted to get away from having a whole bunch of <script> tags in our DOM.

AMD is nice, but it’s important to keep in mind that not all JavaScript libraries support this pattern. Underscore.js, for example, is one of the libraries that does not return a global object that we can just use. In contrast, jQuery (version 1.7+) does support AMD by giving us the $ and even our own module “App” was returned. But with Underscore, we had to use it in the global window object.

To enable Underscore to be implemented into our RequireJS AMD pattern, we can specify this in the shim object. Edit the config function to become the following…

require.config({
    paths: {
        'jquery': 'https://code.jquery.com/jquery-latest',
        'underscore': 'libs/underscore/underscore',
        'app': 'app'
    },
    shim: {
        'underscore': {
             exports: '_'
        }
    }
});

Now we should be able to use the “_” character wherever we want.

require(['jquery', 'underscore', 'app'], function($, _, App){ 
    $('#output').html('We have now loaded jQuery...');
    console.log(_.isEmpty({ name: 'Joe'}));
    App.init();
});

The window object is no longer needed because we now have the “_” available globally. Backbone.js is another popular application library (written by the same developer who developed Underscore) that doesn’t support AMD. However, many developers like using RequireJS for their Backbone applications. As a result of this Backbone would also need to be defined in the shim if you wanted to use RequireJS….

    shim: {
        'underscore': {
             exports: '_'
        },
        'backbone': {
            exports: 'Backbone',
        }	
    }

So there you have it. That concludes our basic rundown of loading JavaScript files with RequireJS. Hopefully, it hasn’t been too painful an experience. Feel free to grab the demo files below from this tutorial below and experiment…