Cross developing for node.js and browsers using Browserify

Posted: May 17th, 2013 | By Amitay Dobo | Filed under: Development, Technology | Tags: browserify, grunt, jasmine, Javascript, jsemine, node, node.js | No Comments »

I have recently started writing a Javascript library. Trying to create an environment forÂ continuousÂ building and testing the environment, I found the node.js ecosystem, has a lot to off nice tools to offer, such as Grunt, and the npm package management was very nice to use.
So I’ve decided to target the library both to Node.js and browsers, why not?

It turned out that some of the issues were not so trivial to solve, so I’ve started working on an example project, to clear up some of the problems in a Â more limited scope.

Code speaks louder than (natural language) words, so try this example atÂ https://github.com/amitayd/grunt-browserify-jasmine-node-exampleÂ .

The example tries to provide solutions for the following issues:

Writing code in CommonJS (node.js) modules, and running it in the browser
Having browser-specific code (such as DOM manipulation)
Having node-specific code (such as File system)
Providing different implementation according to the environment (node/browser)
Being able to use libraries not supported as node.js module in the browser
Writing tests shared for node and the browser, and running it in the two environments
Having node-specific and browser-specific tests
Fully automated continuousÂ build and test (provided by grunt)

I have chosen grunt.js for the build runner, and JasmineÂ as the test framework. Those choices are pretty trivial to replace by any other setup, and the design concepts and the source code can remain the same.
However, the choice in BrowserifyÂ as a solution for packaging for the browser dictates using CommonJS/node.js modulesÂ (see their docs). Further discussion below.

Now to some discussion of the issues, how they are resolved, and why

Module System / Why CommonJS and browserify?

Modules allow us to better organize the code, make it more readable and navigatable, and expose only a subset of it on a “need to use” basis.
There are two common types of modules: AMD (Asynchronous module definition) and CommonJS modules.

AMD modules are arguably more common in the browser, and can be loaded by loaders such as RequireJSÂ . RequireJS can also (through its optimizer) package together different modules. AMD allows (as its name suggests) to load modules upon demand, by having the module initializedÂ asynchronously after its requirements have been provided.

CommonJS modules however, are dominant in the node.js environment, they are simpler (though powerful, as they are not any language construct, but just built in functions and variables). Â It consists of the require() function, which synchronously loads a model (normally into a variable), and module.exports, which defines what will be returned when a call to require() has been made.
However, you can’t simply embed those modules and requirements as scripts in the browser, as it is unaware of any module context. For that you need something like Browserify, which packages a module or a set of those in a wrapper that allows it to be embedded as a script.

I chose the CommonJS module system mostly for their greater elegance (i.e. simpler code), and a feeling I don’t need the “advanced” features of AMD modules.

My feeling is (and I don’t have a great deal of mileage on the issue yet) that for a library which is self contained, and normally is not split on different “packages” or parts of it are requsted on-demand, CommonJS is a good solution. It’s simpler, and since Browserify is a straight-forward solution, for those use cases it will suffice.

However, RequireJS and its alternatives should be explored, especially Â if you think more advance usage of async script loading and its other advanced configuration options.

Browserify Gotchas

Browserify works like that: To create a bundle that contains a specified module(s) it walks through all the module dependencies and bundles them too. You can also define exposed modules, which will be available outside of the bundle through require(), and specify modules which are ignored or external (i.e. in other bundle), which will not be included in the bundle.
This “greedy” Â dependency resolution takes some care. You need to explicitlyÂ tell browserify which modules youÂ don’t want there, otherwise you might find in the bundle modules that you don’t need there. I found myself looking through the bundled js files quite a lot to “debug” what went there and what didn’t. Even transitive dependencies are currently included in the bundle (see issue that I’ve opened).
Also keep in mind that Browserify works just by looking for raw require(‘someModule’), so if you’re requiring through some parameterized function for example (for example require(module + version)) it might fail. This can be exploited to prevent node from including some dependencies.

So you might end up with a configuration like this:

                src: ['src/common/**/*.js', 'src/browser/**/*.js'],
                dest: 'dist/app_bundle.js',
                options: {
                    alias: ["./src/browser/App.js:SampleApp"],
                    externalize: ['src/common/**/*.js', 'src/browser/**/*.js'],
                    ignore: ['src/node/**/*.js'],
                }

Different implementations for node/browser environment

This was pretty straight-forward to achieve. It is demonstrated in PersistentReaderWriter.js:

var Utils = require('./Utils');
/* Offer a concrete implementation according to the environment */
if (Utils.isBrowser) {
module.exports = require("../browser/CookieReaderWriter");
}
else {
module.exports = require("../node/FileReaderWriter");
}

Some care have to be taken as mentioned above so code from src/node/ will be ignored in the bundling process.
The test only tests PersistentReaderWriter on both the environments, and thus the two implementations (separate tests for them would be a more classic approach, but unnecessary for the example purposes).

Testing

Tests are run in 3 ways:

Jasmine-node : Run in a node environment (excludes browser specific tests)
Jasmine in PhantomJS: headless browser forÂ continuousÂ testing Â (excluded node specific tests)
An html file jasmine runnerÂ (excluded node specific tests)

Only the first test is run with the original source code, before the bundling. This makes debugging in the browser not-so-great (but not so bad if you don’t use the minified version). However, if build a library that is mostly not browser-related, your tests will be run with the node runner first, and you can debug and fix them in that environment.
Browserify also provides some source maps support (which is maybe supported by Chrome?), but haven’t got to that.

The tests have to be browserified too to have access to main source code bundle. In that main bundle all the modules are externalized (i.e. exposed), so requires by the test_bundle will be resolved.
This might creates a small problem if you want to have only one exposed module (i.e. entry point). In that case you I suggest having just one test that will run with that bundle, and only check that is loaded successfully. You can consider it as an integration test.

Using External libraries

The project demonstrates 2 ways of using external libraries.

A node package (shown with the underscore library).
Those packages are bundled by browserify.
An external library loaded in the browser into some global variable.
This is demonstrated by jQuery. Note that you can (and probably should) have jquery browserified too, using https://github.com/thlorenz/browserify-shim, which is also included in grunt-contrib-browserify.

A third type could have been:

Using a node package in node environment, and a global environment in browserify.

This can be achieved by using a function like in src/common/Utils.requireOrGlobal(), but I decided against including it. It raises the question of whether a dependent library should be always included in your library package, or can you trust the client to provide it, which I will not go into.

Grunt

The use of grunt was overall a great experience. However, the quality of the plugins varied, and not all conformed to the latest multitask API, which allows defining multiple targets for the same task type. However, using the shell plugin pretty much covered those gaps, and writing custom tasks seems pretty easy too.
I especially enjoyed using “grunt watch” to having instant feedback for each change I made.