Efficient Node Development with TypeScript

Our engineering team at Doctor Evidence places a high value on a fast, efficient, and responsive development environment. Development speed is highly dependent on immediate feedback as you code. Ensuring that the result of any code change can be seen in seconds dramatically improves the ability to stay focused and quickly work through coding challenges. Every millisecond counts towards productivity in the edit/compile/test feedback loop.

Fast automatic reload after code changes

On the front-end, we use Webpack’s hot module reloading for instant feedback to code changes. We also have a NodeJS server that caches most data used on the front-end, allowing for quick application load times.

We use TypeScript on the front-end and in NodeJS, which needs to be compiled. Getting started with incremental compilation in TypeScript, and automatic reloads isn’t too difficult. You can take advantage of TypeScript’s --watch argument to enable incremental compilation. And you can use a process manager like node-supervisor to monitor for file changes and automatically restart. This will get you started with a great environment for relatively quick compilation and reload.

However, there are some limitations to this, and consequently we have written our own process management modules to handle incremental compilation and restart.

First, we use custom transformers in TypeScript (which we recently blogged about), and these can’t be configured and used from the command line. Instead we must use TypeScript compiler API to compile.

Second, our process manager is built to handle graceful shutdowns (which we also recently talked about).

Third, we have implemented some useful optimizations. With a large codebase, TypeScript’s incremental compiler can hold a large amount of memory. Our process manager will keep TypeScript in memory with a inactivity timeout of 5 minutes. For frequent code changes TypeScript will stay in memory, and instantly transpile any code changes. After five minutes the process will exit, freeing up memory, which can be valuable when you are focused more on front-end dev (and it will auto-restart the process once any relevant code changes occur).

Debugging in Chrome DevTools

The ability to debug NodeJS applications with Chrome DevTools is incredibly useful — invaluable for serious NodeJS development. There are a few things to improve this experience.

Our process manager will always start our main NodeJS process with remote debugging enabled (the --inspect flag), so we can easily connect anytime we want (which has become even more convenient now that there is a button directly on the DevTools header for connecting to NodeJS).

We also enable source maps in our TypeScript configuration. This allows you to debug and set breakpoints withing actual TypeScript source, instead of having to set breakpoints in compiled code. This can be enabled by setting ”sourceMap”: true in the compilerOptions in your tsconfig.json.

However, source maps in NodeJS do not always cooperate well with DevTools. In particular, in Windows, NodeJS will assign OS-based file paths to identify modules, but DevTools does not consider these valid URLs, and consequently will fail to follow the relative links in source map directives necessary to actually find the source code and have it mapped to the compiled code. Consequently, source maps do not usually work properly in NodeJS on Windows with DevTools.

Fortunately, there is a workaround. If you force NodeJS to identify modules with file: URLs, Chrome DevTools will be able to properly follow relative references to the source map and source code and properly load map your source code. The workaround is a bit ugly though. We intercept or “monkey patch” Node vm's runInThisContext to alter the file path to a file URL:

let runInThisContext = vm.runInThisContext
    vm.runInThisContext = function(wrapper, options) {
      if (options && options.filename && options.filename.match(/\.js$/) && options.filename.indexOf('node_modules') == -1) {
        options.filename = 'file:///' + options.filename.replace(/\\/g, '/')
      }
      return runInThisContext.apply(this, arguments)
    }

You may want to only run this on Windows, although it can also help on Unix systems to properly handle clicking on stack traces in DevTools.

Source Mapping for Stack Traces

Source maps are wonderful in your debugger, but it can be a pain if your stack traces always show line numbers from compiled code. This can be particularly annoying when you record stack traces in your logs, and you can’t simply look in your source code to find the line number, you have to find the compiled code to match up the line number. Fortunately, support for updating stack traces to use mapped source code is easily added with the excellentsource-map-support package. This can enabled by simply installing the package and running its install() function:

import * as sourceMapSupport from 'source-map-support'
sourceMapSupport.install()

We recently finished a couple pull requests to enable source-map-support to properly work with file URLs as described above (which the project owners very quickly added and published, available in version 0.5.2). By combining these two techniques, we have full source mapping in our stack traces and in our debugger. Furthermore, we can click on stack traces in DevTools console, and because the file URLs are understood both by the stack trace support as well DevTools, clicking on a stack line number will take you directly to the correct source code.

DevTools Workspaces

To tie all this together even further, we can take advantage of DevTools “workspaces” along with source map support and auto-reloading, and we can actually even edit our source files directly in DevTools. As soon as we are done editing, press Ctrl-S and the file is saved, our process manager detects the change, the code is recompiled, the process restarts, the debugger reconnects, our breakpoints are still in place, and we can continue debugging work within seconds, quickly stepping through code and making edits in place. Coupled with our unit tests to quickly re-test changes, programming becomes seamless and productive.

It is well worth the effort to setup a fast development environment with source maps enabled and have workspace editing and unit tests available. All these pieces come together to make a cohesive development environment that is truly enjoyable and extremely productive.

If you are interested in working in a fast, responsive development environment for an advance medical research platform, you can see openings here.