How CLIs in Node.js actually work

I think I’ve written dozen CLI tools throughout my career. I know that many people use Commander to do so (a popular library; if it doesn’t ring the bell, just leave it); I understand why. It does have a very structured way of work and you don’t need to understand what actually happens when you invoke a command through the terminal. You would be surprised, but the best CLI tools I’ve written weren’t using any libraries at all. Oftentimes, the usage of a library or a framework can be a limiting factor. So why limit ourselves?

In this article, I would like to talk about how CLIs work! How come we type a command and something pops over our terminal, and why does it make sense to use Node.JS. By the end of this article, you will truly understand what you’re doing, and the “limiting factor” in question would be completely eliminated, so you can build some really amazing CLIs and get creative.

A CLI is just a process

That’s right. Just like any other thing in our system. A CLI is no exception. Once you invoke a command, you quickly create a process, which would usually terminate itself once a certain function has been invoked. Interactive CLIs on the other hand will remain alive until the “wizard” is complete. Once it’s so, the process will kill itself.

Actually, let me dive deeper. Let’s understand what’s a process. I know that a process has a different implementation between each OS, but I’ll really try to use the rule of thumb here.

A process is a series or set of activities that interact to produce a result; it may occur once-only or be recurrent or periodic. — Wikipedia

So this is the general term, which is quite accurate. I would also like to add, that in the context of computers, a process is the execution of instructions after being loaded from the disk into memory. If so, what would be a better way to instruct the computer than using Node.JS?!

If you know how to run Node.JS scripts you’re halfway there

The assumption is that we all know how to run a simple script with the node command, like so:

node my-cli.js

my-cli.js is an arbitrary file that can execute any function that you would like. It can read/write files, it can deploy a server, or it can perform some heavy calculations. So basically we’re already halfway there in terms of having a CLI. There are only 2 factors that separate the command above from being an actual CLI:

• Accepting arguments.
• Having an alias.

Once we nail these 2 down, we got ourselves a CLI, so let’s go through each.

The I/O of a process (accepting arguments)

Every process we create will be provided with the following parameters:

• An arguments vector, which represents a set of strings provided by the user (aka argv).
• An arguments count, which represents the number of arguments provided by the user (aka argc). In Node.JS it will actually be embedded into argv using the length property.
• Environment variables, which describe the environment of the process caller. These by default will be set to the system’s variables, but may as well get extended or overridden with each execution.

In addition, a process will be bound to 3 different streams, each of them represents a different role:

• A readable stream that will listen to inputs (aka stdin — standard input), like keystrokes or a feed of an image stream.
• A writable stream that the process can write its outputs to (aka stdout — standard output).
• A writable stream that the process can write its errors to (aka stderr — standard error).

Every system has a different way of making these resources available, however, the principles are the same. On Linux for example, the standard i/o streams are all handled exactly like files (see file descriptors).

A CLI may take advantage of all these resources. It can accept arguments from the arguments vector, it can accept an input stream from the standard input, and it can print logs to the standard output. Here’s how you can access these resources in Node.JS:

• process.argv — Arguments vector. This is an array of strings.
• process.env — Environment variables. This is a simple object which maps environment names to string values.
• process.stdin — Standard input. This is a readable stream.
• process.stdout — Standard output. This is a writable stream.
• process.stderr — Standard error. This is a writable stream as well.

Note that Node.JS will define the path to its executable and the path to the current working directory as the first and second argument respectively, so you always want to access the arguments vector using process.argv.slice(2). Oftentimes, the arguments vector will be parsed to an object, and will not be used in its raw form. For example, the arguments --foo and --bar will be parsed to { foo: true, bar: true }. You can implement a parser of your own or use a third-party library, like Minimist.

So if we put everything together, here’s one example of a CLI script that accesses the arguments vector:

To invoke the CLI above we just need to run:

node my-cli.js --foo=1 --bar=2 --baz=3

So we do know how to create a CLI, but how do we make it look like a CLI? With a quick alias to it?

PATH binaries (having an alias)

Every OS has the ability to make a binary (aka executable) accessible globally. Let’s take the node command for example. How come we can run it from every folder? Without specifying a relative path?

In all 3 — Linux, Windows, and Mac, there is an environment variable called PATH which is used to specify lookup directories for global binaries. The paths are absolute and they are separated with colons (:). Here is an example of a PATH value:

/home/{username}/.cargo/bin:/home/{username}/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/{username}/.npm/bin:/home/{username}/n/bin:/home/{username}/.deno/bin

So in my machine for example, if I run the command which node I get the value /usr/{username}/n/bin/node. As you can see, the /usr/{username}/n/bin directory is specified in my PATH, thus, the node binary is available globally in my system.

If so, to make my-cli available globally, I would either need to create a file named my-cli (without a .js extension!) and put it in one of the lookup directories or define it directly in the PATH variable:

export PATH=/home/{username}/my-cli:$PATH

Now, my-cli is detectable, but there are still a couple more issues we need to address. If you will try to run my-cli, you might see the following error on Linux or Mac:

permission denied: my-cli

Unlike invoking a script via Node.JS directly, where we read the file and then interpret it with V8, here we’re trying to execute it directly, something which is not yet achievable. On Unix machines, files are created without execution permission by default. To fix that, we can use chmod (“change mode”, see reference):

chmod +x /home/{username}/my-cli

“+” stands for positive permission and “x” stands for execution. But we’re not yet done. Even though we can execute the script, we’re still not using the right shell to do so. The system would fall back to its default shell, which will result in a syntax error. On Linux or Mac, the default shell would be Bash. To change the execution shell of a given file, we need to define its path at the top of the file, like so:

#!/usr/bin/node

This special line is known as “shebang”. If you want to know the origins of this weird word you are more than welcome to look at this StackOverflow discussion.

Note that different systems might have their node binary defined in various locations, not necessarily /usr/bin/node. In order to overcome this issue, we can use the /usr/bin/env command directly in the shebang to retrieve the system’s node path:

#!/usr/bin/env node

Don’t worry, Node.JS will gracefully ignore the shebang line and will not interpret it. Now we can happily define a Node.JS script and make it available as a CLI!

BONUS: Uploading CLIs as NPM packages

So what is more useful than defining a CLI and running it locally? — Sharing it with everyone! Of course. Using NPM, you can upload your CLI and make it available to everyone:

$ npm install -g my-cli
$ my-cli --help
hello world!

If you’ll look again at the PATH value in the section above, you would see the following directory:

/home/{username}/.npm/bin

This is a directory that was created by NPM, so later on binaries can be directly dropped to it. If you will look inside this directory you may see some of these CLIs that you’ve probably installed at some point: nodemon, tsc, create-react-app, serve, etc.

This is no coincidence. NPM gives you the ability to define packages as binaries that would be linked to the .npm/bin directory once you install them. To do so, you can define a binary name along with its entry point in package.json (see docs):

{
  "bin": {
    "my-cli": "./my-cli.js"
  }
}

Now once you install my-cli, NPM would create a file under .npm/bin/my-cli that would run the my-cli.js script once invoked. If you will look at the contents of the file, you should see the following:

#!/usr/bin/env node
require('../lib/my-cli.js')

Remember everything we’ve talked about in this article? It all came down to the code snippet above 👏.

Congrats! Now you can build a CLI using Node.JS and upload it to NPM. Below are a few examples of some really cool CLIs I’ve built with Node.JS. You can look at the bin field in package.json to get the entry points!

• Appfairy — A CLI for transpiling Webflow designs to React components.
• Git Streamer — A CLI for starting video calls and sharing code simultaneously.
• Tortilla —A CLI for writing step-by-step tutorials using Git.