npm & package.json

npm

npm (Node Package Manager) is a package manager that comes with every installation of Node.js but can be installed/updated separately if needed. npm also has a Command Line Interface (CLI) tool to install the dependencies listed in a package.json file. By default, it will install packages that are hosted on npmjs.com. Still, it can also be used to install the content of a GitHub repository (or any other git online platform like GitLab). There are several popular alternatives to npm, like yarn or PNPm

Installation

You usually don't need to install NPM manually as it is bundled with Node.js. If you install Node.js, you already have NPM installed.

checking the current NPM version

You can use the following command in the VSCode terminal (or your favorite command line tool) to check which version of NPM is currently installed:

npm -v

Updating

To update npm manually to the latest version, open the VSCode terminal (or use your favorite command line tool) and enter the following command:

npm install -g npm@latest

package.json

package.json is a file you will find in most Javascript (Typescript) projects. Package.json is a JSON file that contains metadata about the project, like the project name, a description and the current version, a list of dependencies as well as devDependencies, scripts (custom scripts that can be run from the command line using npm run SCRIPT_NAME), ...

package-lock.json is a file that gets created when you install packages using a package manager. It lists the exact version of each installed package and its dependencies (unlike the package.json itself, which often allows a range of versions to be installed). This file is important as it can be used to create reproducible builds. When you test your project on the staging (pre-production) server and then deploy it to production, you don't want the package versions to be different, as this could introduce bugs after you are done testing if a package uses a newer version in production than previously on the staging server.

creating a package.json

You can create one manually using your IDE or copy a package.json from another project and then edit it to fit your needs, but the easiest way is to use the npm cli and let it create a package.json for you.

Use your VSCode terminal (or favorite command line tool) to execute the following command and have npm guide you step by step through the creation of your package.json file:

npm init

Answer the questions displayed. When you have done this, npm will create a package.json in the root of your project.

dependencies

The part of your package.json that will change the most is the dependencies section.

The dependencies list is an object that maps a dependency name to a version (for all dependencies that come from a registry like npmjs.com)

add a dependency

You can manually add a dependency by editing the package.json and then running the command `npm i' to install it.

But usually, you will use a command. For example, to install react, you would use:

npm i react react-dom

This will install react and react-dom and update your package.json and the package-lock.json file.

    "dependencies": {
        "react": "^18.2.0",
        "react-dom": "^18.2.0",
    },

What are dependencies semver versions?

Packages use semantic versioning (semver) for their version numbers, with semver a version having 3 parts MAJOR.MINOR.PATCH:

• if the patch number changes, it means there has been a fix that does NOT introduce any breaking changes, and also NO new features, but one or more existing features received a fix, meaning if you install that version, nothing is supposed to break in your code
• the second number indicates a minor change has happened, meaning a new feature got added to the package, but this new feature does not introduce breaking changes
• the major version indicates that existing features got modified in a way that potentially will introduce breaking changes, so when updating a package and there is a change in the major version, it is highly recommended that you check out the package's changelog. If you find breaking changes, you need to verify if those will impact your code that uses the package. If there is an impact, then you need to update your code accordingly after installing the update and before committing the update

dependency version pinning (optional)

if (like me) you prefer to pinning versions of package, then add the option --save-exact to your command:

npm i react react-dom --save-exact

This will add the dependencies to your package.json as before, but the difference is that the version will NOT be prefixed with a ^ (carret):

    "dependencies": {
        "react": "18.2.0",
        "react-dom": "18.2.0",
    },

VSCode extension for manual version updates

To make manual updates of my dependencies easier, I use a VSCode extension called versionlens. I like this extension because when I open my package.json, it will fetch all the versions of the packages I have installed and tell me if a new version is available. If there is a new version and I want to use it, all I need to do is click on ↑ latest x.x.x (where x.x.x stands for the new version number), and the version lens will insert the new version for me. When I'm done with updating the versions, I save the package.json and then update everything using the npm update command

Npm cli to automatically update semver versions

To not just update your dependencies but also their semver version in the package.json file, use the update command with the --save flag, like this: npm update --save. Without the save flag npm would only update versions in your package-lock.json

missing binaries due to corrupted lock file

I encountered a problem I wanted to highlight as it took me some time to figure out what was wrong (thank you vercel support for helping me out) where installing my dependencies in my CI/CD kept failing with weird error messages about packages failing to install:

unhandledRejection Error: binary for this platform/architecture not found!

Usually if there is an integrity problem, you get a clear message like this one:

npm error code EINTEGRITY

What caused the problem was a replace all I had done to change the version of a package. Instead of just replacing the version in the package.json I had also replaced matching versions in the package-lock.json. This meant that I now had a lock file where some packages had an updated version that was not matching the integrity hash anymore. This led to the unhandled rejection.

So this is a reminder to myself, to make sure you never manually edit a package-lock.json. Instead if something is not right, you should delete the lock file (keep the package.json) and then use npm i to create a fresh package-lock.json.

The package-lock.json file contains a list of all packages that got installed in your node_modules. Each module can have an integrity value:

integrity: A sha512 or sha1 Standard Subresource Integrity string for the artifact that was unpacked in this location. For git dependencies, this is the commit sha.

The hash value gets used to verify that between the time it got added in your package-lock.json and the time you install the package in production (using the package-lock.json), the package has not been changed. Every change to content of the package would result in the verification hash to be different, which means the hash in your lock file would not match the current hash of the package and hence the suspicious package would not get installed

Git(Hub) URLs as Dependencies

If a package/repository has not been published on npm, for example, one of your own projects on GitHub, and you want to add it as a dependency to your project, then you can do this, too.

"dependencies": {
    "GITHUB_REPOSITORY_NAME": "github:USER_NAME/REPOSITORY_NAME",
},

This will add the package located at https://github.com/USER_NAME/REPOSITORY_NAME as a dependency.

If you want to pin a more specific version of the repository, you can do so by adding a hash symbol at the end, followed by the commit ID or the tag (if the repository author has published a tag) or a release version.

For example, if adding a repository with the latest commit ID being d29f7d9, then you would do like this:

"dependencies": {
    "GITHUB_REPOSITORY_NAME": "github:USER_NAME/REPOSITORY_NAME#d29f7d9",
},

private repositories as a dependency

I will list a few methods here and add some links to the relevant documentation, but I will not add a complete walkthrough as those differ quite a lot depending on what platform your code is hosted on and based on what tools are in your deployment pipeline (if, however, I see that there is interest in such a tutorial in the GitHub discussions for chris.lu then I might write one in the future)

An alternative to using private GitHub repositories is to use npm to host private packages but be aware that this is a paid service

using a GitHub personal access token

For example, this method is recommended if you intend to deploy it on Vercel. In their Private dependencies on Vercel documentation, this is the method they recommend using. If this is the method you chose, then you might first check out the GitHub personal access tokens documentation, which explains how to create and manage personal access tokens

First, you need to create a personal access token, and then by adding the private repository to the package.json like this, it will be possible to access the private repository using the personal access token:

"dependencies": {
    "GITHUB_REPOSITORY_NAME": "git+https://USER_NAME:PERSONAL_TOKEN@github.com:USER_NAME/REPOSITORY_NAME"
}

using a GitHub App access tokens

If you are part of an organization on GitHub, there is an alternative you might want to check out called GitHub App installation access tokens

Using a GitHub repository, deploy key

For example, one such method to access private repositories is to use a GitHub repository deploy key. The public part of that key is then given to your developers, who store it locally in their SSH agent. Then, by adding the private repository to the package.json like this, it will be possible to access the private repository:

"dependencies": {
    "GITHUB_REPOSITORY_NAME": "git+ssh://git@github.com:USER_NAME/REPOSITORY_NAME"
}

The important part is replacing github (from the previous examples) with git+ssh://git@github.com:. This tells npm to use SSH to get your package (instead of https)

Publishing a package on npmjs.com

To publish a package on npm, you can use the version command to bump the version in your package.json and also create a git commit:

npm version MY_PACKAGE_VERSION

So, for example:

npm version 2.0.0

Then, to publish the package, you use this command:

npm publish

Publishing a beta version

To publish a beta version of your version, you need to add -beta.MY_BETA_VERSION at the end of your version number, MY_BETA_VERSION is a number that starts at 0 (zero) when you publish your first beta, the next beta will be -beta.1 and so on

The package version (MY_PACKAGE_VERSION) is the version you will want to use when the package comes out of beta, so for example, 2.1.0

So, to publish the first beta of the 2.1.0 version of your package, you use this command:

npm version 2.1.0-beta.0

Then, to publish the beta, you use the option --tag beta, like so:

npm publish --tag beta

The beta will not impact your latest package, meaning that if users use the regular npm i PACKAGE_NAME command, they will still get the latest version. However, if a user wants to explicitly install the beta, they can do so using this command:

npm i PACKAGE_NAME@beta

NPM alternatives

The two major developer/code platforms GitHub and GitLab each have their own package registries. If you are interested I recommend you check out the documentation on how to use packages hosted on GitHub or the documentation on how to use packages hosted on GitLab

In march 2024 Deno announced jsr.io, JSR is an ESM only (and typescript first) package registry, also good to know is that JSR has a npm compatibility layer, check out the JSR docs if you want to learn more about publishing or installing packages or check out the Ryan Dahl introduces JSR at DevWorld 2024 YouTube video