Speeding up the npm install

问题:

I am trying to speed up the npm install during the build process phase. My package.json has the list of packages pretty much with locked revisions in it. I've also configured the cache directory using the command

npm config set cache /var/tmp/npm-cache --global

However, on trying to install using npm install -g --cache, I find that this step isn't reducing the time to install by just loading the packages from cache as I would expect. In fact, I doubt if it's even using the local cache to look up packages first.

回答1:

As suggested by @Daniel Serodio

You could also include your node_modules folder inside your repository but you should probably zip it first than add to repo, and while installing you can unzip it and just

 npm rebuild

(which works cross platform) it is quite fast.

This would also give you the benefit of full control over all your dependencies.

Also you can set the process flag to false to increase your speed by 2x.

npm set progress=false

Read source for more info

回答2:

You could also include your node_modules folder inside your repository (you are probably using git), and just npm rebuild (which works cross platform) on build/deploy processes, and is pretty fast.

This would also give you the benefit of full control over all your dependencies (I know that's what shrinkwrap usually should be used for)

Edit:

Also you can set the progress flag to false to increase your speed by at least 20%. This works only with npm@v3.x.x, and there will be hopefully fixes for that soon (see second link)

npm set progress=false

• Tweet about finding
• Github Issue Cause identification

回答3:

As very modern solution you can start to use Docker. Docker allows you virtualize and pre-define as image the current state of your code, including installed npm-modules and other goodies.

Once the docker image for your infrastructure/env is built locally, or retrieved from remote repository, it will be stored on the host machine, and you can spin server in seconds. Another benefit of it is that you use same virtualized code infrastructure on any machine where you deploy your code. Docker speeds up install/deployment processes and is widely used technology.

To start using docker is enough to (all the snippets are just mock/example for pre-setup and are not by any means most robust/elegant solution) :

1) Install docker and docker-compose using manuals and get some basic understanding of it at docker.com

2) Write Dockerfile file in root of your application

FROM node:6.9.5
RUN mkdir /usr/local/app
WORKDIR  /usr/local/app
COPY package.json package.json
RUN npm install

3) create docker-compose.yml in the root of your project with such content:

version: "2"
server:
  hostname: server
  container_name: server
  image: server
  build: .
  command: sh -c 'NODE_ENV=development PORT=8080 node app.js' 
  ports:
    - "8080:8080"
  volumes: #list of folders and files to use 
    - ${PWD}/server:/usr/local/server
    - ${PWD}/app.js:/usr/local/app.js

4) To start server you will need to docker-compose up -d. To see the logs docker-compose logs -f server. If you will restart your server it will do it in seconds once it built the image already at once. Then it will cache build layers locally so next run will take only few seconds.

I know this might be bit of a robust solution, but I am sure it is have most potential/flexibility and is widely used in industry. And while it requires some learning for anyone who did not use Docker before, in my humble oppinion, it is the best one for your problem.

回答4:

Proposing two more modern approches:

1) npm ci

Use npm ci, which is available from npm version 5.7.0 (although I recommend 5.7.1 and upwards because of the broken release) - this requires package-lock.json to be present and it skips building your dependency tree off of your package.json file, respecting the already resolved dependency URLs in your lock file.

A very quick boost for your CI/CD envs (our build time was cut down to a quarter of the original!) and/or to make sure all your developers sit on the same versions of dependencies during development (without having to hard-code strict versions in your package.json file).

Note however that npm ci removes the node_modules/ directory before installing, so it won't benefit from any caching strategies.

2) npm i --prefer-offline

Use the --prefer-offline flag with your regular npm install / npm i. With this approach, you need to make sure you've cached your node_modules/ directory between builds (in a CI/CD environment). If it fails to find packages locally with the specific version, it falls back to the network safely.

回答5:

We have been trying to solve this problem to speed up our deployments.

We have settled on using pac, which follows the principles in the other answers. It zips the npm modules and includues them in your repo so you don't have a million files in your commits and code reviews and you can just unzip/rebuild for the target machine.

https://www.npmjs.com/package/pac

回答6:

Nothing helped me more than disabling antivirus (Windows Defender in my case) I got from 2:30 to 1 minute. With npm-cache package I got to ~30 secs. I tried to use yarn, which is very fast, but was randomly failing in my case.