Monolithic repo and multiple node_modules folder

Question

I have a project using monolithic repositories. Each packages has his own package.json file to manage dependencies for the said package. I'm using lerna to install package in all "sub-packages" of our monorepo app.

The current project structure looks like this

Project/ 
| package.json
| node_modules/
|- packages/
|-- package1/ 
|--- package.json 
|--- node_modules/
|-- package2/
|--- package.json
|--- node_modules/

I'm looking into a way of generalize common dependencies in the root node_modules folder so each packages doesn't pull his own copy of a node package when running lerna exec -- npm install but instead use the one that is at the root of the monolithic repo so we avoid installing the same package in multiple repo, hence, reducing the size of the project.

I've seen some solution including to make some symlinks between project but that's doesn't seems to be an exact science since symlink support is very OS opinionated. Also, this doesn't seems to be a supported way of doing it.

Currently, we're just at the beginning and after running lerna exec -- npm install the project is already around 350mb on disk and pulling everything from npm takes around 5 minutes the first time. As the project will grow over time, this time will also extend over time...

So to resume everything, I'm looking for a way to extract the common dependencies in a node_modules folder at the root of the repo and make the sub-packages pull from this folder their common dependencies instead of getting their own copies everytime.

Answer 1

As of August 2017, Yarn includes a feature called Workspaces which does exactly this. But what's even better is that Lerna can be made aware of and integrate with it through --use-workspaces. Running lerna bootstrap as usual will then handle all this automatically.

Answer 2

I recommend not using lerna at all. I think that is a misguided idea.

Instead, just put separate modules in their own files and load with import mod1 from './mod' etc. or go ahead and create separate repos. But don't create separate repos for every single little module -- just things that you or someone else can definitely reuse (i.e. you want to publish it to help other people).

You can take advantage of npm link, scoped packages, listing github repos simply in package.json with dependencies:{"mod1":"myorg/mod1"} to make it easier to have it in a separate repo (when necessary).

In particular npm link should simplify things. http://justjs.com/posts/npm-link-developing-your-own-npm-modules-without-tears

Answer 3

Lerna recently added a --hoist option that looks like it provides exactly what you were looking for here. It installs external dependencies at the repo root so they're available to all packages. Binaries are linked into dependent package node_modules/.bin/ directories so they're available for npm scripts.

It can be passed on the command-line or added to lerna.json for durable configuration.

Documentation is available here: https://github.com/lerna/lerna#--hoist-glob