How to deploy Node.js application with deep node_m

Question

I've run into a curious issue - apparently some Node.js module have so deep folder hierarchies that Windows copy command (or PowerShell's Copy-Item which is what we're actually using) hits the infamous "path too long" error when path is over 250 chars long.

For example, this is a folder hierarchy that a single Node module can create:

node_modules\nodemailer\node_modules\simplesmtp\node_modules\
xoauth2\node_modules\request\node_modules\form-data\node_modules\
combined-stream\node_modules\delayed-stream\...

It seems insane but is a reality with Node modules.

We need to use copy-paste during deployment (we're not using a "clever" target platform like Heroku where Git deployment would be an option) and this is a serious limitation on Windows.

Isn't there a npm command or something that would compact the node_modules folder or maybe include only what's actually necessary at runtime? (Node modules usually contain test folders etc. which we don't need to deploy.) Any other ideas how to work around it? Not using Windows is unfortunately not an option :)

Answer 1

just to add to this... another thing that helped me was listing out all installed modules with npm ls.

which will give you a tree of modules and versions... from there it's pretty easy to identify which ones are duplicates... npm dedupe didn't do anything for me. I'm not sure if that's a bug or what (Node v 10.16)

So once you identify a duplicate module install it to the root node_module directory by using npm install dupemodule@1.2.3 --save-dev. The version is important.

after that, I wiped out my node_modules directory and did a fresh npm install.

Short version

1. npm ls to get a list of all installed modules.
2. look through those modules and identify duplicate modules (version is important)
3. npm install module@version --save-dev to install those modules in the root node_modules directory and update package.json.
4. rmdir node_modules to delete the node_modules directory.
5. npm install to pull down a fresh copy of your dependencies.

Once I did that, everything was much cleaner.

I also recommend commenting your package.json file to show which ones were brought down to flatten the node_modules tree.

Answer 2

This is a not a proper solution, rather a work around when you are in a hurry, but you can use 7-Zip to zip your folder, move the zipped file and unzip it without any issue.

We used that solution to deploy a Node.js application where it was not possible to do a clean npm install.

Answer 3

npm v3(released recently) solves this issue by flattening out the dependencies.. Check the release notes here in https://github.com/npm/npm/releases/tag/v3.0.0 under flat flat section.

And the last comment on this issue https://github.com/npm/npm/issues/3697

Answer 4

I wrote a node module called "npm-flatten" that flattens your dependencies for you here: https://www.npmjs.org/package/npm-flatten

If you are looking for a distrubtion, I also wrote a NuGet package that will integrate a complete node.js environment with your .NET project here: http://www.nuget.org/packages/NodeEnv/

Feedback would be welcome.

Answer 5

1) During release build, You can prevent Visual studio scanning these files / folder by setting the folder properties as a Hidden folder (JUST set it to node_modules). Reference: http://issues.umbraco.org/issue/U4-6219#comment=67-19103

2) You can exclude files or folder that are published during packaging by including following XML node in the CsProject file.

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
  ...
  <OutputPath>bin\</OutputPath>
   <NoWarn>42016,41999,42017,42018,42019,42032,42036,42020,42021,42022</NoWarn>
  <ExcludeFilesFromDeployment>File1.aspx;File2.aspx</ExcludeFilesFromDeployment>
  <ExcludeFoldersFromDeployment>Folder1;Folder2</ExcludeFoldersFromDeployment>
</PropertyGroup>

Answer 6

I found one solution from Microsoft Node.js Guidelines.

• Start in a short path (e.g. c:\src)
• > npm install -g rimraf delete files that exceed max_path
• > npm dedupe moves duplicate packages to top-level
• > npm install -g flatten-packages moves all packages to top-level, but can cause versioning issues
• Upgrade to npm@3 which attempts to the make the node_modules folder heirarchy maximally flat.
  • Ships with Node v5
  • Or… > npm install –g npm-windows-upgrade