Sharing code between Node applications

Question

Our app consists of two components: the API and the client. Both are independent Node applications. While the API doesn't really have any frontend components besides the docs, they both still share some common files, like Jade components, normalizing CSS, utility modules and most importantly Mongoose's schema & model definitions for MongoDB interaction. I really don't want to get used to a workflow where I first make changes to the API, and then copy the changed files to the client or vice versa, so it would be great to find a viable solution for this.

What would be the best way to share miscellaneous code across Node apps?

While browsing the web, I've come across a couple of solutions, but none of them really work in this case.

One solution is to make a node module out of the common files and keep it in sync between the apps with NPM, but that would mean I (and possible future devs) have to know which files are part of the common module, and require them with require('node_modules/mongo/schemas/example.js') instead of require('mongo/schemas/example.js') for example. Since hypothetically there might be hundreds of common files, it would be impossible to remember if a file is common or not, especially for new devs.

Another solution is to make a git submodule of the common module. This would be perfect, if only I could include the submodule path to Node's require paths, so in case it didn't find the required file in the expected location (say, mongo/schemas/example.js), it would look in the common submodule (common/mongo/schemas/example.js). This would be awesome, since it would allow overwriting a common file with a "local" version. It's kind of like Magento works. The problem here is that Node doesn't allow editing the require paths (anymore at least, supposedly it was possible prior to Node 0.5), so that's a bummer.

The third solution is to brutally merge the common repository's contents into the app's root. There are no truly insurmountable problems in this solution, but I just feel like it's a bad practice. After all, it would make it impossible to edit the common files through the app, since there's no way to selectively push the common files into the common git repository. So basically this solution would only make the workflow even more tedious: all modifications to the common files would have to be copied by hand to the common codebase, pushed to the git remote and then copied to the other app. So this solution definitely doesn't cut it for me either.

It seems that the second solution is the only (and best) option I have, unless you guys come up with a better one (which I'm really hoping for :D). If anyone knows a way to change node's require paths, I would be extremely grateful.

Answer 1

One solution is to make a node module out of the common files and keep it in sync between the apps with NPM

It's the only one good solution. If you have a shared code, it should be a module.

but that would mean I (and possible future devs) have to know which files are part of the common module, and require them with require('node_modules/mongo/schemas/example.js') instead of require('mongo/schemas/example.js') for example

No. It's require('mongo/schemas/example.js') instead of require('./mongo/schemas/example.js'). Or better yet, require('mongo').Schemas.example, though it requires some work.

You have to split your codebase into a logical modules. I guess that kind of a nightmare you have there would justify use of NODE_PATH, but I'd really suggest to refactor that...

Answer 2

Excellent question.

"but that would mean I (and possible future devs) have to know which files are part of the common module, and require them with require('node_modules/mongo/schemas/example.js')"

This is actually not true because require will search the project directory tree for a node_modules folder as per the Loading from node_modules Folders doc.

So your first option to use npm is a very good one. I have made a module in npm too actually. Once you set up the package.json, you can use npm version and npm publish to publish to npmjs.org. Then you can put the desired packages in your other project(s) package.json and download the required versions from npm using npm install.

One problem is that you must open source your code on npmjs.org, so if you don't want to, you need to create your own private npm repo.

Cheers.

Answer 3

I tried to find a solution for your exact same problem and did all the things I pointed out in your other question. Everything was working ok but I knew my workflow could be improved.

Attempt 1: add remote dependencies to package.json and load as npm dependencies

Didn't work because when Heroku installs dependencies upon deployment it doesn't have the ssh key to pull from github and basic auth (exposed username and password) was not an option.

Attempt 2: git submodules to the rescue

This brings a bunch of problems already discussed in your other question.

---

I finally ended up with ONE (well thought out) PROJECT THAT HAS EVERYTHING.

I run my app with a flag:

node app --module api

or

node app --module manager

and load dependencies accordingly.

It might not suit every project but if you have models,views,templates,constants and more business logic shared across applications, front-end and whatnot the trade off is pretty good. I don't think storage is a problem nowadays so installing a few more dependencies and unused modules might not be a problem (again,it depends on the size of the project(s) and amount of dependencies, in my case it wasn't a problem at all).

You then can target app specific tasks per service with grunt:

grunt deploy:API, grunt deploy:manager, grunt:build:manager, etc ...