Static-analysis with custom rules for JavaScript?

Question

Does JSLint, JSHint, or some other open-source static code analysis tool support adding custom rules for code compliance, or are there some ECMAScript compliant parsers that I can use to get the results as close as possible to the ones seen in the snippet below?

For example, I’d like to look into JavaScript code and list what functions are called, if it calls a library (or APIs provided by smartphones for HTML5 widgets) to register all that fall under the namespaces of that API, to make a tree of the objects and their properties to see if function is called out from what object can be traced back to, maybe with an output in XML, JSON or other structured format.

Say for example I have this JavaScript code (it does nothing and is just for the sake of the argument):

jobs = mylibrary.getJobs();
found = jobs.find("Python");
list = found.convert("html");

I want my analyzer tool to get this:

{
    "mylibrary": {
        "jobs": {"maker":"getJobs", "parent": "mylibrary"},
        "found": {"maker": "find", "parent": "jobs", "parameters": "Python"},
        "list": {"maker": "convert", "parent": "found"}
    }
}

Answer 1

I tried something with a javascript interpreter that can be accessed from code (in my case python). So interpreters like pynoceros, pynarcissus or pyv8 might help me.

There is an answer here on how to install py8: https://stackoverflow.com/a/11879224/1577343

Since with the above approach I didn't had much success I prefer a static analysis solution that uses a ECMAScript compliant parser.

With static analysis as far I could get is using JSLINT parser( Run JSLint on a .js file from debugging console in chrome or firefox): But I don't know how to use this further.

{
    "string": "(begin)",
    "first": [
        {
            "string": "var",
            "arity": "statement",
            "first": [
                {
                    "string": "jobs"
                },
                {
                    "string": "found"
                },
                {
                    "string": "list"
                }
            ]
        },
        {
            "string": "=",
            "arity": "infix",
            "first": {
                "string": "jobs"
            },
            "second": {
                "string": "(",
                "arity": "infix",
                "first": {
                    "string": ".",
                    "arity": "infix",
                    "first": {
                        "string": "mylibrary"
                    },
                    "second": {
                        "string": "getJobs"
                    }
                },
                "second": []
            }
        },
        {
            "string": "=",
            "arity": "infix",
            "first": {
                "string": "found"
            },
            "second": {
                "string": "(",
                "arity": "infix",
                "first": {
                    "string": ".",
                    "arity": "infix",
                    "first": {
                        "string": "jobs"
                    },
                    "second": {
                        "string": "find"
                    }
                },
                "second": [
                    {
                        "string": "Python",
                        "arity": "string"
                    }
                ]
            }
        },
        {
            "string": "=",
            "arity": "infix",
            "first": {
                "string": "list"
            },
            "second": {
                "string": "(",
                "arity": "infix",
                "first": {
                    "string": ".",
                    "arity": "infix",
                    "first": {
                        "string": "found"
                    },
                    "second": {
                        "string": "convert"
                    }
                },
                "second": [
                    {
                        "string": "html",
                        "arity": "string"
                    }
                ]
            }
        }
    ]
}

Answer 2

You should be able to build something like this using substack's burrito, which uses the parser from Uglify-JS and gives, I think, you all you need. A quick sample:

src.js:

var jobs, found, list;
jobs = mylibrary.getJobs();
found = jobs.find("Python");
list = found.convert("html");

ast.js:

var fs = require('fs'),
    burrito = require('burrito');

var src = fs.readFileSync('./src.js', 'utf8');

burrito(src, function (node) {
    console.log(node.name, node.value);
});

Exactly how you would build your requested structure, I'm not too sure (I'm not very well versed in AST parsing myself) but I'm sure it would entail some effort on your part. Perhaps you wouldn't need a structure in-between, so to speak, but could just validate each node from burrito, where each call node would be validated against it's values (function name, object name etc), with a warning raised if it doesn't validate.

Here is the output from the burrito call above (note: every [Object] or such has been truncated by node.js' console.log. Values are actually nodes in the parse tree from burrito, so each value has it's associated state etc).

var [ [ [ 'jobs' ], [ 'found' ], [ 'list' ] ] ]
stat [ [ { name: 'assign', start: [Object], end: [Object] },
    true,
    [ [Object], 'jobs' ],
    [ [Object], [Object], [] ] ] ]
assign [ true,
  [ { name: 'name', start: [Object], end: [Object] }, 'jobs' ],
  [ { name: 'call', start: [Object], end: [Object] },
    [ 'dot', [Object], 'getJobs' ],
    [] ] ]
name [ 'jobs' ]
call [ [ 'dot', [ 'name', 'mylibrary' ], 'getJobs' ], [] ]
stat [ [ { name: 'assign', start: [Object], end: [Object] },
    true,
    [ [Object], 'found' ],
    [ [Object], [Object], [Object] ] ] ]
assign [ true,
  [ { name: 'name', start: [Object], end: [Object] }, 'found' ],
  [ { name: 'call', start: [Object], end: [Object] },
    [ 'dot', [Object], 'find' ],
    [ [Object] ] ] ]
name [ 'found' ]
call [ [ 'dot', [ 'name', 'jobs' ], 'find' ],
  [ [ [Object], 'Python' ] ] ]
string [ 'Python' ]
stat [ [ { name: 'assign', start: [Object], end: [Object] },
    true,
    [ [Object], 'list' ],
    [ [Object], [Object], [Object] ] ] ]
assign [ true,
  [ { name: 'name', start: [Object], end: [Object] }, 'list' ],
  [ { name: 'call', start: [Object], end: [Object] },
    [ 'dot', [Object], 'convert' ],
    [ [Object] ] ] ]
name [ 'list' ]
call [ [ 'dot', [ 'name', 'found' ], 'convert' ],
  [ [ [Object], 'html' ] ] ]
string [ 'html' ]

Update:

Another option is the newer(?) ES parser Esprima, which seems to be both more actively developed and better documented. It's also reportedly faster than Uglify. You can try out e.g. parsing on the Parsing Demo page. You sould be able to build a good solution using this, methinks.

Answer 3

PMD supports ECMAScript static analysis with custom rules:

Use one of the current rulesets as an example. Copy and paste it into your new file, delete all the old rules from it, and change the name and description.

Notice that you can customize individual referenced rules. Everything but the class of the rule can be overridden in your custom ruleset.

You can also exclude certain files from being processed by a ruleset using exclude patterns, with an optional overriding include pattern. A file will be excluded from processing when there is a matching exclude pattern, but no matching include pattern.

Path separators in the source file path are normalized to be the '/' character, so the same ruleset can be used on multiple platforms transparently.

Additionally, this exclude/include technique works regardless of how PMD is used (e.g. command line, IDE, Ant), making it easier to keep application of your PMD rules consistent throughout your environment.

You can specify the full path to your custom ruleset name alongside of the built-in PMD rulesets

To see it in your IDE, add it to rulesets/rulesets.properties

References

• pmd/pmd-javascript/src/main/resources/rulesets/ecmascript at master · pmd/pmd

• Analyzing Javascript with PMD Maven

• PMD – PMD Properties

• PMD – How to make a rule set

• combine two PMD checks

• Using regular expressions in PMD rules

• PMD – IDE Integrations