Server side browser that can execute JavaScript

Question

Is there any programming libraries available that will parse an HTML document, execute JavaScript and then allow me to navigate the DOM?  This needs to be performed server side, not client side. Any language will do, but Java, PHP, or Ruby are preferred.

Answer 1

There are now several projects that do a really good job of this:

• PhantomJS is a headless version of WebKit, and there are some helpful wrappers such as CasperJS.

• Zombie.js which is a wrapper over jsdom written in Javascript (Node.js).

You need to write JavaScript code to interact with both of these projects. I like Zombie.js better so far, since it is easier to set up, and you can use any Node.js/npm modules in your code.

Answer 2

For Java, be sure to check out HtmlUnit and HttpUnit.

Answer 3

Java has support for javascript with Rhino, also look at this page for server side javascript solutions: http://en.wikipedia.org/wiki/Server-side_JavaScript

Answer 4

in java: http://lobobrowser.org/cobra/java-html-parser.jsp
this is a a Javascript-aware, CSS-aware HTML parser
the most important feature in relation to your question: It is Javascript-aware. DOM modifications that occur during parsing will be reflected in the resulting DOM.

Answer 5

PHP has DOMDocument for navigating the DOM. I haven't heard of anything for executing JavaScript.

Answer 6

node.js ?

Node can run any javascript file in its console. I would try node first & see if it can do what you want as it likely has the largest user base & documentation.