C# httpwebrequest and javascript

Question

I am using C# HttpWebRequest to get some data of a webpage. The problem is that some of the data is updated using javascript/ajax after the page is loaded and I am not getting it in the response string. Is there a way to have the webrequest wait untill all the scripts in the page have finished executing?

Thanks

Amit

Answer 1

HttpWebRequest does not emulate a web browser, it just downloads the resource you point it at. This means it will not execute or even download JavaScript files.

You would have to use something like FireBug to get the URL for the data being pulled in via JavaScript, and point your HttpWebRequest at that.