I am making a multi-threaded [workers] application. Each thread should have it's own Non-GUI WebBrowser that Navigates to a web page and writes data to fields and click a button. I also need each WebBrowser to have it's own proxy. I tried the classic Windows.Forms.WebBrowser but I got stuck at the proxy part as it depends on IE global settings which won't work in my case. Any recommendations are welcome.
note: I tried doing it through HttpWebRequest/Response but it will never work as the data to be passed to the page contains a field called [ab_test_data] which gets its value from javascript code that calculate the value according to AB testing which I don't even fully understand. So a WebBrowser would be my best solution, unless someone can tell me how to convert that Javascript code that calculates ab_test_data to C# code. The algorithm used by the page I am trying to access is really sophisticated.
note2: ab_test_data value depends on Window.Event and Timestamp which can't be simulated on a httpWebRequest/Response.
note3: I tried Gecko, But it won't let me do anything to the webPage unless GeckoWebBrowser is drawn on the form (which I don't want).
Any solutions are welcome.
edit: If you know any WebBrowser that works like I want in any different language (Maybe Java) I would like to know.
Thanks in advance.
CefSharp: .Net binding for the Chromium Embedded Framework
use http://webkitdotnet.sourceforge.net/
As a question that may help, I wonder why browsers don't allow a proxy per (say) window/tab? I think a lot of it is because of lack of usefulness with respect to development time.
It may also be because the browsers [presumably] have centralized engines for things like web requests and caches, etc... Perhaps, allowing a proxy per window and/or tab would fundamentally alter the design of the modern browser and or have negative performance impacts. I don't really know. To illustrate the point further, consider things like Incognito mode and Private Browsing. In these cases, the browsers have, at least, conceptually made separate caches per windows...but I still bet an Incognito window and a standard window (in Chrome) use the same underlying web request engine.
Right now there are so many people who want a JavaScript and DOM parser and interpreter. Projects like the HtmlAgility Pack and Jint are helping, but there doesn't seem to be a unified and standard solution; at least not one with the simplicity of a web browser.
[rant below]...
Unfortunately, projects like Jint and HtmlAgility are worrisome. For one, they're not IE, Chrome, Safari or FireFox. You don't exactly know what you're getting yourself into. For instance, you know that in Chrome page xyz.com loads and renders perfectly. You can fire up FireFox and see that maybe something is not quite the same and so on with the other browsers. But, with these libraries you don't really know what if not everything is working right (there's no visual display to do a quick check). Plus, who knows what pace they're being developed at. Do they keep up with HTML5? Do they lag behind the major browsers? What about performance? Even more so, browsers already have things like caching and performance enhancements, which I doubt you'll get with individual libraries.
The best browser control would of course be something like:
IWebBrowser browser = new IE();
IWebBrowser browser = new Chrome();
IWebBrowser browser = new Safari();
IWebBrowser browser = new FireFox();
I think that is a dream, unfortunately. For one, what if you ever wanted to load plug-ins with these? What about user profiles, user logins, and so on? I think most of us just want the muscle of the browsers without these extras.
I really do hope that you find a good Chrome solution. I don't know what, if any, luck you'll have in the FireFox realm - maybe you can keep us updated? These solutions are evolving so quickly - I had never even heard of CefSharp or WebKit.NET before today and I looked for the same thing (Chrome and/or FireFox .NET browsers) several months ago for my own use. It would be great if a lot of people got together, made a standard interface and then each company built their embedded browser against the spec. Here's to wishing.