I am trying to make a crawler, and as the data is not showing in the page source, I can only execute the javascript with the web driver and get the response, and then do the data analysis. The script is simplified, like this, use Promise.
var res = ""
function f1() {
p = window.Promise
a = p.resolve(5).then(function(value) {
console.log(value)
res = value
return res
})
return a
}
console.log(f1()) // Promise object
console.log("result = ", res) // res is empty
My program is like this, writing with c#:
public void GetParameters(string url)
{
IWebDriver driver = new ChromeDriver();
driver.Navigate().GoToUrl(url);
IJavaScriptExecutor js = driver as IJavaScriptExecutor;
string script = readFile(@"C:\myscript.js", Encoding.UTF8).TrimEnd(); // The script is a bit long, so I save it to a local file.
var json = js.ExecuteScript(script);
Console.WriteLine("Get the return value");
Console.WriteLine(json);
driver.Close();
}
I want to get the value, and I know then
always return Promise object, so I define a variable, and want to store the value to it. But seems that, the Promise is executed asynchronous, so the res always be empty.
OK, I'm writing a crawler, and use Selenium to get the response from the server (Selenium.webdriver can open a web browser, and then execute script), I need get the result, as it will use by another program. So, I can not just add another .then, just output the value. Maybe I can save to a local file, and then read it, but I think it is inefficient.
Anyone can help?