How to scrape page requiring cookies and javascrip

2019-08-13 19:05发布

Is there an easy way to emulate cookies and javascript with a php script scraping a web page requiring it?

The current response shows:

<body><noscript>This site requires JavaScript and Cookies to be enabled. Please change your browser settings or upgrade your browser.</noscript></body>

I put this in the code and it made no difference:

$strCookie = 'PHPSESSID=' . $_COOKIE['PHPSESSID'] . '; path=/';
curl_setopt( $ch, CURLOPT_COOKIE, $strCookie );

2条回答
走好不送
2楼-- · 2019-08-13 19:47

HTML inside the <noscript> </noscript> will only get displayed if Javascript is disabled. So this is the test for whether Javascript is enabled.

There is jQuery-cookie-plugin for reading, writing and deleting cookies. But it will not work if Javascript is disabled.

For checking if cookie is enabled with PHP code, you need to set the cookie and read the cookie with another request. This post have some guides on it.

查看更多
唯我独甜
3楼-- · 2019-08-13 19:54

Is there an easy way to emulate cookies and javascript with a php script scraping a web page requiring it?

No.

查看更多
登录 后发表回答