PHP cURL: HTTP headers show 302 and cookies set, c

Question

This is kind of a carry on from a question asked yesterday: Can't seem to get a web page's contents via cURL - user agent and HTTP headers both set?

I'm attempting to access a url's contents, the problem is the way this url handles request.

The url: http://www.deindeal.ch/deals/atlas-grand-hotel-2-naechte-30-2/

First request (without cookies):

After "learning" to use curl in the command line (props to @d3v3us), a simple request curl -i http://www.deindeal.ch/deals/atlas-grand-hotel-2-naechte-30-2/ shows the following:

curl -i http://www.deindeal.ch/deals/atlas-grand-hote
l-2-naechte-30-2/
HTTP/1.1 302 FOUND
Date: Fri, 30 Dec 2011 13:15:00 GMT
Server: Apache/2.2.16 (Debian)
Vary: Accept-Language,Cookie,Accept-Encoding
Content-Language: de
Set-Cookie: csrftoken=edc8c77fc74f5e788c53488afba4e50a; Domain=www.deindeal.ch;
Max-Age=31449600; Path=/
Set-Cookie: generic_cookie=1; Path=/
Set-Cookie: sessionid=740a8a2cb9fb51166dcf865e35b91888; expires=Fri, 27-Jan-2012
 13:15:00 GMT; Max-Age=2419200; Path=/
Location: http://www.deindeal.ch/welcome/?deal_slug=atlas-grand-hotel-2-naechte-
30-2
Content-Length: 0
Connection: close
Content-Type: text/html; charset=utf-8

Second request (with cookies):

So, I save the cookie using -c, check that it saves as cookie.txt, and run the request again with the addition of -b cookie.txt, getting this:

curl -i -b cookie.txt http://www.deindeal.ch/deals/atlas-grand-hotel-2-naechte-3
0-2/
HTTP/1.1 302 FOUND
Date: Fri, 30 Dec 2011 13:38:17 GMT
Server: Apache/2.2.16 (Debian)
Vary: Accept-Language,Cookie,Accept-Encoding
Content-Language: de
Set-Cookie: csrftoken=49f5c804d399f8581253630631692f5f; Domain=www.deindeal.ch; Max-Age=31449600; P
ath=/
Location: http://www.deindeal.ch/welcome/?deal_slug=atlas-grand-hotel-2-naechte-30-2
Content-Length: 0
Connection: close
Content-Type: text/html; charset=utf-8

To me this looks like exactly the same contents, minus one or two parameters in the cookie, but maybe I'm overlooking something?

I'm attempting to get the curl request to function and return the same contents as when requesting that url via a browser, but I'm not sure what I should do next.

Note: I've tagged this PHP, as I am using PHP to make the requests, I've simply using command line to easily show the returned headers - so if there's any other PHP libraries or methods that would work (better, or in a place that cURL wouldn't), please feel free to suggest any.

Any help would be greatly appreciated ;).

Answer 1

I may be misunderstanding your question, but a 302 response means content found, and you just need to follow the "Location" right? cUrl will only perform one request, unlike your browser which will see that 302 (set the cookies, just like you're doing) then follow that location header. It looks like your location has a "?" in it that isn't in the original. Run cUrl, with that same cookie jar, on the Location url.

http://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_Redirection

Answer 2

You need this,

curl -iL  -c cookie.txt -b cookie.txt http://www.deindeal.ch/deals/atlas-grand-hotel-2-naechte-3

-b flag is used to read cookie from . For a file to be used to save cookie after the http transaction use -c flag. Its called cookie jar.

Using WebGet (Sorry, Its written by me) pulling the contents is quite simple.

require "WebGet.php";
$w = new WebGet();
$w->cookieFile = 'cookie.txt'; // must be writable
$w->requestContent("https://github.com/shiplu/dxtool");
print_r($w->responseHeaders) // prints response headers
print_r($w->cachedContent) // prints url content