HTTP::Request in perl using LWP:UserAgent for auth

2019-09-17 04:49发布

问题:

I'm trying to log in one website using perl HTTP:Request and LWP:UserAgent. I added alI http headers which i found using Firebug and anyway get the error that CRFS Token is not defined

my $ua = LWP::UserAgent->new(keep_alive=>1);
$useragent->credentials('www.refer.org:80','','maila@gmail.com','pwd');
$request = HTTP::Request->new('POST','https://www.refer.org/account/signin', 
HTTP::Headers->new(<add all headers found in the header>));
$response = $useragent->request($request);
print $response->as_string;

headers found by Firebug:

Request URL:https://bla/login
Request Method:POST
Status Code:200 OK
Request Headersview source
Accept:*/*
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8,de;q=0.6
Connection:keep-alive
Content-Length:58
Content-Type:application/x-www-form-urlencoded
Cookie:logout=1364426556.61; sessionid=47b306354faa7357281a6cb1f0298df1;    maestro_user=%7B%22id%22%3A%22%22%2C%22email_address%22%3A%22%22%2C%22external_id%22%3A%226c104964ceb5d7ceb4575cab729ba7aa%22%2C%22photo_24%22%3A%22%22%2C%22photo_60%22%3A%22%22%2C%22photo_120%22%3A%22%22%2C%22display_name%22%3A%22%22%2C%22full_name%22%3A%22%22%2C%22privacy%22%3A100%2C%22groups%22%3A%5B%5D%2C%22is_superuser%22%3Afalse%2C%22is_staff%22%3Afalse%2C%22identity_verified%22%3Afalse%2C%22locale%22%3A%22en_US%22%2C%22timezone%22%3A%22%22%7D; __utma=158142248.1347071395.1348726747.1364423066.1364426537.88; __utmb=158142248.4.10.1364426537; __utmc=158142248; __utmz=158142248.1348726747.1.1.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided); csrftoken=zUZft9KwWmmogYbjR906daJB
Host:https://www.referer.org/
Origin:https://www.referer.org/
Referer:https://www.referer.org/account/signin
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.172 Safari/537.22
X-CSRFToken:zUZft9KwWmmogYbjR906daJB
X-Requested-With:XMLHttpRequest

here is a response header in the Firebug if anybody is interested

Response Headersview source
Cache-Control:no-cache, no-store, must-revalidate
Connection:keep-alive
Content-Encoding:gzip
Content-Length:725
Content-Type:application/json
Date:Wed, 27 Mar 2013 23:23:18 GMT
Server:nginx/1.2.6
Set-Cookie:sessionid=1ac9a133760f02c6fb8c61daebe7fc6d; expires=Wed, 10-Apr-2013 23:23:18 GMT; httponly; Max-Age=1209600; Path=/
Set- Cookie:maestro_login="cuPT1ZexESKY8gOQaLRRoBzxTnS0diEitb7Dy4g9h9FwfWO4PM5ppRYnQlLFM6++HX5TcA1lrrly5Fi/ie1bjw==|mRCAxgo374DL1N6yNRkDOh6Zony+s8InBTugfXb/ovuNff0LfudF6Z6mVP2qz2zxIgZ/kGUCbgRcb7+KUEvLPGY8AWBa2wCAV71fgUaAysm5NAPEaXV0k4C5ErQhOldAMVvyTspAR2PIXT+T2GY0mUGtUUTvZ1G2PI5knDjxQ2lnLuJNjEn0knrOA9bRspfAq8RwCl1cCSO5VjmrSquRlCEUf8MdUBD9Ea3abyKpDyfFx0vMBa2QMjxzOBYGqou8UPDizbjL4E6E5axmXl+wRt+QwpZNHASTh3l3h5Q90R2bWtLWlNQdC+mOlC4p0UXsQkIed9J7WXgQXpYbFNf6R7395LNJhr8mz0lQBWRimGBmqJCfpeKtYYACeH22QtXnRkgQxx44VmZ3XbaiKGKOdL7b/2kw9tJQxFZC/5bPQwemWxmJMfLW8YZtxdcugoKACnpyENjuxlHm7Ndt36KXKIq2rZdtwP8joLYpQQdkc6g="; expires=Fri, 26-Apr-2013 23:23:18 GMT; Max-Age=2592000; Path=/
Vary:Cookie
Vary:Accept-Encoding

and the response as i executed the perl code

HTTP/1.1 403 FORBIDDEN
Cache-Control: no-cache, no-store, must-revalidate
Connection: keep-alive
Date: Thu, 28 Mar 2013 07:17:48 GMT
Server: nginx/1.2.6
Vary: Accept-Encoding
Content-Length: 1006
Content-Type: text/html; charset=utf-8
Content-Type: text/html; charset=utf-8
Client-Date: Thu, 28 Mar 2013 07:17:48 GMT
Client-Peer: xxx
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /C=US/ST=Arizona/L=Scottsdale/O=GoDaddy.com,    
Inc./OU=http://certificates.godaddy.com/repository/CN=Go Daddy Secure Certification   
Authority/serialNumber=07969287
Client-SSL-Cert-Subject: /O=*.refer.org/OU=Domain Control Validated/CN=*.refer.org
Client-SSL-Cipher: AES256-SHA
Client-SSL-Warning: Peer certificate not verified
Title: 403 Forbidden
X-Meta-Robots: NONE,NOARCHIVE


<!DOCTYPE html>
<html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<meta name="robots" content="NONE,NOARCHIVE">
<title>403 Forbidden</title>
<style type="text/css">
html * { padding:0; margin:0; }
body * { padding:10px 20px; }
body * * { padding:0; }
body { font:small sans-serif; background:#eee; }
body>div { border-bottom:1px solid #ddd; }
h1 { font-weight:normal; margin-bottom:.4em; }
h1 span { font-size:60%; color:#666; font-weight:normal; }
#info { background:#f6f6f6; }
#info ul { margin: 0.5em 4em; }
#info p, #summary p { padding-top:10px; }
#summary { background: #ffc; }
#explanation { background:#eee; border-bottom: 0px none; }
</style>
</head>
<body>
<div id="summary">
<h1>Forbidden <span>(403)</span></h1>
<p>CSRF verification failed. Request aborted.</p>

</div>
<div id="explanation">
<p><small>More information is available with DEBUG=True.</small></p>
</div>

</body>
</html>

i didn't use 'https:// bla/login' as the website on the link is permanently unavailable

i will try WWW:Mechanism tomorrow. But here i wonder if there are any other way to define CSRF header?

回答1:

I see a lot of issues in your code:

  • You don't POST but GETing instead (you're not using your $req objuect!).
  • You don't sending cookies to the target site but Firefox sends some cookies.
  • You don't set Referer header too.

Life is short so use WWW::Mechanize instead plain LWP...

P. S. you can set headers like this:

$req->header("X-CSRFToken" => "zUZft9KwWmmogYbjR906daJB", Referer => 'http://ww
w.test.com/', "X-Requested-With" => "XMLHttpRequest");

emphasized text