PHP and rotating proxies

Question

Has anyone of you tried using rotating proxies? How easy is this to implement? Does it work well? Your experience please

PS: i see questions like "how to make php script use a list of proxies" gather a lot of minuses. Can you explain this before putting -1?

Answer 1

------ UPDATED 04 March 2017 -------

---

I've been there and found that the best solution was:

If you don't have a dedicated server or at least a vps and a little patience don't bother to read the rest of the post ...

1 - Install Squid 3.2 from source (check notes below)
2 - Add a list of 20 or so ip's to squid.conf (costs about 25$ month)
3 - Use the the new feature ACLrandom to rotate the outgoing ip.

This way, you don't need to rotate the list of ip's on your php script, instead, you'll be connecting to the same ip (ex: 192.168.1.1:3129) but the visible outgoing ip (tcp_outgoing_address) will rotate on every request based on the random settings .

You'll need to compile squid 3.2 with '-enable-http-violations' in order to make it an elite anonymous proxy.

Step by Step Install:

yum -y groupinstall 'Development Tools'
yum -y install openssl-devel
wget http://www.squid-cache.org/Versions/v3/3.2/squid-3.2.13.tar.gz
tar -xvf squid-3.2.13.tar.gz
cd squid-3.2.13
./configure -prefix=/squid32 '--enable-removal-policies=heap,lru' '--enable-ssl' '--with-openssl' '--enable-linux-netfilter' '--with-pthreads' '--enable-ntlm-auth-helpers=SMB,fakeauth' '--enable-external-acl-helpers=ip_user,ldap_group,unix_group,wbinfo_group' '--enable-auth-basic' '--enable-auth-digest' '--enable-auth-negotiate' '--enable-auth-ntlm' '--with-winbind-auth-challenge' '--enable-useragent-log' '--enable-referer-log' '--disable-dependency-tracking' '--enable-cachemgr-hostname=localhost' '--enable-underscores' '--enable-build-info' '--enable-cache-digests' '--enable-ident-lookups' '--enable-follow-x-forwarded-for' '--enable-wccpv2' '--enable-fd-config' '--with-maxfd=16384' '-enable-http-violations'
make
make install

Sample squid.conf (located in this case /squid32/etc/squid.conf) :

#this will be the ip and port where squid will run
http_port 5.5.5.5:33333 # change this ip and port ...

#Extra parameters on squid.conf to make **an elite proxy**

request_header_access Allow allow all 
request_header_access Authorization allow all 
request_header_access WWW-Authenticate allow all 
request_header_access Proxy-Authorization allow all 
request_header_access Proxy-Authenticate allow all 
request_header_access Cache-Control allow all 
request_header_access Content-Encoding allow all 
request_header_access Content-Length allow all 
request_header_access Content-Type allow all 
request_header_access Date allow all 
request_header_access Expires allow all 
request_header_access Host allow all 
request_header_access If-Modified-Since allow all 
request_header_access Last-Modified allow all 
request_header_access Location allow all 
request_header_access Pragma allow all 
request_header_access Accept allow all 
request_header_access Accept-Charset allow all 
request_header_access Accept-Encoding allow all 
request_header_access Accept-Language allow all 
request_header_access Content-Language allow all 
request_header_access Mime-Version allow all 
request_header_access Retry-After allow all 
request_header_access Title allow all 
request_header_access Connection allow all 
request_header_access Proxy-Connection allow all 
request_header_access User-Agent allow all 
request_header_access Cookie allow all 
request_header_access All deny all 

via off
forwarded_for off
follow_x_forwarded_for deny all

acl vinte1 random 1/5 
acl vinte2 random 1/5
acl vinte3 random 1/5
acl vinte4 random 1/5
acl vinte5 random 1/5

tcp_outgoing_address 1.1.1.1 vinte1 # fake ip's , replace with yours
tcp_outgoing_address 1.1.1.2 vinte2
tcp_outgoing_address 1.1.1.3 vinte3
tcp_outgoing_address 1.1.1.4 vinte4
tcp_outgoing_address 1.1.1.5 vinte5

tcp_outgoing_address 1.1.1.6 # this will be the default tcp outgoing address

Sample PHP CURL request using a squid proxy:

$proxy = "1.1.1.1:33333";
$useragent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1";
$url = "https://api.ipify.org/";

$ch = curl_init();
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT,15);
curl_setopt($ch, CURLOPT_HTTP_VERSION,'CURL_HTTP_VERSION_1_1' );
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
curl_setopt($ch, CURLOPT_PROXY, $proxy);
curl_setopt($ch, CURLOPT_PROXYUSERPWD,'USER:PASS');
curl_setopt($ch, CURLOPT_USERAGENT,$useragent);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION,1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER,0);
$result=curl_exec ($ch);
curl_close ($ch);
echo $result

Usefull Links:
Squid 3.2 Source : http://www.squid-cache.org/Versions/v3/3.2/squid-3.2.13.tar.gz
Rotating_three_IPs : http://wiki.squid-cache.org/ConfigExamples/Strange/RotatingIPs#Example:_Rotating_three_IPs_based_on_time_of_day
AclRandom : http://wiki.squid-cache.org/Features/AclRandom
Installing Squid 3.2 on CentOS 5.3 - http://www.guldmyr.com/blog/installing-squid-3-2-on-centos-5-3/
Add password to Squid: How to set up a squid Proxy with basic username and password authentication?

I've found this to be the most reliable and secure way to rotate proxies because you don't rely on third party proxy providers and your information (passwords, data, etc) will be safer. It may sound a little difficult to setup at first but it will pay off every second you've spent, GL :)

Answer 2

PHP Curl supports a lot of proxy commands.

CURLOPT_PROXYAUTH CURLOPT_PROXYPORT CURLOPT_PROXYTYPE CURLOPT_PROXY CURLOPT_PROXY_SERVICE_NAME CURLOPT_PROXYUSERPWD CURLOPT_PROXYHEADER CURLOPT_HTTPPROXYTUNNEL

See for more information: http://php.net/manual/en/function.curl-setopt.php

Simple example below.

$proxy      =   array();
$proxy[]    =   '1.2.3.4';
$proxy[]    =   '5.6.7.8';

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_PROXY, $proxy[array_rand($proxy)]);

curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 15);


$result =   curl_exec($ch);
curl_close($ch);