使用Web客户端使用socks代理(Use WebClient with socks proxy)

2019-07-03 11:08发布

有没有办法使用带有SOCKS代理WebClient ? 特异性地与DownloadString ,它提供的方法?

我不希望使用任何第三方的东西一样Privoxy的,freecap什么,我不能用商业库像那些从奇尔卡特。 我尝试使用的东西从http://www.mentalis.org/其实我用自己WebRequest实现,但他们似乎并没有对WebClient的类似的东西。

Answer 1:

SOCKS不被直接支持WebRequest / WebResponse类和推而广之,在Web客户端类(它依赖于WebRequest做它的工作)。

这真的不能,因为它工作在传输层(TCP / IP),而不是通过简单的重定向到HTTP请求转发(这是该级别的服务器WebRequest / WebResponse类上工作)。

您可以创建一个专门的推导WebRequest / WebResponse (使用ProxySocket处理低级别的握手,然后),然后创建一个专门WebClient类将覆盖GetWebRequestGetWebResponse方法。

一旦你有一个类你取代WebClient情况下,它应该正常工作(您可能需要设置代理在你使用它,虽然每种情况下)。



Answer 2:

这里是我终于实现了,谢谢casperOne的答案

public class SocksWebClient : WebClient
    {
        public IProxyDetails ProxyDetails { get; set; }
        public string UserAgent { get; set; }

        protected override WebRequest GetWebRequest(Uri address)
        {
            WebRequest result = null;

            if (ProxyDetails != null)
            {
                if (ProxyDetails.ProxyType == ProxyType.Proxy)
                {
                    result = (HttpWebRequest)WebRequest.Create(address);
                    result.Proxy = new WebProxy(ProxyDetails.FullProxyAddress);
                    if (!string.IsNullOrEmpty(UserAgent))
                        ((HttpWebRequest)result).UserAgent = UserAgent;
                }
                else if (ProxyDetails.ProxyType == ProxyType.Socks)
                {
                    result = SocksHttpWebRequest.Create(address);
                    result.Proxy = new WebProxy(ProxyDetails.FullProxyAddress);
                    //TODO: implement user and password

                }
                else if (ProxyDetails.ProxyType == ProxyType.None)
                {
                    result = (HttpWebRequest)WebRequest.Create(address);
                    if (!string.IsNullOrEmpty(UserAgent))
                        ((HttpWebRequest)result).UserAgent = UserAgent;
                }
            }
            else
            {
                result = (HttpWebRequest)WebRequest.Create(address);
                if (!string.IsNullOrEmpty(UserAgent))
                    ((HttpWebRequest)result).UserAgent = UserAgent;
            }


            return result;
        }

    }

所述SocksHttpWebRequest类是从所拍摄的博客链接到由@casperOne,码为哪些如下:

using System;
using System.Collections.Specialized;
using System.IO;
using System.Net;
using System.Net.Sockets;
using System.Text;
using Org.Mentalis.Network.ProxySocket;

namespace Ditrans
{
    public class SocksHttpWebRequest : WebRequest
    {

        #region Member Variables

        private readonly Uri _requestUri;
        private WebHeaderCollection _requestHeaders;
        private string _method;
        private SocksHttpWebResponse _response;
        private string _requestMessage;
        private byte[] _requestContentBuffer;

        // darn MS for making everything internal (yeah, I'm talking about you, System.net.KnownHttpVerb)
        static readonly StringCollection validHttpVerbs =
            new StringCollection { "GET", "HEAD", "POST", "PUT", "DELETE", "TRACE", "OPTIONS" };

        #endregion

        #region Constructor

        private SocksHttpWebRequest(Uri requestUri)
        {
            _requestUri = requestUri;
        }

        #endregion

        #region WebRequest Members

        public override WebResponse GetResponse()
        {
            if (Proxy == null)
            {
                throw new InvalidOperationException("Proxy property cannot be null.");
            }
            if (String.IsNullOrEmpty(Method))
            {
                throw new InvalidOperationException("Method has not been set.");
            }

            if (RequestSubmitted)
            {
                return _response;
            }
            _response = InternalGetResponse();
            RequestSubmitted = true;
            return _response;
        }

        public override Uri RequestUri
        {
            get { return _requestUri; }
        }

        public override IWebProxy Proxy { get; set; }

        public override WebHeaderCollection Headers
        {
            get
            {
                if (_requestHeaders == null)
                {
                    _requestHeaders = new WebHeaderCollection();
                }
                return _requestHeaders;
            }
            set
            {
                if (RequestSubmitted)
                {
                    throw new InvalidOperationException("This operation cannot be performed after the request has been submitted.");
                }
                _requestHeaders = value;
            }
        }

        public bool RequestSubmitted { get; private set; }

        public override string Method
        {
            get
            {
                return _method ?? "GET";
            }
            set
            {
                if (validHttpVerbs.Contains(value))
                {
                    _method = value;
                }
                else
                {
                    throw new ArgumentOutOfRangeException("value", string.Format("'{0}' is not a known HTTP verb.", value));
                }
            }
        }

        public override long ContentLength { get; set; }

        public override string ContentType { get; set; }

        public override Stream GetRequestStream()
        {
            if (RequestSubmitted)
            {
                throw new InvalidOperationException("This operation cannot be performed after the request has been submitted.");
            }

            if (_requestContentBuffer == null)
            {
                _requestContentBuffer = new byte[ContentLength];
            }
            else if (ContentLength == default(long))
            {
                _requestContentBuffer = new byte[int.MaxValue];
            }
            else if (_requestContentBuffer.Length != ContentLength)
            {
                Array.Resize(ref _requestContentBuffer, (int) ContentLength);
            }
            return new MemoryStream(_requestContentBuffer);
        }

        #endregion

        #region Methods

        public static new WebRequest Create(string requestUri)
        {
            return new SocksHttpWebRequest(new Uri(requestUri));
        }

        public static new WebRequest Create(Uri requestUri)
        {
            return new SocksHttpWebRequest(requestUri);
        }

        private string BuildHttpRequestMessage()
        {
            if (RequestSubmitted)
            {
                throw new InvalidOperationException("This operation cannot be performed after the request has been submitted.");
            }

            var message = new StringBuilder();
            message.AppendFormat("{0} {1} HTTP/1.0\r\nHost: {2}\r\n", Method, RequestUri.PathAndQuery, RequestUri.Host);

            // add the headers
            foreach (var key in Headers.Keys)
            {
                message.AppendFormat("{0}: {1}\r\n", key, Headers[key.ToString()]);
            }

            if (!string.IsNullOrEmpty(ContentType))
            {
                message.AppendFormat("Content-Type: {0}\r\n", ContentType);
            }
            if (ContentLength > 0)
            {
                message.AppendFormat("Content-Length: {0}\r\n", ContentLength);
            }

            // add a blank line to indicate the end of the headers
            message.Append("\r\n");

            // add content
            if(_requestContentBuffer != null && _requestContentBuffer.Length > 0)
            {
                using (var stream = new MemoryStream(_requestContentBuffer, false))
                {
                    using (var reader = new StreamReader(stream))
                    {
                        message.Append(reader.ReadToEnd());
                    }
                }
            }

            return message.ToString();
        }

        private SocksHttpWebResponse InternalGetResponse()
        {
            var response = new StringBuilder();
            using (var _socksConnection =
                new ProxySocket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp))
            {
                var proxyUri = Proxy.GetProxy(RequestUri);
                var ipAddress = GetProxyIpAddress(proxyUri);
                _socksConnection.ProxyEndPoint = new IPEndPoint(ipAddress, proxyUri.Port);
                _socksConnection.ProxyType = ProxyTypes.Socks5;

                // open connection
                _socksConnection.Connect(RequestUri.Host, 80);
                // send an HTTP request
                _socksConnection.Send(Encoding.ASCII.GetBytes(RequestMessage));
                // read the HTTP reply
                var buffer = new byte[1024];

                var bytesReceived = _socksConnection.Receive(buffer);
                while (bytesReceived > 0)
                {
                    response.Append(Encoding.ASCII.GetString(buffer, 0, bytesReceived));
                    bytesReceived = _socksConnection.Receive(buffer);
                }
            }
            return new SocksHttpWebResponse(response.ToString());
        }

        private static IPAddress GetProxyIpAddress(Uri proxyUri)
        {
            IPAddress ipAddress;
            if (!IPAddress.TryParse(proxyUri.Host, out ipAddress))
            {
                try
                {
                    return Dns.GetHostEntry(proxyUri.Host).AddressList[0];
                }
                catch (Exception e)
                {
                    throw new InvalidOperationException(
                        string.Format("Unable to resolve proxy hostname '{0}' to a valid IP address.", proxyUri.Host), e);
                }
            }
            return ipAddress;
        }

        #endregion

        #region Properties

        public string RequestMessage
        {
            get
            {
                if (string.IsNullOrEmpty(_requestMessage))
                {
                    _requestMessage = BuildHttpRequestMessage();
                }
                return _requestMessage;
            }
        }

        #endregion

    }
}

需要注意的是,作为@casperOne指出,这使得使用(免费)的第三方调用的库的ProxySocket 。



Answer 3:

我碰到这个藏汉,发现漂亮的BetterHttpClient

它从WebClient的派生,并允许您指定一个socks代理:

BetterHttpClient.HttpClient client = new BetterHttpClient.HttpClient(new BetterHttpClient.Proxy("IP address", port, BetterHttpClient.ProxyTypeEnum.Socks));


Answer 4:

这里是我终于实现了,谢谢casperOne的答案

public class SocksWebClient : WebClient
{
    public IProxyDetails ProxyDetails { get; set; }
    public string UserAgent { get; set; }

    protected override WebRequest GetWebRequest(Uri address)
    {
        WebRequest result = null;

        if (ProxyDetails != null)
        {
            if (ProxyDetails.ProxyType == ProxyType.Proxy)
            {
                result = (HttpWebRequest)WebRequest.Create(address);
                result.Proxy = new WebProxy(ProxyDetails.FullProxyAddress);
                if (!string.IsNullOrEmpty(UserAgent))
                    ((HttpWebRequest)result).UserAgent = UserAgent;
            }
            else if (ProxyDetails.ProxyType == ProxyType.Socks)
            {
                result = SocksHttpWebRequest.Create(address);
                result.Proxy = new WebProxy(ProxyDetails.FullProxyAddress);
                //TODO: implement user and password

            }
            else if (ProxyDetails.ProxyType == ProxyType.None)
            {
                result = (HttpWebRequest)WebRequest.Create(address);
                if (!string.IsNullOrEmpty(UserAgent))
                    ((HttpWebRequest)result).UserAgent = UserAgent;
            }
        }
        else
        {
            result = (HttpWebRequest)WebRequest.Create(address);
            if (!string.IsNullOrEmpty(UserAgent))
                ((HttpWebRequest)result).UserAgent = UserAgent;
        }

        return result;
    }
}

我在寻找相同的代码,但它并没有真正为我工作。 我不知道你有IProxyDetails -object。 它似乎不是在博客,而不是在我从那里得到的文件执行



文章来源: Use WebClient with socks proxy