如何使用的urllib2加载URL时设置TCP_NODELAY标志?(How to set TCP_

2019-07-04 11:38发布

我使用的urllib2加载网页,我的代码是:

httpRequest = urllib2.Request("http:/www....com")
pageContent = urllib2.urlopen(httpRequest)
pageContent.readline()

我怎样才能插座属性保持设置TCP_NODELAY

在正常的插座,我将使用功能:

socket.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

Answer 1:

如果您需要访问这些低级别属性使用的插座上,你就必须重载一些对象。

首先,你需要创建的子类的HttpHandler ,在标准库做:

class HTTPHandler(AbstractHTTPHandler):

    def http_open(self, req):
        return self.do_open(httplib.HTTPConnection, req)

    http_request = AbstractHTTPHandler.do_request_

正如你所看到的,它使用了HTTPConnection打开连接......你必须覆盖它太;)升级connect()方法。

像这样的东西应该是一个良好的开端:

class LowLevelHTTPConnection(httplib.HTTPConnection):

    def connect(self):
        httplib.HTTPConnection.connect(self)
        self.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)


class LowLevelHTTPHandler(HTTPHandler):

    def http_open(self, req):
        return self.do_open(LowLevelHTTPConnection, req)

urllib2的是足够聪明,让你继承了一些处理,然后用它的urllib2.build_opener针对此提出:

urllib2.install_opener(urllib2.build_opener(LowLevelHTTPHandler)) # tell urllib2 to use your HTTPHandler in replacement of the standard HTTPHandler
httpRequest = urllib2.Request("http:/www....com")
pageContent = urllib2.urlopen(httpRequest)
pageContent.readline()


Answer 2:

如果有需求,类似乎是在request.packages.urllib3 ; 有2个班,HttpConnection的,和HTTPSConnection。 他们应在模块顶层到位monkeypatchable:

from requests.packages.urllib3 import connectionpool

_HTTPConnection = connectionpool.HTTPConnection
_HTTPSConnection = connectionpool.HTTPSConnection

class HTTPConnection(_HTTPConnection):
    def connect(self):
        _HTTPConnection.connect(self)
        self.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

class HTTPSConnection(_HTTPSConnection):
    def connect(self):
        _HTTPSConnection.connect(self)
        self.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

connectionpool.HTTPConnection = HTTPConnection
connectionpool.HTTPSConnection = HTTPSConnection


Answer 3:

你必须使用的urllib2?

或者,你使用httplib2,其中有TCP_NODELAY选项设置。

https://code.google.com/p/httplib2/

它增加了一个依赖于你的项目,但似乎比猴子修补不太脆。



文章来源: How to set TCP_NODELAY flag when loading URL with urllib2?