pip, proxy authentication and “Not supported proxy

Question

Trying to install pip on a new python installation. I am stuck with proxy errors. Looks like a bug in get-pip or urllib3??

Question is do I have to go through the pain of setting up CNTLM as described here or is there a shortcut?

get-pip.py documentation says use --proxy="[user:passwd@]proxy.server:port" option to specify proxy and relevant authentication. But seems like pip passes on the whole thing as it is to urllib3 which interprets "myusr" as the url scheme, because of the ':' I guess (?).

C:\ProgFiles\Python27>get-pip.py --proxy myusr:mypswd@111.222.333.444:80
Downloading/unpacking pip
Cleaning up...
Exception:
Traceback (most recent call last):
  File "c:\users\sg0219~1\appdata\local\temp\tmpxwg_en\pip.zip\pip\basecommand.py", line 122, in main
    status = self.run(options, args)
  File "c:\users\sg0219~1\appdata\local\temp\tmpxwg_en\pip.zip\pip\commands\install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "c:\users\sg0219~1\appdata\local\temp\tmpxwg_en\pip.zip\pip\req.py", line 1177, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "c:\users\sg0219~1\appdata\local\temp\tmpxwg_en\pip.zip\pip\index.py", line 194, in find_requirement
    page = self._get_page(main_index_url, req)
  File "c:\users\sg0219~1\appdata\local\temp\tmpxwg_en\pip.zip\pip\index.py", line 568, in _get_page
    session=self.session,
  File "c:\users\sg0219~1\appdata\local\temp\tmpxwg_en\pip.zip\pip\index.py", line 670, in get_page
    resp = session.get(url, headers={"Accept": "text/html"})
  File "c:\users\sg0219~1\appdata\local\temp\tmpxwg_en\pip.zip\pip\_vendor\requests\sessions.py", line 468, in get
    return self.request('GET', url, **kwargs)
  File "c:\users\sg0219~1\appdata\local\temp\tmpxwg_en\pip.zip\pip\download.py", line 237, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "c:\users\sg0219~1\appdata\local\temp\tmpxwg_en\pip.zip\pip\_vendor\requests\sessions.py", line 456, in request
    resp = self.send(prep, **send_kwargs)
  File "c:\users\sg0219~1\appdata\local\temp\tmpxwg_en\pip.zip\pip\_vendor\requests\sessions.py", line 559, in send
    r = adapter.send(request, **kwargs)
  File "c:\users\sg0219~1\appdata\local\temp\tmpxwg_en\pip.zip\pip\_vendor\requests\adapters.py", line 305, in send
    conn = self.get_connection(request.url, proxies)
  File "c:\users\sg0219~1\appdata\local\temp\tmpxwg_en\pip.zip\pip\_vendor\requests\adapters.py", line 215, in get_connection
    block=self._pool_block)
  File "c:\users\sg0219~1\appdata\local\temp\tmpxwg_en\pip.zip\pip\_vendor\requests\packages\urllib3\poolmanager.py", line 258, in proxy_fro
m_url
    return ProxyManager(proxy_url=url, **kw)
  File "c:\users\sg0219~1\appdata\local\temp\tmpxwg_en\pip.zip\pip\_vendor\requests\packages\urllib3\poolmanager.py", line 214, in __init__
    'Not supported proxy scheme %s' % self.proxy.scheme
AssertionError: Not supported proxy scheme myusr

Storing debug log for failure in C:\Users\myusr\pip\pip.log

C:\ProgFiles\Python27>

When I run the command without the usrname and password it works fine, but proxy rejects the request saying it needs authentication ("407 authenticationrequired").

C:\ProgFiles\Python27>get-pip.py --proxy 111.222.333.444:80
Downloading/unpacking pip
  Cannot fetch index base URL https://pypi.python.org/simple/
  Could not find any downloads that satisfy the requirement pip
Cleaning up...
No distributions at all found for pip
Storing debug log for failure in C:\Users\sg0219898\pip\pip.log

C:\ProgFiles\Python27>cat C:\Users\sg0219898\pip\pip.log
------------------------------------------------------------
C:\ProgFiles\Python27\get-pip.py run on 09/29/14 16:23:26
Downloading/unpacking pip
  Getting page https://pypi.python.org/simple/pip/
  Could not fetch URL https://pypi.python.org/simple/pip/: connection error: ('Cannot connect to proxy.', error('Tunnel connection failed: 407 authenticationrequired',))
  Will skip URL https://pypi.python.org/simple/pip/ when looking for download links for pip
  Getting page https://pypi.python.org/simple/
  Could not fetch URL https://pypi.python.org/simple/: connection error: ('Cannot connect to proxy.', error('Tunnel connection failed: 407 authenticationrequired',))
  Will skip URL https://pypi.python.org/simple/ when looking for download links for pip
  Cannot fetch index base URL https://pypi.python.org/simple/
  URLs to search for versions for pip:
  * https://pypi.python.org/simple/pip/
  Getting page https://pypi.python.org/simple/pip/
  Could not fetch URL https://pypi.python.org/simple/pip/: connection error: ('Cannot connect to proxy.', error('Tunnel connection failed: 407 authenticationrequired',))
  Will skip URL https://pypi.python.org/simple/pip/ when looking for download links for pip
  Could not find any downloads that satisfy the requirement pip
Cleaning up...
  Removing temporary dir c:\users\sg0219~1\appdata\local\temp\pip_build_SG0219898...
No distributions at all found for pip
Exception information:
Traceback (most recent call last):
  File "c:\users\sg0219~1\appdata\local\temp\tmp36ynxd\pip.zip\pip\basecommand.py", line 122, in main
    status = self.run(options, args)
  File "c:\users\sg0219~1\appdata\local\temp\tmp36ynxd\pip.zip\pip\commands\install.py", line 278, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "c:\users\sg0219~1\appdata\local\temp\tmp36ynxd\pip.zip\pip\req.py", line 1177, in prepare_files
    url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
  File "c:\users\sg0219~1\appdata\local\temp\tmp36ynxd\pip.zip\pip\index.py", line 277, in find_requirement
    raise DistributionNotFound('No distributions at all found for %s' % req)
DistributionNotFound: No distributions at all found for pip

C:\ProgFiles\Python27>

I had a brief look at urllib3\poolmanager.py and it doesn't seem to have anything to do with username/passwords.

Answer 1

For the problem you have mentioned, it depends on how the proxy server authentication has been set up. For example, my intranet uses Windows AD and probably the proxy server is using Windows Integrated auth. Thus when I do pip install --proxy http://<server-ip>:<port> <module-name>, it works fine. Note that I did not have to type in username & password ,likely due to integrated auth.

So, you'll need to find out the authentication being used by your proxy server.You can use Fiddler(or any other network analyzer) tool to check the WWW-Authenticate headers in the 407 response, to check what auth mechs the server supports.

Answer 2

The below things worked for me :

Step 1) Set HTTPS_PROXY and HTTP_PROXY:

export HTTPS_PROXY="http://proxy-address:8080"

export HTTP_PROXY="http://proxy-address:8080"

step 2) : Run pip with below configuration

./pip.exe install pyspark --trusted-host pypi.python.org --trusted-host files.pythonhosted.org --trusted-host pypi.org --proxy "proxy-address:8080"

Answer 3

This is complaining about the scheme for the URL (which urlparse is understanding to be myusr), to work around that you should instead be doing:

get-pip.py --proxy http://myusr:mypswd@111.222.333.444:80

Answer 4

This is because the script requires the environment variables http_proxy or https_proxy contains the scheme in the URL. Set the environment variables

 export http_proxy="http://<hostname>:<port>"
 export https_proxy="https://<hostname>:<port>"

before run the "python get-pip.py"

Answer 5

This can be a case when you use ";" in your passwords. This cannot be parsed properly that results in ProxySchemeUnknown error raised.