certificate verify failed: unable to get local iss

问题:

I am trying to get data from the web using python. I imported urllib.request package for it but while executing, I get error:

certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)

• I am using Python 3.7 on Mac OS High Sierra.
• I am trying to get CSV file from: https://s3.amazonaws.com/assets.datacamp.com/production/course_1606/datasets/winequality-red.csv

When I changed the URL to 'http' - I am able to get data. But, I believe, this avoids checking SSL certificate.

So I checked on the internet and found one solution: Run /Applications/Python\ 3.7/Install\ Certificates.command

This solved my problem. But I have no knowledge on SSL and the likes. Can you help me understand what it actually did to solve my issue.

If possible, please recommend me any good resource to learn about the security and certificates. I am new to this.

Thanks!

Note: I did go through the link - openssl, python requests error: "certificate verify failed"

My question differs from the one in link because, I want to know what actually happens when I install certifi package or run Install\ Certificates.command to fix the error. I have a poor understanding of securities.

回答1:

For anyone who still wonders on how to fix this, i got mine by installing the "Install Certificates.command"

Here is how I did,

Just double click on that file wait for it to install and in my case, you will be ready to go

回答2:

I hit the same issue on OSX, while my code was totally fine on Linux, and you gave the answer in your question!

After inspecting the file you pointed to /Applications/Python 3.7/Install Certificates.command, it turned out that what this command replaces the root certificates of the default Python installation with the ones shipped through the certifi package.

certifi is a set of root certificates. Each SSL certificate relies a chain of trust: you trust one specific certificate because you trust the parent of that certificate, for which you trust the parent, etc. At some point, there is no "parent" and those are "root" certificates. For those, there is no other solution than bundling commonly trusted root certificates (usually big trust companies like eg. "DigiCert").

You can for instance see the root certificates in your browser security settings (for instance for Firefox->Preference->Privacy and security->view certificates->Authorities).

Coming back to the initial problem, and prior to running the .command file, executing this returns for me an empty list on a clean installation:

import os
import ssl                                        
openssl_dir, openssl_cafile = os.path.split(      
    ssl.get_default_verify_paths().openssl_cafile)
# no content in this folder
os.listdir(openssl_dir)
# non existent file
print(os.path.exists(openssl_cafile))

This means that there is no default certificate authority for the Python installation on OSX. A possible default is exactly the one provided by the certifi package.

After that, you just can create an SSL context that has the proper default as the following (certifi.where() gives the location of a certificate authority):

import platform
# ...

ssl_context = ssl.SSLContext(ssl.PROTOCOL_TLS)
ssl_context.verify_mode = ssl.CERT_REQUIRED
ssl_context.check_hostname = True
ssl_context.load_default_certs()

if platform.system().lower() == 'darwin':
    import certifi
    ssl_context.load_verify_locations(
        cafile=os.path.relpath(certifi.where()),
        capath=None,
        cadata=None)

and make request to an url from python like this:

import urllib
# previous context
https_handler = urllib.request.HTTPSHandler(context=ssl_context)

opener = urllib.request.build_opener(https_handler)
ret = opener.open(url, timeout=2)

回答3:

I would like to provide a reference. I use cmd + space, then type Install Certificates.command, and then press Enter. After a short while, the command line interface pops up to start the installation.

 -- removing any existing file or link
 -- creating symlink to certifi certificate bundle
 -- setting permissions
 -- update complete

Finally, it fixes the errors.

回答4:

I had the error with conda on linux. My solution was simple.

conda install -c conda-forge certifi

I had to use the conda forge since the default certifi appears to have problems.

回答5:

This worked in all OS:

import ssl
import certifi

urlopen(request, context=ssl.create_default_context(cafile=certifi.where()))