I have a request over tor that used to worked in python3.6. I switched to python3.7 and now I get 403 response status.
Below is a way to replicate the issue with conda environments and requests:
In python3.6:
conda create -n python3.6 python=3.6.5
conda activate python3.6
conda install requests
conda install ipython[all]
ipython
import requests
session = requests.session()
session.proxies = {}
session.proxies['http'] = 'socks5h://localhost:9050'
session.proxies['https'] = 'socks5h://localhost:9050'
url='https://dubicars.com'
session.get(url, headers={'User-Agent': "Mozilla/5.0 (Windows NT 6.1; rv:10.0) Gecko/20100101 Firefox/10.0"})
Return: <Response [200]>
In python3.7:
conda create -n python3.7
conda activate python3.7
conda install requests
conda install ipython[all]
ipython
import requests
session = requests.session()
session.proxies = {}
session.proxies['http'] = 'socks5h://localhost:9050'
session.proxies['https'] = 'socks5h://localhost:9050'
url='https://dubicars.com'
session.get(url, headers={'User-Agent': "Mozilla/5.0 (Windows NT 6.1; rv:10.0) Gecko/20100101 Firefox/10.0"})
Return: <Response [403]>
Both should result in the same response (if successful without timeouts or interruptions), but I get a 200 in the first and a 403 in the second. Any explanations and workarounds?
This happens to a few other sites as well.
Diffing the packages of each environment I see only 3 version differences one of which is obviously python:
cryptography 2.3.1
openssl 1.0.2t
python 3.6.5
cryptography 2.7
openssl 1.1.1d
python 3.7.4
Edit: I downgraded openssl to 1.0.2t, it downgraded cryptography to the same version as python 3.6.5 above and downgraded python to 3.7.0, and upgrading python will upgrade along openssl and cryptography. With the downgrade python 3.7 request worked fine.
I believe the issue is from openssl, so is this an intended behaviour? Should I set up the request differently?
