Error installing scrapy in python 3.5 [duplicate]

Question

This question already has an answer here:

• python spyder conda install failure 2 answers
• Why does “pip install” inside Python raise a SyntaxError? 5 answers

I am installing Scrapy in Python 3.5. I am referring to the installation guide here .

I have installed pip and conda. I tried both ways mentioned in the guide, but am getting this error:

I have already installed pip and minoconda:

conda install -c scrapinghub scrapy

and

 pip install Scrapy

What is wrong with the installation?

Answer 1

You can pip install scrapy on python 3.5+, you just have to pip install all the deppendencies first - from here https://docs.scrapy.org/en/latest/intro/install.html

• pip install lxml
• pip install parsel
• pip install w3lib
• pip install twisted
• pip install cryptography
• pip install cryptography
• pip install pyOpenSSL

and finally

• pip install scrapy - worked for me

Answer 2

Before installing Scrapy on Python 3.5 on Windows 10 in virtualenv Install lxml,PyDispatcher then simpy install Scrapy code is attached.

    (pymote_env) C:\Users\DDSHARMA\Documents\pymote_env>pip install PyDispatcher
Collecting PyDispatcher
Installing collected packages: PyDispatcher
Successfully installed PyDispatcher-2.0.5

(pymote_env) C:\Users\DDSHARMA\Documents\pymote_env>pip install Scrapy
Collecting Scrapy
  Using cached Scrapy-1.2.1-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): six>=1.5.2 in c:\users\ddsharma\documents\pymote_env\lib\site-packages (from Scrapy)
Collecting service-identity (from Scrapy)
  Using cached service_identity-16.0.0-py2.py3-none-any.whl
Collecting Twisted>=10.0.0 (from Scrapy)
  Using cached Twisted-16.4.1.tar.bz2
Collecting parsel>=0.9.3 (from Scrapy)
  Using cached parsel-1.0.3-py2.py3-none-any.whl
Collecting pyOpenSSL (from Scrapy)
  Using cached pyOpenSSL-16.2.0-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): PyDispatcher>=2.0.5 in c:\users\ddsharma\documents\pymote_env\lib\site-packages (from Scrapy)
Requirement already satisfied (use --upgrade to upgrade): lxml in c:\users\ddsharma\documents\pymote_env\lib\site-packages (from Scrapy)
Collecting queuelib (from Scrapy)
  Using cached queuelib-1.4.2-py2.py3-none-any.whl
Collecting w3lib>=1.15.0 (from Scrapy)
  Using cached w3lib-1.15.0-py2.py3-none-any.whl
Collecting cssselect>=0.9 (from Scrapy)
  Using cached cssselect-1.0.0-py2.py3-none-any.whl
Collecting pyasn1-modules (from service-identity->Scrapy)
  Using cached pyasn1_modules-0.0.8-py2.py3-none-any.whl
Collecting pyasn1 (from service-identity->Scrapy)
  Using cached pyasn1-0.1.9-py2.py3-none-any.whl
Collecting attrs (from service-identity->Scrapy)
  Using cached attrs-16.2.0-py2.py3-none-any.whl
Collecting zope.interface>=4.0.2 (from Twisted>=10.0.0->Scrapy)
  Using cached zope.interface-4.3.2-cp35-cp35m-win32.whl
Collecting cryptography>=1.3.4 (from pyOpenSSL->Scrapy)
  Using cached cryptography-1.5.2-cp35-cp35m-win32.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools in c:\users\ddsharma\documents\pymote_env\lib\site-packages (from zope.interface>=4.0.2->Twisted>=10.0.0->Scrapy)
Collecting idna>=2.0 (from cryptography>=1.3.4->pyOpenSSL->Scrapy)
  Using cached idna-2.1-py2.py3-none-any.whl
Collecting cffi>=1.4.1 (from cryptography>=1.3.4->pyOpenSSL->Scrapy)
  Using cached cffi-1.8.3-cp35-cp35m-win32.whl
Collecting pycparser (from cffi>=1.4.1->cryptography>=1.3.4->pyOpenSSL->Scrapy)
  Downloading pycparser-2.16.tar.gz (230kB)
    100% |################################| 235kB 286kB/s
Building wheels for collected packages: Twisted, pycparser
  Running setup.py bdist_wheel for Twisted ... done
  Stored in directory: C:\Users\DDSHARMA\AppData\Local\pip\Cache\wheels\0e\53\62\e7b4cea7df9113fb2818b224eb5d143be981568d9c43057a0a
  Running setup.py bdist_wheel for pycparser ... done
  Stored in directory: C:\Users\DDSHARMA\AppData\Local\pip\Cache\wheels\34\cc\b5\34e805b130a9227f0681d5517cf554d950bb0361c3a373496f
Successfully built Twisted pycparser
Installing collected packages: pyasn1, pyasn1-modules, idna, pycparser, cffi, cryptography, pyOpenSSL, attrs, service-identity, zope.interface, Twisted, w3lib, cssselect, parsel, queuelib, Scrapy
Successfully installed Scrapy-1.2.1 Twisted-16.4.1 attrs-16.2.0 cffi-1.8.3 cryptography-1.5.2 cssselect-1.0.0 idna-2.1 parsel-1.0.3 pyOpenSSL-16.2.0 pyasn1-0.1.9 pyasn1-modules-0.0.8 pycparser-2.16 queuelib-1.4.2 service-identity-16.0.0 w3lib-1.15.0 zope.interface-4.3.2

(pymote_env) C:\Users\DDSHARMA\Documents\pymote_env>

Answer 3

Scrapy doesn't work with Python 3. At the moment, scrapy works only with Python 2.7.

Answer 4

You're trying to install scrapy in the python interpreter, you should install the package in your terminal. What's more, one requirement of installing scrapy is python 2.7, you're using python 3.5

---

Update:

Good news: Python 3 is Coming to Scrapy, you can try pip install scrapy==1.1.0rc1 now. Note that there are still some limitations, but won't be long that scrapy will have full of 3.x support. Pay close attention to the scrapy development ;)

Answer 5

I was able to install scrapy on Python 3.5.

1. Check to see if you are able to install these Twisted, lxml and pyOpenSSL.

2. In my case I got error in installing lxml. I fixed it by downloading lxml‑3.6.4‑cp35‑cp35m‑win_amd64.whl from http://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml based on your Python and windows version. (Credit to Christoph Gohlke's generosity for sharing this library.). Installed it using the following command:

---

pip install lxml‑3.6.4‑cp35‑cp35m‑win_amd64.whl