Trouble installing Scrapy on Windows 64 bit

Question

I am attempting to install Scrapy on my Windows 7 64 bit machine. I started by following the instructions here on Scrapy's documentation.

I got up until the command 'pip install Scrapy'. Everything works except that it cannot find 'libxml2':

Could not find function xmlCheckVersion in library libxml2. Is libxml2     installed?

I then visited this website to get the binaries of libxml2:

ftp://ftp.zlatkovic.com/libxml/64bit/

The instructions for installation of libxml2 are here: https://www.zlatkovic.com/libxml.en.html

They state that you should unzip the binaries and place the contents of the BIN folder in a path such as C:\WINDOWS. I did this. However, after attempting to install Scrapy again, I continue to receive the same error. Is there something I am missing?

Answer 1

It is recommended to use conda to install Scrapy on Windows. You can download Miniconda from here: http://conda.pydata.org/miniconda.html

Once you have conda installed you can follow this guide to learn about its subcommands: http://conda.pydata.org/docs/test-drive.html

In short, you can use this commands in your terminal:

1. Create an environment for scrapy: conda create -n scrapyenv python=2
2. Enable the scrapy environment: activate scrapyenv
3. Install scrapy: conda install -c scrapinghub scrapy

Edit: Scrapy's conda package has been moved to conda-forge channel. Use the command: conda install -c conda-forge scrapy

Answer 2

Just installled Scrapy on Windows10 x64. It installs only with Python 2.7 on Windows. I used tutotial from here.

1. Download from the official site and install Python 2.7

2. Open Command promt (cmd) with full admin permissions and type:

   c:\python27\python.exe c:\python27\tools\scripts\win_add2path.py and press Enter

3. Install Microsoft Visual C++ for Python 2.7

4. Install OpenSSL (sorry, can't attach link because of small reputation). Download it to some folder, e.g. C:\Python27\Scripts and execute command from there: pip install pyOpenSSL-16.0.0-py2.py3-none-any.whl (the name of file could not be the same)

5. Install lxml just like previous file: pip install lxml-3.6.0-cp27-cp27m-win32.whl

6. Install Pywin32

7. And finally, install Scrapy - pip install Scrapy

Can recommend to download and install all needed soft for x32 architecture (even if you have x64 system).

Answer 3

Hi this is what I did to get it installed under Win7/Win10:

• easy_install lxml
• pip install Scrapy
• install pywin32 as indicated by the docs

Then, follow the tutorial to be sure everything is working