install scrapy on win 7 (64-bit)

Question

I'm trying to install scrapy for python2.6, but it seems not going well. Here is the packages installed:

G:\Python26\Scripts>pip freeze
Scrapy==0.16.4
Twisted==12.3.0
libxml2-python==2.7.7
lxml==2.3.6
pyopenssl==0.13
w3lib==1.2
zope.interface==3.8.0

I also got iconv and zlib. And this is the log when installing scrapy with pip. I don't know what I should do next, am I missing sth? Need instructions, thank you.

win 7 64-bit, Visual C++ installed

C:\Users\d>pip install scrapy
Downloading/unpacking scrapy
  Downloading Scrapy-0.16.4.tar.gz (679kB): 679kB downloaded
  Running setup.py egg_info for package scrapy
    warning: no files found matching 'license.txt' under directory 'scrapy'
    no previously-included directories found matching 'docs\build'
Requirement already satisfied (use --upgrade to upgrade): Twisted>=8.0 in g:\pyt
hon26\lib\site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): w3lib>=1.2 in g:\pytho
n26\lib\site-packages (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): lxml in g:\python26\li
b\site-packages\lxml-2.3.6-py2.6-win32.egg (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): pyOpenSSL in g:\python
26\lib\site-packages\pyopenssl-0.13-py2.6-win32.egg (from scrapy)
Requirement already satisfied (use --upgrade to upgrade): zope.interface>=3.6.0
in g:\python26\lib\site-packages\zope.interface-3.8.0-py2.6-win32.egg (from Twis
ted>=8.0->scrapy)
Requirement already satisfied (use --upgrade to upgrade): setuptools in g:\pytho
n26\lib\site-packages (from zope.interface>=3.6.0->Twisted>=8.0->scrapy)
Installing collected packages: scrapy
  Running setup.py install for scrapy
    warning: no files found matching 'license.txt' under directory 'scrapy'
    no previously-included directories found matching 'docs\build'
Successfully installed scrapy
Cleaning up...


C:\Users\d>scrapy
Traceback (most recent call last):
  File "G:\Python26\lib\runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "G:\Python26\lib\runpy.py", line 34, in _run_code
    exec code in run_globals
  File "G:\Python26\lib\site-packages\scrapy\cmdline.py", line 7, in <module>
    from scrapy.crawler import CrawlerProcess
  File "G:\Python26\lib\site-packages\scrapy\crawler.py", line 5, in <module>
    from scrapy.core.engine import ExecutionEngine
  File "G:\Python26\lib\site-packages\scrapy\core\engine.py", line 14, in <modul
e>
    from scrapy.core.downloader import Downloader
  File "G:\Python26\lib\site-packages\scrapy\core\downloader\__init__.py", line
16, in <module>
    from .middleware import DownloaderMiddlewareManager
  File "G:\Python26\lib\site-packages\scrapy\core\downloader\middleware.py", lin
e 7, in <module>
    from scrapy.http import Request, Response
  File "G:\Python26\lib\site-packages\scrapy\http\__init__.py", line 11, in <mod
ule>
    from scrapy.http.request.form import FormRequest
  File "G:\Python26\lib\site-packages\scrapy\http\request\form.py", line 9, in <
module>
    import lxml.html
  File "G:\Python26\lib\site-packages\lxml-2.3.6-py2.6-win32.egg\lxml\html\__ini
t__.py", line 12, in <module>
    from lxml import etree
ImportError: DLL load failed: %1 is not a valid Win32 application

Answer 1

I just managed to solve this just now and for people who might stumble upon this, the problem is you need PyWin32. For windows users:

1. pip install PyWin32 here, choose the right python version and 32/64bit. Now in cmd:

   pip install c:\users\desktop\pywin32-220.1-cp27-cp27m-win_amd64.whl

2. Now, install pywin32 system files, run "python.exe Scripts\pywin32_postinstall.py -install" from an elevated command prompt. Run cmd as admin and in the command prompt

   python "C:\Python27\Scripts\pywin32_postinstall.py" -install

Answer 2

I just met with the same problem but I have already solved it.

Please kindly check your installed python package whether the version(32/64bit) is matched your Python installing version.

Since I am now working in a Scrapy project and I installed a 64bit lxml package to my computer, however, I just found out that the default installing version of Python in my computer is 32bit. Therefore, the version is not matched and the problem you posted would be occurred.