Getting error: DLL load failed: The operating syst

Question

I cannot successfully run the first command in the Scrapy tutorial (http://doc.scrapy.org/en/latest/intro/tutorial.html). The code copy pasted below is the result (with the error at the end).

Python 2.7 is installed, and I followed the installation instructions for scrapy (http://doc.scrapy.org/en/latest/intro/install.html). I am running Python 2.7.6 32 bit on Windows 7 (64 bit).

Other aspects of installation:

1. Twisted-13.2.0.dist-info
2. zope.interface-4.1.1-py2.7
3. Scrapy-0.22.2-py2.7
4. lxml-3.3.3-py2.7
5. cssselect-0.9.1-py2.7
6. cryptography-0.3.dist-info
7. pyOpenSSL-0.14-py2.7
8. pywin32_system32

And here's the error message:

C:\Python27\Scripts>scrapy startproject tutorial
    Traceback (most recent call last):
      File "C:\Python27\lib\runpy.py", line 162, in _run_module_as_main
        "__main__", fname, loader, pkg_name)
      File "C:\Python27\lib\runpy.py", line 72, in _run_code
        exec code in run_globals
      File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 9, in <module>
        from scrapy.crawler import CrawlerProcess
      File "C:\Python27\lib\site-packages\scrapy\crawler.py", line 3, in <module>
        from twisted.internet import reactor, defer
      File "twisted\internet\reactor.py", line 38, in <module>
        from twisted.internet import default
      File "twisted\internet\default.py", line 56, in <module>
        install = _getInstallFunction(platform)
      File "twisted\internet\default.py", line 50, in _getInstallFunction
        from twisted.internet.selectreactor import install
      File "twisted\internet\selectreactor.py", line 18, in <module>
        from twisted.internet import posixbase
      File "twisted\internet\posixbase.py", line 24, in <module>
        from twisted.internet import error, udp, tcp
      File "twisted\internet\tcp.py", line 29, in <module>
        from twisted.internet._newtls import (
      File "twisted\internet\_newtls.py", line 21, in <module>
        from twisted.protocols.tls import TLSMemoryBIOFactory, TLSMemoryBIOProtocol
      File "twisted\protocols\tls.py", line 40, in <module>
        from OpenSSL.SSL import Error, ZeroReturnError, WantReadError
      File "build\bdist.win32\egg\OpenSSL\__init__.py", line 8, in <module>
      File "build\bdist.win32\egg\OpenSSL\rand.py", line 11, in <module>
      File "build\bdist.win32\egg\OpenSSL\_util.py", line 4, in <module>
      File "C:\Python27\lib\site-packages\cryptography\hazmat\bindings\openssl\binding.py",      l3, in __init__
      self._ensure_ffi_initialized()
      File "C:\Python27\lib\site-packages\cryptography\hazmat\bindings\openssl\binding.py", l9, in _ensure_ffi_initialized libraries)
      File "C:\Python27\lib\site-packages\cryptography\hazmat\bindings\utils.py", line 77, ind_ffi
        ext_package="cryptography",
      File "C:\Python27\lib\site-packages\cffi\api.py", line 341, in verify
        lib = self.verifier.load_library()
      File "C:\Python27\lib\site-packages\cffi\verifier.py", line 75, in load_library
        return self._load_library()
      File "C:\Python27\lib\site-packages\cffi\verifier.py", line 151, in _load_library
        return self._vengine.load_library()
      File "C:\Python27\lib\site-packages\cffi\vengine_cpy.py", line 138, in load_library
        raise ffiplatform.VerificationError(error)

cffi.ffiplatform.VerificationError: importing 'C:\\Python27\\lib\\site-packages\\cryptogr
\_Cryptography_cffi_48bbf0ebx93c91939.pyd': DLL load failed: The operating system cannot
1.**

Answer 1

I just reinstall cryptography to make it work.

pip uninstall cryptography

pip install cryptography

Answer 2

@crazyzh1984's method is a little complex, but his supplement that posted at the bottom of his answer is very useful. I download the "Win32 OpenSSL v1.0.1g Light" at http://slproweb.com/products/Win32OpenSSL.html and then I could install pyOpenSSL successfully.

Answer 3

As others have said above, there are two .dll files that are loaded instead of the openssl that is downloaded with Scrapy. Those two .dll files are not compatible.

I resolved this by placing the downloaded .dll files (i used Anaconda to install scrapy) as a higher priority in the Environment Variables as shown in the attached image.

In Environment Variables, within System Variables and the "Path" i placed the Anaconda3 location as shown above at the top and therefore makes it the priority when python comes to load openssl

Answer 4

I'll write my own take on this due to my extreme frustration and hopes that this might help some other poor chap with his issues getting scrapy to work on windows..

Had a similar issues with faulty libeay32.dll and ssleay32.dll, or so it would seem, but confirmed it with Process Monitor.

It all went to hell after those dll's were found in System32:

So i wasnt suffering from wrong PATHs (but i did change their order so system32 comes before C:\Program Files (x86)\Intel\iCLS Client\.

Next i used this site the other guys here pointed to. Ive used Win32 OpenSSL v1.0.2j Light, since the newer one (1.1.0b) didnt generate these dlls for me for some reason.

Generate them in a different bin folder so you see they are new, and then copy them to system32. Voila, scrapy command line works.

Also, as of today, anaconda's scrapy 1.1.2 does not create a key file for running scrapy commands, ive opened and closed a question about this issue here.

Answer 5

this is due to _Cryptography_cffi_48bbf0ebx93c91939.pyd depends on openssl dll ssleay32.dll and libeay32.dll. after you compile openssl on windows, u can copy these 2 files to system32.

Answer 6

I had this problem due to another (older?) version of libeay32.dll and ssleay32.dll being on the PATH before those of my own OpenSSL 1.0.1g installation. I recommend you use Process Monitor from SysInternals to monitor python.exe to see where it is actually loading your OpenSSL DLLs from.

In my case, the offender was some Intel components at C:\Program Files (x86)\Intel\iCLS Client\ that came with my drivers. After moving my own OpenSSL bin directory earlier in the PATH environment variable, everything worked as expected.