Why is python so much slower on windows?

Question

I learned about pystones today and so I decided to see what my various environments were like. I ran pystones on my laptop that is running windows on the bare metal and got these results

Python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from test import pystone
>>> for i in range(0,10):
...   pystone.pystones()
...
(1.636334799754252, 30556.094026423627)
(2.1157907919853756, 23631.82607155689)
(2.5324817108003685, 19743.479207278437)
(2.541626695533182, 19672.4405231788)
(2.536022267835051, 19715.915208695682)
(2.540327088340973, 19682.50475676099)
(2.544761766911506, 19648.20465716261)
(2.540296805235016, 19682.739393664764)
(2.533851636391205, 19732.804905346253)
(2.536483186973612, 19712.3325148696)

Then I ran it on some of our linux VMs and got 2.7-3.4 times better performance. So I fired up my vmware Linux VM on my laptop and reran the same test and got these results:

Python 2.7.2+ (default, Oct  4 2011, 20:03:08) 
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> for i in range(0,10):
...   pystone.pystones()
... 
(1.75, 28571.428571428572)
(1.17, 42735.042735042734)
(1.6600000000000001, 30120.48192771084)
(1.8399999999999999, 27173.913043478264)
(1.8200000000000003, 27472.52747252747)
(1.8099999999999987, 27624.30939226521)
(1.3099999999999987, 38167.938931297744)
(1.7800000000000011, 28089.88764044942)
(1.8200000000000038, 27472.527472527414)
(1.490000000000002, 33557.04697986573)

I can't quite understand how the linux VM running inside the same windows is actually FASTER than python running on the same bare metal under windows.

What is so different about python on windows that it performs slower on the bare OS than it does inside a VM running Linux on the same box?

• More details Windows platform Win7x64 32 bit python running on both platforms 32 bit linux VM running the windows platform in VMWare

Answer 1

Had similar problem on windows 10 - it was because of windows defender.

I had to exclude python directories and process in windows defender settings and restart computer.

Before: I had to wait like ~20 seconds to run any python code - now it's milliseconds.

Answer 2

Do you run antivirus software on that Windows box? This perhaps could explain it. I personally like to add Python, Cygwin and my sources directory to antivirus exclusion list - I think I get a small, but noticeable speedup. Maybe that explains your results.

Answer 3

Benchmark your startup, but there are just simply some slow modules to initialize on windows. A tiny hack that saves me a second on startup every time:

import os
import mimetypes #mimetypes gets imported later in dep chain

if __name__ == "__main__":
   # stub this out, so registry db wont ever be read, not needed
   mimetypes._winreg = None

Another source of slowness is, multiple standard library modules compile and cache their regexes at import time. re.compile just looks like its slow on windows

Answer 4

I can't answer your question, however consider this list of things that could be making a difference:

• You're using different versions of Python. "2.7.2+" indicates that your linux Python was built from a version control checkout rather than a release.

• They were compiled with different compilers (and conceivably meaningfully different optimization levels).

• You haven't mentioned reproducing this much. It's conceivable it was a fluke if you haven't.

• Your VM might be timing inaccurately.

• You're linking different implementations of Python's dependencies, notably libc as Ignacio Vazquez-Abrams points out.

• I don't know what pystone's actual benchmarks are like, but many things work differently--things like unicode handling or disk IO could be system-dependent factors.