Python, Java, C# and parallel algorithms

Question

I was wondering: would there be any merit in attempting to create parallel algorithms in Python? Say I want to research a new parallel algorithm, and I have the choice of C, C# and Python, would one or the other be "better" to test and benchmark these algorithms, or they are just "functionally equivalent" and, besides the constants associated to interpreted/compiled/vm languages, it would all be the same? Thanks

Answer 1

C has a lot of alternatives, but it's a low level language. Intel's Thread Building Blocks is a commercial parallel library based on Cilk that I hear is good.

It depends what you want to do for Python. While you can sidestep the GIL by using Cython, a pre-compiled Python variant, Cython is slow to compile and loses platform independence. Also, Python is a scripting language, so you usually want to switch to a faster language before you parallelize. That being said, using the multiprocessing module in Python (which forks to sidestep the GIL) you can write some simple parallel codes.

Java is probably your best bet. The synchronized keyword, as well as decent primitives (e.g. a threadsafe nonblocking queue) makes it relatively pleasant. Java's old though, so you waste time writing more verbose code than Python.

I'm slowly (incrementally) learning parallelism in Haskell. It doesn't seem to be the easiest thing to get something efficient running, but if your task is pretty parallel, then it seems to work well. And, of course, Haskell is incredibly expressive (after you get over it being pretty different syntactically -- different for good reason imho), and since it's functional you don't get weird state corruption issues; the worst case is that your code doesn't scale. There's a variety of libraries, like STM (software transactional memory), which seem to solve certain problems much much better (in theory) than primitives that imperative programmers use, like locks.

Answer 2

Python is not suitable for this because of global interpreter lock (GIL), it doesn't work this way. C is hard to use in multithreaded environment, but there is an alternative - Cilk language. C# is a pretty nice choice for parallel programming. You can use Task Parallel Library, concurrent data structures and PLINQ from .NET Framework 4.

Answer 3

I would definitively use Python, because writing Python programs is fast and easy, and because it probably already includes all the tools you would need. It will allow you to focus on algorithm, instead of working on syntax and declarations.

For benchmarking, you can use cProfile or timeit module.

What kind of parallel algorithms do you want to test? There are many ways to do parallel programming. For algorithms that need to scale to many processes, the easiest way is probably using MPI. mpi4py is a very good implementation for Python, and is tightly integrated with numpy, if you need to work on arrays of numbers.

As it was already mentioned, multithreading can only use 1 processor, because of the GIL. But this limitation only apply to this case. You can use multiprocessing on as many processors you want.

Finally, there are tools in Python for functional programming. And coroutines can be great for concurrent programming.