Swap Array Data in NumPy

Question

I have many large multidimensional NP arrays (2D and 3D) used in an algorithm. There are numerous iterations in this, and during each iteration the arrays are recalculated by performing calculations and saving into temporary arrays of the same size. At the end of a single iteration the contents of the temporary arrays are copied into the actual data arrays.

Example:

global A, B # ndarrays
A_temp = numpy.zeros(A.shape)
B_temp = numpy.zeros(B.shape)
for i in xrange(num_iters):
    # Calculate new values from A and B storing in A_temp and B_temp...
    # Then copy values from temps to A and B
    A[:] = A_temp
    B[:] = B_temp

This works fine, however it seems a bit wasteful to copy all those values when A and B could just swap. The following would swap the arrays:

A, A_temp = A_temp, A
B, B_temp = B_temp, B

However there can be other references to the arrays in other scopes which this won't change.

It seems like NumPy could have an internal method for swapping the internal data pointer of two arrays, such as numpy.swap(A, A_temp). Then all variables pointing to A would be pointing to the changed data.

Answer 1

I realize this is an old question, but for what it's worth you could also swap data between two ndarray buffers (without a temp copy) by performing an xor swap:

A_bytes = A.view('ubyte')
A_temp_bytes = A.view('ubyte')
A_bytes ^= A_temp_bytes
A_temp_bytes ^= A_bytes
A_bytes ^= A_temp_bytes

Since this was done on views, if you look at the original A and A_temp arrays (in whatever their original dtype was) their values should be correctly swapped. This is basically equivalent to the numpy.swap(A, A_temp) you were looking for. It's unfortunate that it requires 3 loops--if this were implemented as a ufunc (maybe it should be) it would be a lot faster.

Answer 2

Even though you way should work as good (I suspect the problem is somewhere else), you can try doing it explicitly:

import numpy as np
A, A_temp = np.frombuffer(A_temp), np.frombuffer(A)

It's not hard to verify that your method works as well:

>>> import numpy as np
>>> arr = np.zeros(100)
>>> arr2 = np.ones(100)
>>> print arr.__array_interface__['data'][0], arr2.__array_interface__['data'][0]
152523144 152228040

>>> arr, arr2 = arr2, arr
>>> print arr.__array_interface__['data'][0], arr2.__array_interface__['data'][0]
152228040 152523144

... pointers succsessfully switched

Answer 3

Perhaps you could solve this by adding a level of indirection.

You could have an "array holder" class. All that would do is keep a reference to the underlying NumPy array. Implementing a cheap swap operation for a pair of these would be trivial.

If all external references are to these holder objects and not directly to the arrays, none of those references would get invalidated by a swap.