move values of 2D array knowing new coordinates wi

Question

I would like to "move" elements of a 2D array to new coordinates which are stored in 2 other arrays. I'm looking to automate this, because in reality my arrays are large (400x200x100). Some values wont find his coordinates and wont be used, Some of these coordinates are masked, which I have indicated in the example below by using the value 0. If the coordinate is masked, the elements in the array I want to reshuffle won't be used.

import numpy as np

#My new coordinates in X and Y directions   
mx = np.array([[ 1.,  2.,  3.,  4.,  0.],
       [ 1.,  2.,  3.,  4.,  0.],
       [ 1.,  2.,  3.,  4.,  0.],
       [ 1.,  2.,  3.,  4.,  0.],
       [ 1.,  2.,  3.,  4.,  0.]])

my = np.array([[ 0.,  2.,  2.,  2.,  2.],
       [ 0.,  3.,  3.,  3.,  3.],
       [ 0.,  4.,  4.,  4.,  4.],
       [ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.]])

#The array with values to move
IRtest = np.array([[-0.07383495, -0.08606554, -0.08480594, -0.08099556, -0.08218414],
       [-0.07866761, -0.08373   , -0.08253587, -0.08106102, -0.08220205],
       [-0.07727436, -0.08271511, -0.0807254 , -0.07832416, -0.08021686],
       [-0.07612349, -0.08190446, -0.07996929, -0.07842754, -0.08024891],
       [-0.07488144, -0.08150557, -0.08038229, -0.07895656, -0.07997815]])

#Creation of zeros array to get new array
b = np.zeros((5,5))    

# I tried this but it doesn't work...
for i in range(IRtest.shape[0]):
    for j in range(IRtest.shape[1]):
        b[my[i,j], mx[i,j]] = IRtest[i,j]

plt.imshow(b)
plt.colorbar()
plt.show()

So the array expected looks like :

array_expected = np.array([[-0.08271511, -0.0807254 , -0.07832416, -0.08021686, 0],
       [-0.08190446, -0.07996929, -0.07842754, -0.08024891, 0],
       [-0.08150557, -0.08038229, -0.07895656, -0.07997815, 0],
       [0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0]])

-----EDIT LATER -----------------

It s better in this "orientation" :

for i in range(IRtest.shape[0]):
    for j in range(IRtest.shape[1]):
        b[j, i] = IRtest[my[j,i],mx[j,i]]

And i get that :

array([[-0.08606554, -0.0807254 , -0.07832416, -0.08021686, -0.07727436],
       [-0.08606554, -0.07996929, -0.07842754, -0.08024891, -0.07612349],
       [-0.08606554, -0.08038229, -0.07895656, -0.07997815, -0.07488144],
       [-0.08606554, -0.08480594, -0.08099556, -0.08218414, -0.07383495],
       [-0.08606554, -0.08480594, -0.08099556, -0.08218414, -0.07383495]])

So the last problem it s to deal with the masked values...

So i try :

mask_mx = np.array([[False, False, False, False, True],
        [False, False, False, False, True],
        [False, False, False, False, True],
        [False, False, False, False, True],
        [False, False, False, False, True]], dtype=int)

mask_my = np.array([[True, False, False, False, False],
        [True, False, False, False, False],
        [True, False, False, False, False],
        [True, True, True, True, True],
        [True, True, True, True, True]], dtype=int)


mx3 = np.where(mask_mx, 'nan', mx)

my3 = np.where(mask_my, 'nan', my)

for i in range(IRtest.shape[0]):
    for j in range(IRtest.shape[1]):
        b[j, i] = IRtest[my3[j,i],mx3[j,i]]

But i get the error below, it doesn t like 'nan' as coordinates : invalid literal for int() with base 10: 'nan'

Answer 1

Thanks to numpy's fancy-indexing, the loop isn't necessary, and in fact, your code will run much faster without it.

Also, note that:

1. your index-arrays are best represented as int-arrays (fancy-indexing wouldn't work with floats)
2. masks are best represented as bool-arrays (which allow logical operations as bitwise operations, are more memory efficient, and much faster).

So some preprocessing is required:

mx = np.array(mx, dtype=int)
my = np.array(my, dtype=int)
mask_mx = np.array(mask_mx, dtype=bool)
mask_my = np.array(mask_my, dtype=bool)
unified_mask = mask_my | mask_mx  # logical-or as bitwise-or

---

So, first without the mask, you can simply fancy-index your array:

b = IRtest[my,mx]

Then to apply the mask, you can either do:

b *= ~unified_mask

or explicitly assign zeros (again, fancy-indexing, this time using a mask):

b[unified_mask] = 0

As a one-liner:

b = IRtest[my,mx] * ~unified_mask

This is shorter, and way faster, than looping. Neat, eah?

With numpy, more often than not, you can and should avoid the loops.

Answer 2

You probably want to use -1 as mask and not 0, so you can access the 0 index in IRtest.

mx = np.array([[ 1.,  2.,  3.,  4.,  -1.],
               [ 1.,  2.,  3.,  4.,  -1.],
               [ 1.,  2.,  3.,  4.,  -1.],
               [ 1.,  2.,  3.,  4.,  -1.],
               [ 1.,  2.,  3.,  4.,  -1.]])

my = np.array([[ 2.,  2.,  2.,  2.,  2.],
               [ 3.,  3.,  3.,  3.,  3.],
               [ 4.,  4.,  4.,  4.,  4.],
               [ -1.,  -1.,  -1.,  -1.,  -1.],
               [ -1.,  -1.,  -1.,  -1.,  -1.]])

b = np.zeros_like(IRtest)

for i in range(IRtest.shape[0]):
    for j in range(IRtest.shape[1]):
        b[j, i] = IRtest[my[j,i],mx[j,i]]*(mx[j,i]!=-1)*(my[j,i]!=-1)

Answer 3

Your code does work, but your arrays of coordinates are not what you have in mind. Let's analyze your double for-loop:

During the iteration where i is 2 and j is 1, you're effectively handling the case:

b[my_test[2,1], mx_test[2,1]] = IRtest[2,1]

IRtest[2,1] is -0.08271511. my[2,1] is 4 and mx[2,1] is 2, so the line above boils down to:

b[4, 2] = -0.08271511

which is exactly what you see in the output.

Basically the problem is that you should change your indices arrays my and mx to get the output you desire.