I am trying to convert my code base from numpy array to dask because my numpy arrays are exceeding the Memory Error
limit. But, I come to know that the feature of mutable arrays are not yet implemented in dask arrays
so I am getting
NotImplementedError: Item assignment with <class 'tuple'> not supported
Is there any workaround for my code below-
for i, mask in enumerate(masks):
bounds = find_boundaries(mask, mode='inner')
X2, Y2 = np.nonzero(bounds)
X2 = da.from_array(X2, 'auto')
Y2 = da.from_array(Y2, 'auto')
xSum = (X2.reshape(-1, 1) - X1.reshape(1, -1)) ** 2
ySum = (Y2.reshape(-1, 1) - Y1.reshape(1, -1)) ** 2
#Failing on below line
distMap[:,i] = da.sqrt(xSum + ySum).min(axis=0)
break
I also tried computing all other computations in dask and using distMap
as a numpy array but still got the Memory Error
.
Any workarounds are welcome.
Maybe you can construct many dask arrays and then use
da.concatenate
orda.stack
to merge them into a single dask array?