Convection of an image using optical flow

Question

I have two images (frame1 and frame2) and I am able to calculate u,v using opencv:

flow = cv2.calcOpticalFlowFarneback(prvs,next, 0.5, 1, 3, 15, 3, 5, 1, 0)

I want to translate frame1 using this u,v to quantify the quality of the difference using various optical flow methods. I intend to extrapolate using these u,v.

Is there a simple way to achieve this?

Answer 1

One way to compute a simple translation is to average the flow:

avg_u = np.mean(flow[:, :, 0])
avg_v = np.mean(flow[:, :, 1])

This gives the vector (avg_u, avg_v) needed to translate.

---

Regarding your comments, it seems you want to move every pixel (x,y) to the location specified by its flow vector (u(x,y), v(x,y)).

First, generate a Cartesian grid:

height, width = flow.shape[0, 1]
R2 = np.dstack(np.meshgrid(np.arange(width), np.arange(height)))

Then, the desired mapping is simply the addition of this grid with the flow:

pixel_map = R2 + flow

Finally, perform a cv2.remap:

new_frame = cv2.remap(prev_frame, pixel_map)