fast way to display video from arrays in jupyter-l

Question

I'm trying to display a video from some arrays in an notebook in jupyter-lab. The arrays are produced at runtime. What method to display the images can deliver a (relatively) high framerate? Using matplotlib and imshow is a bit slow. The pictures are around 1.8 megapixel large. Above some very small example to visualize what I want to achieve.

while(True): #should run at least 30 times per second 
    array=get_image() #returns RGBA numpy array  
    show_frame(array) #function I search for

Answer 1

The fastest way (to be used for debug purpose for instance) is to use matplotlib inline and the matplotlib animation package. Something like this worked for me

%matplotlib inline
from matplotlib import pyplot as plt
from matplotlib import animation
from IPython.display import HTML

# np array with shape (frames, height, width, channels)
video = np.array([...]) 

fig = plt.figure()
im = plt.imshow(video[0,:,:,:])

plt.close() # this is required to not display the generated image

def init():
    im.set_data(video[0,:,:,:])

def animate(i):
    im.set_data(video[i,:,:,:])
    return im

anim = animation.FuncAnimation(fig, animate, init_func=init, frames=video.shape[0],
                               interval=50)
HTML(anim.to_html5_video())

The video will be reproduced in a loop with a specified framerate (in the code above I set the interval to 50 ms, i.e., 20 fps).

Please note that this is a quick workaround and that IPython.display has a Video package (you can find the documentation here) that allows you to reproduce a video from file or from an URL (e.g., from YouTube). So you might also consider storing your data locally and leverage the built-in Jupyter video player.