VideoCapture() to read multiple videos and frame r

According to the answer from this article which refer to the way to combine single image into a 4 side. From there, I want to change from using only single video to use 4 videos as an input.

This is my code which used single video as an input

import cv2
import numpy as np


def make4side(image, scale=0.5):

   # image = cv2.imread(image)
   h = int((scale*image.shape[0])) #height
   w = int((scale*image.shape[1])) #width
   image = cv2.resize(image, (w,h ), interpolation = cv2.INTER_AREA) #shrink image to half

   output = np.zeros((w+h+h , w + h + h, 3), dtype="uint8")

   # top 
   output[0:h, h:h+w] = image 
   # left >> rotate 90
   output[h:h+w, 0:h] = np.rot90(image,1) 
   # right >> rotate 270
   output[h:h + w, h + w:h +w +h] = np.rot90(image,3)  
   # bottom >> rotate 180
   output[h+w:h+w+h, h:h+w] = np.rot90(image,2) 

   return output
   #cv2.imwrite('test.jpg', output)

def process(video):
   cap = cv2.VideoCapture(video)
   fourcc = cv2.VideoWriter_fourcc(*'XVID')
   holo = None
   ret = False
   while(not ret):
    ret, frame = cap.read()
    if ret:
        frame = cv2.resize(frame, (640, 480), interpolation = cv2.INTER_AREA)
        holo = make4side(frame)
   out = cv2.VideoWriter('hologram640x480.avi',fourcc, 23.98, (holo.shape[0],holo.shape[1]))
   total_frames = cap.get(cv2.CAP_PROP_FRAME_COUNT)
   count = 0
   print("Processing %d frames"%(total_frames))
   while(True):
       # Capture frame-by-frame
       ret, frame = cap.read()
       if ret:
           frame = cv2.resize(frame, (640, 480), interpolation = cv2.INTER_AREA)
           holo = make4side(frame)
           out.write(holo)
           count += 1
           print("Total:%d of %d"%(count,total_frames))
       if(count>=total_frames-1):
           break

   cap.release()
   out.release()
   return

process('g.mov')

the result is like this.

In this code, the overall frame's height and width are based only on the input which is the single video which I concern on this too since I use 4 videos and of course the frame resolution are not the same (but all are landscape). Variables h and w in the function make4side() are the main part that help positioning each of the small frame. So for this case, what should the big frame (variable output) resolution be?

I have to read 4 videos and write it into one so, how can I use VideoCapture object to do it for this purpose

To make my question clear, I want to have a single video that consist of 4 input videos, each of them are going to be placed at each of the position (top, bottom, left and right). I have a problem with the big frame resolution which I don't know what to use if I have 4 videos instead of one. Another problem is about the VideoCapture object. How can I read frames of all videos at the same time or any other way to do this?

Thank you

Edit: top side

left side

back side

right side

These are not the real frames I will be used but just a simple idea what I am going to use for my video. Another thing, input files may not have the same resolution. How can I use many videocapture objects to read each of them and place it on each side of the big frame to write a single video

So everything depends on what you want to do, so it will come on what type of images you will be processing. First of all, you can always have 4 instances of the VideoCapture class, and each of them loads a new video, something like:

videoTop = cv2.VideoCapture(videoTopFileName)
videoLeft = cv2.VideoCapture(videoLeftFileName)
videoRight = cv2.VideoCapture(videoRightFileName)
videoBottom = cv2.VideoCapture(videoBottomFileName)

readCorrect = True
while( readCorrect ):
  readCorrect , topFrame = videoTop.read()
  ret, leftFrame = videoLeft.read()
  readCorrect = readCorrect and ret
  ret, rightFrame = videoRight.read()
  readCorrect  = readCorrect and ret
  ret, bottomFrame = videoBottom.read()
  readCorrect = readCorrect and ret
  if readCorrect :
     holo = make4side(topFrame, leftFrame, rightFrame, bottomFrame )

You can in this loop already save your image in the VideoWriter.

Now it comes the tricky part, your images are not of equal size... you can do something like:

import cv2
import numpy as np

# load images, in your case frames from videos
top = cv2.imread("D:\\testing\\1.jpg")
left = cv2.imread("D:\\testing\\2.jpg")
bottom = cv2.imread("D:\\testing\\3.jpg")
right = cv2.imread("D:\\testing\\4.jpg")

targetSize = (200,200)

h = targetSize[1] #height
w = targetSize[0] #width

top = cv2.resize(top,  targetSize )
left = cv2.resize(left,  targetSize )
bottom = cv2.resize(bottom,  targetSize )
right = cv2.resize(right,  targetSize )

output = np.zeros((w+h+h , w + h + h, 3), dtype="uint8")

# top
output[0:h, h:h+w] = top
# left >> rotate 90
output[h:h+w, 0:h] = np.rot90(left,1)
# right >> rotate 270
output[h:h + w, h + w:h +w +h] = np.rot90(bottom,3)
# bottom >> rotate 180
output[h+w:h+w+h, h:h+w] = np.rot90(right,2)

cv2.imshow("frame", output )
cv2.waitKey(0)import cv2
import numpy as np

# load images, in your case frames from videos
top = cv2.imread("D:\\testing\\1.jpg")
left = cv2.imread("D:\\testing\\2.jpg")
bottom = cv2.imread("D:\\testing\\3.jpg")
right = cv2.imread("D:\\testing\\4.jpg")

targetSize = (200,200)

h = targetSize[1] #height
w = targetSize[0] #width

top = cv2.resize(top,  targetSize )
left = cv2.resize(left,  targetSize )
bottom = cv2.resize(bottom,  targetSize )
right = cv2.resize(right,  targetSize )

output = np.zeros((w+h+h , w + h + h, 3), dtype="uint8")

# top
output[0:h, h:h+w] = top
# left >> rotate 90
output[h:h+w, 0:h] = np.rot90(left,1)
# right >> rotate 270
output[h:h + w, h + w:h +w +h] = np.rot90(bottom,3)
# bottom >> rotate 180
output[h+w:h+w+h, h:h+w] = np.rot90(right,2)

cv2.imshow("frame", output )
cv2.waitKey(0)

But this generates a "bad" image like this one:

To make it not distorted, you should find the aspect ratio and try to resize them something similar. In case that the aspect ratio is different, then you will have to pad the images. This is the part that depends on your task, you can crop the image or pad it.

But basically that is what should be done. I hope this helps you.

Update:

Just to clarify the loop part:

  readCorrect , topFrame = videoTop.read()
  ret, leftFrame = videoLeft.read()
  readCorrect = readCorrect and ret

In the first line I assigned to the readCorrect variable the boolean return from read. Then in the next image I assigned to ret and do a logic and with the previous result. This way you know if all of them are true, or if ANY is false.

I also corrected something in the loop that had a mistake (I put while not readCorrect and it should be without the not).

One more thing before the loop you should create the VideoWriter object, you can always get the sizes of each video before reading with the get with the argument CV_CAP_PROP_FRAME_WIDTH and CV_CAP_PROP_FRAME_HEIGHT, like videoTop.get(CV_CAP_PROP_FRAME_WIDTH).

And then inside the loop, specifically inside the if after getting the image you can write it to.