I am trying to send my macbook air webcam video in realtime and receive it using python on another computer. The motivation for this is to be able to offload realtime image recognition tasks to a server. The server requires having access to the realtime video frames in python so that I can then pass the frames to my image recognition algorithm (a deep neural net).
I was able to do this successfully using https://github.com/atuldo/videoStream which uses the socket library to send the video frames in string format. However, this method results is a very low frame rate (see below for code snippet).
Attempt 1:
from flask import Flask, render_template, Response
from camera import VideoCamera
app = Flask(__name__)
@app.route('/')
def index():
return render_template('index.html')
def gen(camera):
while True:
frame = camera.get_frame()
yield (b'--frame\r\n'
b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')
@app.route('/video_feed')
def video_feed():
return Response(gen(VideoCamera()),
mimetype='multipart/x-mixed-replace; boundary=frame')
if __name__ == '__main__':
app.run(host='0.0.0.0', debug=True)
I also used this https://github.com/log0/video_streaming_with_flask_example which uses flask to stream the video to a web page. Looking at the webpage in chrome it has a much better frame rate than my first solution. However, this solution does not receive the video in python and would require me to read the video from the webpage, which I am not exactly sure how to go about.
Attempt 2 (I replaced pygame from original github code with cv2 to be able to access my mac webcam) :
### Server
import socket
from threading import *
import socket
import pygame
import pygame.camera
from pygame.locals import *
import cv2
def server():
#host_name = socket.gethostname()
host_name = 'localhost'
print "\nServer started at " + str(socket.gethostbyname(host_name)) + " at port " + str(90)
#port = 90
port = 1024
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.bind(("",port))
serversocket.listen(10)
#pygame.camera.init()
#cam = pygame.camera.Camera(0,(640,480),"RGB")
#cam.start()
#img = pygame.Surface((640,480))
video = cv2.VideoCapture(0)
while True:
connection, address = serversocket.accept()
print "GOT_CONNECTION"
#cam.get_image(img)
#data = pygame.image.tostring(img,"RGB")
success, image = video.read()
data = cv2.imencode('.jpg', image)[1].tostring()
connection.sendall(data)
connection.close()
server()
###Reciever (snippet, for full code please refer to the github link above)
clientsocket=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
clientsocket.connect((self.ipAddress, self.port))
received = []
while True:
recvd_data = clientsocket.recv(230400)
if not recvd_data:
break
else:
received.append(recvd_data)
dataset = ''.join(received)
#image = pygame.image.fromstring(dataset,(640, 480),"RGB") # convert received image from string
nparr = np.fromstring(dataset, np.uint8)
image = cv2.imdecode(nparr, cv2.CV_LOAD_IMAGE_COLOR)
cv2.imwrite('foo.jpg', image)
#pygame.image.save(image, "foo.jpg")
self.ids.image_source.reload()
The video stream is displayed on the webpage like so:
<html>
<head>
<title>Video Streaming Demonstration</title>
</head>
<body>
<h1>Video Streaming Demonstration</h1>
<img id="bg" src="{{ url_for('video_feed') }}">
</body>
</html>
How can I get a high frame rate with either one of the two approaches or is there a better way to go about this?
