I am working on a project for estimating a UAV location using optical-flow algorithm. I am currently using cv::calcOpticalFlowFarneback
for this purpose.
My hardware is an Odroid U3 that will finally be connected to the UAV flight controller.
The problem is that this method is really heavy for this hardware and I am looking for some other ways to optimize / accelerate it.
Things that I've already tried:
- Reducing resolution to 320x240 or even 160x120.
- Using OpenCV TBB (compiled using
WITH_TBB=ON BUILD_TBB=ON
and adding-ltbb
). - Changing optical-flow parameters as suggested here
Adding the relevant part of my code:
int opticalFlow(){
// capture from camera
VideoCapture cap(0);
if( !cap.isOpened() )
return -1;
// Set Resolution - The Default Resolution Is 640 x 480
cap.set(CV_CAP_PROP_FRAME_WIDTH,WIDTH_RES);
cap.set(CV_CAP_PROP_FRAME_HEIGHT,HEIGHT_RES);
Mat flow, cflow, undistortFrame, processedFrame, origFrame, croppedFrame;
UMat gray, prevgray, uflow;
currLocation.x = 0;
currLocation.y = 0;
// for each frame calculate optical flow
for(;;)
{
// take out frame- still distorted
cap >> origFrame;
// Convert to gray
cvtColor(origFrame, processedFrame, COLOR_BGR2GRAY);
// rotate image - perspective transformation
rotateImage(processedFrame, gray, eulerFromSensors.roll, eulerFromSensors.pitch, 0, 0, 0, 1, cameraMatrix.at<double>(0,0),
cameraMatrix.at<double>(0,2),cameraMatrix.at<double>(1,2));
if( !prevgray.empty() )
{
// calculate flow
calcOpticalFlowFarneback(prevgray, gray, uflow, 0.5, 3, 10, 3, 3, 1.2, 0);
uflow.copyTo(flow);
// get average
calcAvgOpticalFlow(flow, 16, corners);
/*
Some other calculations
.
.
.
Updating currLocation struct
*/
}
//break conditions
if(waitKey(1)>=0)
break;
if(end_run)
break;
std::swap(prevgray, gray);
}
return 0;
}
Notes:
- I've ran
callgrind
and the bottleneck is as expected thecalcOpticalFlowFarneback
function. - I checked the CPU cores load while running the program, and it is not using all 4 cores heavily, only one core is on 100% at a given time (even with TBB):