In OpenCV, what data types does cv2.filter2D() exp

Question

I'm teaching myself about edge detectors, and I'm trying to use OpenCV's filter2D to implement my own gradient calculator, similar to cv2.Sobel(). In the Python interface to OpenCV, cv2.filter2D() allows users to convolve an image with a custom filter. In OpenCV nomenclature, this filter is called a "kernel."

Using an image (per00001.png) from the MIT pedestrian dataset, I find that cv2.Sobel() produces a reasonable looking output. (Code is below, output image is here.)

#OpenCV's Sobel code (outputs nice-looking gradient)
import cv2, numpy
img = cv2.imread("./per00001.png")

gradientX = cv2.Sobel(img, -1, 1, 0)

compression_params = [cv2.cv.CV_IMWRITE_PNG_COMPRESSION, 9]
cv2.imwrite("gradientX.png", gradientX, compression_params)

↑ GOOD

---

↓ BROKEN

When I try to implement my own Sobel()-like code (below), I get an all-black image. I'm speculating that the problem is with the data type of the kernel parameter (horizontalSobelMtx) that I'm passing into cv2.filter2D(). However, I haven't been able to find any documentation about the kernel data type for cv2.filter2D().

#Custom Sobel code (outputs all-black image)
import cv2, numpy
img = cv2.imread("./per00001.png")

horizontalSobelMtx = [[-1,0,1],[-2,0,2],[-1,0,1]]
horizontalSobelMtx = numpy.asanyarray(horizontalSobelMtx) #guessing about appropriate datatype.
gradientX_customSobel = cv2.filter2D(img, -1, horizontalSobelMtx)

compression_params = [cv2.cv.CV_IMWRITE_PNG_COMPRESSION, 9]
cv2.imwrite("gradientX_customSobel.png", gradientX_customSobel, compression_params)

So, here are my questions:

1) What data type does cv2.filter2D(..., kernel, ...) expect for the kernel parameter?

2) If the data type of kernel isn't the problem here, then what's causing my custom Sobel code to output a blank image?

Answer 1

The coefficients of the convolution kernel should always be floating- point numbers. This means that you should use CV_32FC1 when allocating that matrix. In this particular example, try:

horizontalSobelMtx = [[-1,0,1],[-2,0,2],[-1,0,1]]
horizontalSobelMtx = numpy.asanyarray(horizontalSobelMtx, np.float32)

Answer 2

I had the same problem but I believe I have a partial answer. Basically, you need to weight your kernel. The way the filter algorithm works is it takes your filter, multiplies it, then adds all the values together and uses them as the new value. The key is the ADDING part. Adding 8 different values (some negative, some positive) will usually result in a large number, and displayed, appears black (or in my case, all white). So you have to compensate for the addition. Divide all the values in your kernel by the size/area of the kernel. See the example here and note the line

  kernel = Mat::ones( kernel_size, kernel_size, CV_32F )/ (float)(kernel_size*kernel_size);

They divide everything by kernel_size squared.

Myself, I did this, and it helped (something showed) but it was washed out, so I found my perfect weight by adding this to my code:

 int d2weight = 1;
 //... start loop, make your Mat kernel
 kernel = kernel / d2weight;  //put right before your filter2d() call
 createTrackbar("kernel", "kernel", &d2weight, 255, NULL);

and fiddled with the bar and right around d2weight = 140 my picture 'popped' into view.