Convert CMSampleBuffer to UIImage

Question

I am trying to convert sampleBuffer to a UIImage and display it in an image view with colorspaceGray. But it displays as the following image. I think there is a problem regarding the conversion. How can I convert the CMSampleBuffer?

func captureOutput(_ captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, from connection: AVCaptureConnection!) {
    print("buffered")
    let imageBuffer: CVImageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!
    CVPixelBufferLockBaseAddress(imageBuffer, CVPixelBufferLockFlags(rawValue: 0))
    let width: Int = CVPixelBufferGetWidth(imageBuffer)
    let height: Int = CVPixelBufferGetHeight(imageBuffer)
    let bytesPerRow: Int = CVPixelBufferGetBytesPerRow(imageBuffer)
    let lumaBuffer = CVPixelBufferGetBaseAddress(imageBuffer)

    //let planeCount : Int = CVPixelBufferGetPlaneCount(imageBuffer)
    let grayColorSpace: CGColorSpace = CGColorSpaceCreateDeviceGray()
    let context: CGContext = CGContext(data: lumaBuffer, width: width, height: height, bitsPerComponent: 8, bytesPerRow: bytesPerRow , space: grayColorSpace, bitmapInfo: CGImageAlphaInfo.none.rawValue)!
    let dstImageFilter: CGImage = context.makeImage()!
    let imageRect : CGRect = CGRect(x: 0, y: 0, width: width, height: height)
    context.draw(dstImageFilter, in: imageRect)
    let image = UIImage(cgImage: dstImageFilter)
    DispatchQueue.main.sync(execute: {() -> Void in
        self.imageTest.image = image
    })
}

Answer 1

Looks like the CMSampleBuffer is giving you RGBA data from which you then directly construct the grayscale image. You will either need to construct a new buffer where for each pixel you do something like gray = (pixel.red+pixel.green+pixel.blue)/3. Or you need to create a normal RGBA image from the data you received and then convert it to grayscale.

But in your code you do no transition at all. You took the raw pointer to buffer using CVPixelBufferGetBaseAddress regardless of what sort of data is in there. And then you just pass the same pointer in creating the image which assumes the data received are grayscale.

Answer 2

The conversion is simple :

func captureOutput(_ captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, from connection: AVCaptureConnection!) {

     let imageBuffer: CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!
     let ciimage : CIImage = CIImage(cvPixelBuffer: imageBuffer)   
     let image : UIImage = self.convert(cmage: ciimage) 
}

// Convert CIImage to CGImage
func convert(cmage:CIImage) -> UIImage
{
     let context:CIContext = CIContext.init(options: nil)
     let cgImage:CGImage = context.createCGImage(cmage, from: cmage.extent)!
     let image:UIImage = UIImage.init(cgImage: cgImage)
     return image
}