how to convert AVFrame to texture used by glTexIma

Question

as you know that AVFrame has 2 property:pFrame->data, pFrame->linesize. After i read frame from video /sdcard/test.mp4 (android platform), and convert to RGB AVFrame vice:

img_convert_ctx = sws_getContext(pCodecCtx->width, pCodecCtx->height, 
                   pCodecCtx->pix_fmt, 
                   target_width, target_height, PIX_FMT_RGB24, SWS_BICUBIC, 
                   NULL, NULL, NULL);
            if(img_convert_ctx == NULL) {
                LOGE("could not initialize conversion context\n");
                return;
            }
            sws_scale(img_convert_ctx, (const uint8_t* const*)pFrame->data, pFrame->linesize, 0, pCodecCtx->height, pFrameRGB->data, pFrameRGB->linesize);

I get pFrameRGB after converted. I need to texture it in opengl by using glTextImage2D:

// Create The Texture
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, 256, 256, 0, GL_RGB, GL_UNSIGNED_BYTE, data); 

// i don't know data in here is pFrameRGB->data or not,if not, how to convert it to approriate format for glTextImage so i can display video in opengl, using AVFrame and gltextImage2D. All helps from u are very appreciated.

Answer 1

Two things: (1) is target_width and target_height powers of 2? If not, you'll get a blank screen. You need to scale the picture to a power of 2. (2) where it says "data" in glTexImage, you need to use pFrameRGB->data[0] as Kieran pointed out.

One more thing, you don't need to use RGBA (unless it's faster. I haven't tested it out). If you do, you change the ffmpeg target format to PIX_FMT_RGBA and then change both instances of GL_RGB to GL_RGBA.

Other than that, I don't see anything wrong with your code. I have almost the same exact code and it runs just fine.

Answer 2

The actual pixel data for the pFrameRGB is in pFrameRGB->data[0]... In your glTexImage2D call use GL_RGBA instead of GL_RGB. Thats what I did to get it working anyway. Assuming you have your texture and vertex coordinates created correctly then that should get it working. I can post some code examples if that doesn't help.