what is 'linesize alignment' meaning?

Question

I'm following ffmpeg tutorial in http://dranger.com/ffmpeg/tutorial01.html.

I have just found that avpicture_get_size function is deprecated.

So I have checked ffmpeg's document(https://www.ffmpeg.org/doxygen/3.0/group__lavu__picture.html#ga24a67963c3ae0054a2a4bab35930e694) and found substitute av_image_get_buffer_size.

But I can't understand align parameter meaning 'linesize alignment'......

What is it meaning?

Answer 1

If you look at the definition of avpicture_get_size in version 3.2 you see the following code:

int avpicture_get_size(enum AVPixelFormat pix_fmt, int width, int height)
{
    return av_image_get_buffer_size(pix_fmt, width, height, 1);
}

It simply calls the suggested function: av_image_get_buffer_size with the align parameter set to 1. I did not go further to find out the full significance of why 1 is used for the depreciated function. As usual with ffmpeg, one can probably figure it out by reading the right code and enough code (with some code experiments).

Answer 2

Some parts of FFmpeg, notably libavcodec, require aligned linesizes[], which means that it requires:

assert(linesize[0] % 32 == 0);
assert(linesize[1] % 32 == 0);
assert(linesize[2] % 32 == 0);

This allows it to use fast/aligned SIMD routines (for example SSE2/AVX2 movdqa or vmovdqa instructions) for data access instead of their slower unaligned counterparts.

The align parameter to this av_image_get_buffer_size function is this line alignment, and you need it because the size of the buffer is affected by it. E.g., the size of a Y plane in a YUV buffer isn't actually width * height, it's linesize[0] * height. You'll see that (especially for image sizes that are not a multiple of 16 or 32), as you increase align to higher powers of 2, the return value slowly increases.

Practically speaking, if you're going to use this picture as output buffer for calls to e.g. avcodec_decode_video2, this should be 32. For swscale/avfilter, I believe there is no absolute requirement, but you're recommended to still make it 32.

Answer 3

My practice：

1.avpicture deprecated problem, I replace avpicture functions with AVFrame & imgutils functions. code sample:

   //AVPicture           _picture;
   AVFrame             *_pictureFrame;
   uint8_t             *_pictureFrameData;

...

//_pictureValid = avpicture_alloc(&_picture,
//                                AV_PIX_FMT_RGB24,
//                               _videoCodecCtx->width,
//                               _videoCodecCtx->height) == 0;

_pictureFrame = av_frame_alloc();
_pictureFrame->width = _videoCodecCtx->width;
_pictureFrame->height = _videoCodecCtx->height;
_pictureFrame->format = AV_PIX_FMT_RGB24;

int size = av_image_get_buffer_size(_pictureFrame->format,
                                    _pictureFrame->width,
                                    _pictureFrame->height,
                                    1);

//dont forget to free _pictureFrameData at last                                    
_pictureFrameData = (uint8_t*)av_malloc(size);

 av_image_fill_arrays(_pictureFrame->data,
                      _pictureFrame->linesize,
                      _pictureFrameData,
                      _pictureFrame->format,
                      _pictureFrame->width,
                      _pictureFrame->height,
                      1);

...

if (_pictureFrame) {
    av_free(_pictureFrame);
    if (_pictureFrameData) {
        free(_pictureFrameData);
    }
}

2.align parameter

first I set align to 32, but for some video streams it did not work, cause distorted images. Then I set it to 16(my environment : mac, Xcode, iPhone6), the some streams works well. But at last i set align to 1, for I have found this

Fill in the AVPicture fields, always assume a linesize alignment of 1.