How to extract orientation information from videos

Question

After surfing through tons of documentation on the web it seems that the iPhone always shoots the video at a 480x360 aspect ratio and applies a transformation matrix on the video track. (480x360 may change but its always the same for a given device)

Here is a way of modifying the ffmpeg source within a iOS project and accessing the matrix http://www.seqoy.com/correct-orientation-for-iphone-recorded-movies-with-ffmpeg/

Here is a cleaner way of finding the transformation matrix in iOS-4 How to detect (iPhone SDK) if a video file was recorded in portrait orientation, or landscape.

How can the orientation of the video be extracted in either of the options below -
- iOS 3.2
- ffmpeg (through the command line server side)
- ruby

Any help will be appreciated.

Answer 1

From what I've found thus far, ffmpeg doesn't have the ability to detect iPhone's orientation. But, the open source library, mediainfo can. A command line example:

$ mediainfo test.mp4 | grep Rotation
Rotation                         : 90°

More example output from the same iphone video:

Video
ID                               : 1
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : Baseline@L3.0
Format settings, CABAC           : No
Format settings, ReFrames        : 1 frame
Codec ID                         : avc1
Codec ID/Info                    : Advanced Video Coding
Duration                         : 7s 941ms
Bit rate mode                    : Variable
Bit rate                         : 724 Kbps
Width                            : 480 pixels
Height                           : 360 pixels
Display aspect ratio             : 4:3
Rotation                         : 90°
Frame rate mode                  : Variable
Frame rate                       : 29.970 fps
Minimum frame rate               : 28.571 fps
Maximum frame rate               : 31.579 fps
Color space                      : YUV
Chroma subsampling               : 4:2:0
Bit depth                        : 8 bits
Scan type                        : Progressive
Bits/(Pixel*Frame)               : 0.140
Stream size                      : 702 KiB (91%)
Title                            : Core Media Video
Encoded date                     : UTC 2011-06-22 15:58:25
Tagged date                      : UTC 2011-06-22 15:58:34
Color primaries                  : BT.601-6 525, BT.1358 525, BT.1700 NTSC, SMPTE 170M
Transfer characteristics         : BT.709-5, BT.1361
Matrix coefficients              : BT.601-6 525, BT.1358 525, BT.1700 NTSC, SMPTE 170M

Answer 2

Similar to @HdN8's answer, but without the python regex:

$ ffprobe   -show_streams any.MOV  2>/dev/null  | grep rotate
TAG:rotate=180

Or JSON:

$ ffprobe -of json  -show_streams IMG_8738.MOV  2>/dev/null  | grep rotate
"rotate": "180",

Or you could parse the JSON (or other output format).

Answer 3

You can use ffprobe. No need for any grep, or any other additional processes, or any regex operations to parse the output as shown in other answers.

If you want the rotate metadata:

Command:

ffprobe -loglevel error -select_streams v:0 -show_entries stream_tags=rotate -of default=nw=1:nk=1 input.mp4

Example output:

90

If you want the display matrix rotation side data:

Command:

ffprobe -loglevel error -select_streams v:0 -show_entries side_data=rotation -of default=nw=1:nk=1 input.mp4

Example output:

-90

If you want the display matrix:

Command:

ffprobe -loglevel error -select_streams v:0 -show_entries side_data=displaymatrix -of default=nw=1:nk=1 input.mp4

Example output:

00000000:            0       65536           0
00000001:       -65536           0           0
00000002:     15728640           0  1073741824

---

What the options mean

• -loglevel error Omit the header and other info from output.

• -select_streams v:0 Only process the first video stream and ignore everything else. Useful if your input contains multiple video streams and you only want info from one.

• -show_entries stream_tags=rotate Chooses to output the rotate tag from the video stream.

• -of default=nw=1:nk=1 Use default output format, but omit including the section header/footer wrappers and each field key.

Output format

The output from ffprobe can be formatted in several ways. For example, JSON:

ffprobe -loglevel error -show_entries stream_tags=rotate -of json input.mp4
{
    "streams": [
        {
            "tags": {
                "rotate": "90"
            },
            "side_data_list": [
                {

                }
            ]
        }
    ]

Answer 4

I have extracted on iOS using an AVAssetExportSession, AVMutableComposition and the input AVAssetTrack's preferredTransform. I concatenate the preferred transform with a transformation to fill the target size.

After exporting to a file, I upload using ASIHTTPRequest to my rails server and send the data to Amazon S3 using paperclip.

Answer 5

ffmpeg reports the metadata with the rotation value for .mov files:

ffmpeg -i myrotatedMOV.mov

....

Duration: 00:00:14.31, start: 0.000000, bitrate: 778 kb/s
    Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p, 480x360, 702 kb/s, 29.98 fps, 30 tbr, 600 tbn, 1200 tbc
    Metadata:
      rotate          : 180
      creation_time   : 2013-01-09 12:47:36
      handler_name    : Core Media Data Handler
    Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, s16, 62 kb/s
    Metadata:
      creation_time   : 2013-01-09 12:47:36
      handler_name    : Core Media Data Handler

In my application I pull it out with regex, ie in python:

import subprocess, re    
cmd = 'ffmpeg -i %s' % pathtofile

p = subprocess.Popen(
    cmd.split(" "),
    stderr = subprocess.PIPE,
    close_fds=True
)
stdout, stderr = p.communicate()

reo_rotation = re.compile('rotate\s+:\s(?P<rotation>.*)')
match_rotation = reo_rotation.search(stderr)
rotation = match_rotation.groups()[0]

I havent tried this with a wide range of videos, only a couple .movs recorded from an iphone5, using ffmpeg version 1.0. But so far so good.

Answer 6

Since most Cameras store their rotation/orientation within the exif-metadata, i would suggest using exifttool and the a ruby wrapper gem called mini_exiftool which is actively maintained.

Install exiftool:

apt-get exiftool || brew install exiftool || port install exiftool

or use what ever package manager is available

Install mini_exiftool:

gem install mini_exiftool

Try it:

irb>
require 'mini_exiftool'
movie = MiniExiftool.new('test_movie.mov')
movie.orientation #=> 90

cheers