Can ffmpeg extract closed caption data

Question

I am currently using ffmpeg to convert videos in various formats to flv files. One request has also come up and that is to get closed caption info out o the file as well. Does anyone have any experience with this or know it can even be done. I don't see any options for it but thought I would ask and see.

Answer 1

For getting just the subtitles and not any meta junk, I've found that

ffmpeg -i input.mov -an -vn -bsf:s mov2textsub -scodec copy -f rawvideo sub.txt 

works best for me.

Answer 2

If anyone, like me, ends up on this thread, here's a bit more detailed explanation on ffmpeg command that worked for me.

ffmpeg -f lavfi -i movie=input.ts[out+subcc]  -map 0:1  output.srt

There seems a hard requirement on source to be of mpegts format (file extension .ts). Otherwise the lavfi filter does not seem to work. The spec out+subcc forces ffmpeg to treat closed captions (which are embedded into frame data) as separate stream. Later -map 0:1 makes ffmpeg map only that stream and discard everything else. Result is saved to output.srt. Depending on your input the mapping might be different. One easy way to figure out the closed captions mapping is to run ffprobe command, like so

$ ffprobe -f lavfi -i movie=input.ts[out+subcc]
ffprobe version N-79653-g4efd3ec Copyright (c) 2007-2016 the FFmpeg developers
  libavutil      55. 22.101 / 55. 22.101
  libavcodec     57. 38.100 / 57. 38.100
  libavformat    57. 34.103 / 57. 34.103
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 44.100 /  6. 44.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
[h264 @ 0x7fe869826200] Increasing reorder buffer to 1
Input #0, lavfi, from 'movie=input.ts[out+subcc]':
  Duration: N/A, start: 1562.233011, bitrate: N/A
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 90k fps, 30 tbr, 90k tbn
    Stream #0:1: Subtitle: eia_608

Stream Subtitle: eia_608 has "index" 0:1, so that is what should be mapped.

Few parting notes, order of arguments matters for ffmpeg, -f lavfi must go before -i move=..., otherwise the spec will not be recognized. Also this feature is pretty recent, so double check your ffmpeg version and upgrade if needed.

Answer 3

If the caption are included as a separate stream, then extracting them is (relatively) straightforward:

ffmpeg -i input.mov -an -vn -c:s copy -f rawvideo -map 0:s sub.txt

If it's "burned in" on the video file, then you're probably out of luck, but I that would be more common for subtitles than closed captions.

Answer 4

Closed Captions are not separate streams, nor are they burned into the picture, they are interlaced in scanlines IN the picture - this a strange American idea, not really used elsewhere. Though you can buy DVD's which has this kind of subtitling, usually people outside the US can't see the subtitles, and may not even know there are some in the file.

CCExtractor can extract this information from VOB's by looking at the TS metadata: http://ccextractor.sourceforge.net/

Answer 5

Closed caption are of 2 format
1) ATSC American standard (support is there in ffmpeg)
2) ISDB Japanese standard (support is yet not there in ffmpeg)

you can use following command

ffmpeg -f lavfi -i "movie=test.ts[out0+subcc]" -map s output.srt

This thing has been recently developed so please check out your version of ffmpeg.