In my program I am piping a webm from a stream to ffmpeg and then pipe the output to a http request. Part of the process is adding metadata for the mp3. This has so far worked great. However after adding an image as album art it has started to act unexpected.
First this is the command line I am using inside the program:
val parameters = listOf("ffmpeg",
"-i", "-",
"-i", albumImage.absolutePath,
"-map", "0",
"-map", "1",
"-c:v", "copy",
"-f", "mp3",
"-id3v2_version", "4",
"-metadata", "title=${info.title}",
"-metadata", "album=YouTube",
"-metadata", "artist=${info.author}",
"-metadata:s:v", "title=Album Cover",
"-metadata:s:v", "comment=Cover (front)",
"-"
)
It creates a valid mp3 file and I can find both the metadata and the image in the mp3 file, however when playing it none of them are displayed in VLC or anywhere else. To test various configurations I have converted it to the command line.
In a first try I have saved the video and the image and stopped using pipes altogether, which results in this:
ffmpeg -i video.webm -i image.jpeg -map 0 -map 1 -c:v copy -f mp3 -id3v2_version 4 -metadata title="Tiësto & KSHMR feat. Vassy - Secrets (Official Music Video)" -metadata album="YouTube" -metadata artist="Spinnin' Records" -metadata:s:v title="Album Cover" -metadata:s:v comment="Cover (front)" output3.mp3
In this case all metadata including the album art is displayed in VLC.
I then recreated the same thing as in my program, piping both video input and audio output, looking like this:
ffmpeg -i - -i image.jpeg -map 0 -map 1 -c:v copy -f mp3 -id3v2_version 4 -metadata title="Tiësto & KSHMR feat. Vassy - Secrets (Official Music Video)" -metadata album="YouTube" -metadata artist="Spinnin' Records" -metadata:s:v title="Album Cover" -metadata:s:v comment="Cover (front)" - < video.webm > output3.mp3
This file is the same as my programs output. Neither title nor album nor album image are displayed (however it can play the file)
To test a few more options I have hardcoded the output file but pipe the input file like this:
ffmpeg -i - -i image.jpeg -map 0 -map 1 -c:v copy -f mp3 -id3v2_version 4 -metadata title="Tiësto & KSHMR feat. Vassy - Secrets (Official Music Video)" -metadata album="YouTube" -metadata artist="Spinnin' Records" -metadata:s:v title="Album Cover" -metadata:s:v comment="Cover (front)" output3.mp3 < video.webm
Now the metadata is working again. When hardcoding the input video and piping the output, its again gone.
So to sum up: When piping the output of ffmpeg the metadata in the file is not properly working. Interestingly the stderr output of ffmpeg looks quite similar
Hardcoded output3.mp3:
ffmpeg version 3.4.4-0ubuntu0.18.04.1 Copyright (c) 2000-2018 the FFmpeg developers
Input #0, matroska,webm, from 'pipe:':
Metadata:
encoder : google/video-file
Duration: 00:03:39.58, start: -0.007000, bitrate: N/A
Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
Input #1, image2, from 'image.jpeg':
Duration: 00:00:00.04, start: 0.000000, bitrate: 1466 kb/s
Stream #1:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 320x180, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
Stream #0:0 -> #0:0 (opus (native) -> mp3 (libmp3lame))
Stream #1:0 -> #0:1 (copy)
Output #0, mp3, to 'output3.mp3':
Metadata:
TPE1 : Spinnin' Records
TIT2 : Tiësto & KSHMR feat. Vassy - Secrets (Official Music Video)
TALB : YouTube
TSSE : Lavf57.83.100
Stream #0:0(eng): Audio: mp3 (libmp3lame), 48000 Hz, stereo, fltp (default)
Metadata:
encoder : Lavc57.107.100 libmp3lame
Stream #0:1: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 320x180, q=2-31, 25 tbr, 25 tbn, 25 tbc
Metadata:
title : Album Cover
comment : Cover (front)
With pipe output:
ffmpeg version 3.4.4-0ubuntu0.18.04.1 Copyright (c) 2000-2018 the FFmpeg developers
Input #0, matroska,webm, from 'pipe:':
Metadata:
encoder : google/video-file
Duration: 00:03:39.58, start: -0.007000, bitrate: N/A
Stream #0:0(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
Input #1, image2, from 'image.jpeg':
Duration: 00:00:00.04, start: 0.000000, bitrate: 1466 kb/s
Stream #1:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 320x180, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
Stream #0:0 -> #0:0 (opus (native) -> mp3 (libmp3lame))
Stream #1:0 -> #0:1 (copy)
Output #0, mp3, to 'pipe:':
Metadata:
TPE1 : Spinnin' Records
TIT2 : Tiësto & KSHMR feat. Vassy - Secrets (Official Music Video)
TALB : YouTube
TSSE : Lavf57.83.100
Stream #0:0(eng): Audio: mp3 (libmp3lame), 48000 Hz, stereo, fltp (default)
Metadata:
encoder : Lavc57.107.100 libmp3lame
Stream #0:1: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 320x180, q=2-31, 25 tbr, 25 tbn, 25 tbc
Metadata:
title : Album Cover
comment : Cover (front)
Yes, the ID3 header size cannot be filled in when the ID3v2 metadata has to be written in two steps (such as when an image packet has to be inserted) & the output is not seekable.
You can still work around this to a degree by telling ffmpeg to not flush the data quickly. However, ffmpeg will flush if its buffer exceeds 256 kB. Make a small allowance for the other parts of the ID3 header, and that gives you a ceiling for the maximum size of the image.
ffmpeg -i - -i image.jpeg -map 0 -map 1 -c:v copy -f mp3 -id3v2_version 4 -metadata title="Tiësto & KSHMR feat. Vassy - Secrets (Official Music Video)" -metadata album="YouTube" -metadata artist="Spinnin' Records" -metadata:s:v title="Album Cover" -metadata:s:v comment="Cover (front)" -flush_packets 0 - > output3.mp3 < video.webm