I've been using grafika's MoviePlayer and bigFlake's ExtractMpegFramesTest to implement a seeking and extract frame feature in our app. These work fine for most of our user, however some of them encounter a IllegalStateException
on MediaCodec.configure
when setting up the ExtractMpegFramesTest
(this happens on Samsung Galaxy S4 mini, Samsung Galaxy J7, Samsung Galaxy A5, Huawei Ascend G7). The relevant code is as the following:
MoviePlayer
public void prepareResources() throws IOException {
// The MediaExtractor error messages aren't very useful. Check to see if the input
// file exists so we can throw a better one if it's not there.
if (!mSourceFile.canRead()) {
throw new FileNotFoundException("Unable to read " + mSourceFile);
}
try {
mExtractor = new MediaExtractor();
mExtractor.setDataSource(mSourceFile.toString());
mTrackIndex = selectTrack(mExtractor);
if (mTrackIndex < 0) {
throw new RuntimeException("No video track found in " + mSourceFile);
}
mExtractor.selectTrack(mTrackIndex);
MediaFormat format = mExtractor.getTrackFormat(mTrackIndex);
// Create a MediaCodec decoder, and configure it with the MediaFormat from the
// extractor. It's very important to use the format from the extractor because
// it contains a copy of the CSD-0/CSD-1 codec-specific data chunks.
String mime = format.getString(MediaFormat.KEY_MIME);
mDecoder = MediaCodec.createDecoderByType(mime);
mDecoder.configure(format, mOutputSurface, null, 0);
mDecoder.start();
mDecoderInputBuffers = mDecoder.getInputBuffers();
} catch (Exception e) {
e.printStackTrace();
}
}
ExtractMpegFramesTest
private void extractMpegFrames() throws IOException {
MediaCodec decoder = null;
CodecOutputSurface outputSurface = null;
MediaExtractor extractor = null;
/*int saveWidth = 640;
int saveHeight = 480;*/
try {
long start1 = System.currentTimeMillis();
File inputFile = new File(mInputVideoPath); // must be an absolute path
// The MediaExtractor error messages aren't very useful. Check to see if the input
// file exists so we can throw a better one if it's not there.
if (!inputFile.canRead()) {
throw new FileNotFoundException("Unable to read " + inputFile);
}
extractor = new MediaExtractor();
extractor.setDataSource(inputFile.toString());
int trackIndex = selectTrack(extractor);
if (trackIndex < 0) {
throw new RuntimeException("No video track found in " + inputFile);
}
extractor.selectTrack(trackIndex);
MediaFormat format = extractor.getTrackFormat(trackIndex);
if (VERBOSE) {
Log.d(TAG, "Video size is " + format.getInteger(MediaFormat.KEY_WIDTH) + "x" +
format.getInteger(MediaFormat.KEY_HEIGHT));
}
// Could use width/height from the MediaFormat to get full-size frames.
outputSurface = new CodecOutputSurface(format.getInteger(MediaFormat.KEY_WIDTH), format.getInteger(MediaFormat.KEY_HEIGHT));
// Create a MediaCodec decoder, and configure it with the MediaFormat from the
// extractor. It's very important to use the format from the extractor because
// it contains a copy of the CSD-0/CSD-1 codec-specific data chunks.
String mime = format.getString(MediaFormat.KEY_MIME);
decoder = MediaCodec.createDecoderByType(mime);
decoder.configure(format, outputSurface.getSurface(), null, 0); //fails right here
decoder.start();
long end1 = System.currentTimeMillis();
Timber.d("extractMpegFrames(): FRAME_EXTRACT setup in: %d millis", (end1-start1));
long start2 = System.currentTimeMillis();
doExtract(extractor, trackIndex, decoder, outputSurface);
long end2 = System.currentTimeMillis();
Timber.d("extractMpegFrames(): FRAME_EXTRACT doExtract in: %d millis", (end2-start2));
} finally {
// release everything we grabbed
if (outputSurface != null) {
outputSurface.release();
outputSurface = null;
}
if (decoder != null) {
decoder.stop();
decoder.release();
decoder = null;
}
if (extractor != null) {
extractor.release();
extractor = null;
}
}
}
I'm aware of this question, however what I don't understand is why MediaCodec
can be configured OK in MoviePlayer
(the video runs just fine) but it fails in ExtractMpegFramesTest
. At first, I think the device has some issues with the OpenGL set up, but it turn out to be MediaCodec problem.
Any insight would be greatly appreciated.
EDIT: After obtaining a Galaxy S4 mini testing device, I've been able to get a much more helpful log:
D/MoviePlayer: Extractor selected track 0 (video/avc): {max-input-size=1572864, height=1080, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20], width=1920, durationUs=5131211, csd-1=java.nio.ByteArrayBuffer[position=0,limit=9,capacity=9], mime=video/avc, isDMCMMExtractor=1}
D/MoviePlayer: Video size is 1920x1080
D/MoviePlayer: Extractor selected track 0 (video/avc): {max-input-size=1572864, height=1080, csd-0=java.nio.ByteArrayBuffer[position=0,limit=20,capacity=20], width=1920, durationUs=5131211, csd-1=java.nio.ByteArrayBuffer[position=0,limit=9,capacity=9], mime=video/avc, isDMCMMExtractor=1}
I/OMXClient: Using client-side OMX mux.
E/ACodec: [OMX.qcom.video.decoder.avc] storeMetaDataInBuffers failed w/ err -2147483648
E/ACodec: configureCodec multi window instance fail appPid : 3283
E/ACodec: [OMX.qcom.video.decoder.avc] configureCodec returning error -38
E/MediaCodec: Codec reported an error. (omx error 0x80001001, internalError -38)
W/System.err: java.lang.IllegalStateException
W/System.err: at android.media.MediaCodec.native_configure(Native Method)
W/System.err: at android.media.MediaCodec.configure(MediaCodec.java:262)
W/System.err: at co.test.testing.player.MoviePlayer.prepareResources(MoviePlayer.java:237)
W/System.err: at co.test.testing.activities.VideoActivity.surfaceCreated(VideoActivity.java:276)
W/System.err: at android.view.SurfaceView.updateWindow(SurfaceView.java:602)
W/System.err: at android.view.SurfaceView.access$000(SurfaceView.java:94)
W/System.err: at android.view.SurfaceView$3.onPreDraw(SurfaceView.java:183)
W/System.err: at android.view.ViewTreeObserver.dispatchOnPreDraw(ViewTreeObserver.java:891)
W/System.err: at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:2201)
W/System.err: at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:1256)
W/System.err: at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:6635)
W/System.err: at android.view.Choreographer$CallbackRecord.run(Choreographer.java:813)
W/System.err: at android.view.Choreographer.doCallbacks(Choreographer.java:613)
W/System.err: at android.view.Choreographer.doFrame(Choreographer.java:583)
W/System.err: at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:799)
W/System.err: at android.os.Handler.handleCallback(Handler.java:733)
W/System.err: at android.os.Handler.dispatchMessage(Handler.java:95)
W/System.err: at android.os.Looper.loop(Looper.java:146)
W/System.err: at android.app.ActivityThread.main(ActivityThread.java:5593)
W/System.err: at java.lang.reflect.Method.invokeNative(Native Method)
W/System.err: at java.lang.reflect.Method.invoke(Method.java:515)
W/System.err: at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1283)
W/System.err: at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1099)
W/System.err: at dalvik.system.NativeStart.main(Native Method)
These are the codec info retrieved from devices having this problem:
OMX.qcom.video.encoder.avc
OMX.SEC.avc.enc
OMX.Exynos.AVC.Encoder
The strange thing is the Galaxy S4 mini can handle 720p videos just fine, it's just having problem with 1080p videos.