Adding transparency to a video from black and whit

Question

I'd like to create a video with transparency (and semi transparency) in android. The images of the source video are split into two images, the top one with color information (looks like the normal video image), the bottom one with alpha information (the same shapes but only in black and white and gray, black means transparent).

This is the solution for iOS: https://medium.com/@quentinfasquel/ios-transparent-video-with-coreimage-52cfb2544d54

What would be the best way to to this in android?

Can this be solved with javacv, FFmpegFrameGrabber, FFmpegFrameFilter etc.?

Or better with OpenGL ES and shaders?

Are there any samples?

Thanks!

Answer 1

We use exactly the same approach to add transparency to videos in our android app. We use OpenGL ES with shaders and it works flawlessly. What you need to do is just render those video to surface texture (can be easily done using ExoPlayer) and afterwards it can be accessed from OpenGL as regular texture. For the shader you can start from something like this

private static final String VERTEX_SHADER = "attribute vec4 position;\n" +
        "attribute vec4 inputTextureCoordinate;\n" +
        " \n" +
        "varying vec2 textureCoordinate;\n" +
        "varying vec2 textureCoordinate2;\n" +
        " \n" +
        "void main()\n" +
        "{\n" +
        "    gl_Position = position;\n" +
        "    textureCoordinate.x = inputTextureCoordinate.x;\n" +
        "    textureCoordinate.y = inputTextureCoordinate.y * 0.5;\n" +
        "    textureCoordinate2.x = inputTextureCoordinate.x;\n" +
        "    textureCoordinate2.y = inputTextureCoordinate.y * 0.5 + 0.5;\n" +
        "}";


public static final String FRAGMENT_SHADER = "#extension GL_OES_EGL_image_external : require\n"+
        "varying highp vec2 textureCoordinate;\n"+
        "varying highp vec2 textureCoordinate2;\n"+
        "uniform samplerExternalOES inputImageTexture;\n" +
        "void main() {\n"+
        "    lowp vec4 rgbcolor = texture2D(inputImageTexture, textureCoordinate);\n"+
        "    lowp vec4 alphaValue = texture2D(inputImageTexture, textureCoordinate2);\n"+
        "    gl_FragColor.a = alphaValue.g;\n"+
        "    gl_FragColor.rgb = rgbcolor.rgb * alphaValue.g;\n" +
        "}";

and dont forget to render this into a TextureView and enable transparency with TextureView.setOpaque(false);