Video Processing Library for Java

Question

I want to extract frames from a video and apply some filters on it such as gabor/hough etc. Which Java library would be perfect for handling all kinds of video encodings? I have been looking at GStreamer, JMF, Xuggler etc. but am unable to decide which one would be the best. I'm also looking to edit the frames and make the video with the new frames.

Answer 1

You can try Marvin Framework. It uses JavaCV for video encodings and device access, but all image processing algorithms are pure Java.

It's very easy to load a video and process the frames in real time, as shown in the edge detection example below.

Source code:

import static marvin.MarvinPluginCollection.*;

public class SimpleVideoProcessing extends JFrame implements Runnable{

    private MarvinVideoInterface    videoAdapter = new MarvinJavaCVAdapter();
    private MarvinImagePanel        videoPanel = new MarvinImagePanel();
    private MarvinImage             videoFrame, videoOut = new MarvinImage(640,480);

    public SimpleVideoProcessing() throws MarvinVideoInterfaceException{
        super("Simple Video Processing using Marvin");
        add(videoPanel);
        // Load video file and start the processing thread
        videoAdapter.loadResource("./res/snooker.wmv");
        new Thread(this).start();
        setSize(640,500);
        setVisible(true);
    }

    public void run() {
        try {
            while(true){
                // Request, process and show the video frame.
                videoOut.clear();
                videoFrame = videoAdapter.getFrame();
                prewitt(videoFrame.clone(), videoOut);
                videoPanel.setImage(videoOut);
            }
        } catch (MarvinVideoInterfaceException e) {
            e.printStackTrace();
        }
    }

    public static void main(String[] args) throws MarvinVideoInterfaceException {
        new SimpleVideoProcessing().setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
    }
}

Answer 2

If you're looking to do low level operations such as extracting frames and manipulating them, then Xuggler would be the best choice, because the APIs are geared around this low level. It works on ffmpeg so can handle all types of video encodings.

Don't use JMF for anything, it's old, outdated and buggy - GStreamer is good, but the API lends itself more to playing videos rather than manipulating the frames.

Answer 3

JMF is a good choice. But if the process time is important in your code, it's better to use Xuggler. Obviously, JMF is more general than Xuggler.

Answer 4

Xuggler, yes. But if you are going to be working on a lot of Image processing, you should go with OpenImaj. This library uses Xuggler as its dependency, but that's not all what it does. Think of having capabilities of Opencv without the lack of speed which you get in Java. Also, all it requires is adding a maven dependency and you are good to go. The amount of code is also reduced.

Note: I am still reviewing the library and will keep updating my answer on how this goes.

Introductory Video: https://www.youtube.com/watch?v=TNEQ0eNqLgA