How to read a video frame by frame?

image-processing, java, video

Solution

Marvin Framework provides methods to process media files frame by frame. Below it is shown a simple video processing example that highlights the path of a snooker ball.

At the left the video is played frame by frame with an interval of 30ms between frames. At the right the video is played frame by frame, but keeping all positions of the white ball through a simple image processing approach.

The source code can be checked here and the video here.

Below the essential source code to request media file frames using Marvin:

public class MediaFileExample implements Runnable{

    private MarvinVideoInterface    videoAdapter;
    private MarvinImage             videoFrame;

    public MediaFileExample(){
        try{
            // Create the VideoAdapter used to load the video file
            videoAdapter = new MarvinJavaCVAdapter();
            videoAdapter.loadResource("./res/snooker.wmv");

            // Start the thread for requesting the video frames 
            new Thread(this).start();
        }
        catch(MarvinVideoInterfaceException e){e.printStackTrace();}
    }

    @Override
    public void run() {
        try{
            while(true){
                // Request a video frame
                videoFrame = videoAdapter.getFrame();
            }
        }catch(MarvinVideoInterfaceException e){e.printStackTrace();}
    }

    public static void main(String[] args) {
        MediaFileExample m = new MediaFileExample();
    }
}

Problem

i would like to read a Mp4 file in java8-64bit frame by frame and write each frame as a jpg to my harddisk. my first attempt was to use JavaFX 2.2 media player to play the file on a View component. i thought maybe there would be an option to register an observer to get an event each time a new frame was loaded and ready to be painted on the component surface but seems there is no such method. it would be enough to grab just those frames/pixels that got painted on the component. Can this be done by using the media player? the reason why i use the media player is bcs it was the simplest solution i got workin. i tryed vlcj, just 32bit, and gstreamer but without luck :( what i got so far: ``` public class VideoGrabber extends extends JFrame { // code for scene setup omitted final MediaView view = createMediaView(...) // some other stuff happens here // now start the video view.getMediaPlayer().seek(Duration.ZERO); view.getMediaPlayer().play(); view.getMediaPlayer().setOnEndOfMedia(new Runnable() { // save image when done BufferedImage img = new BufferedImage(getWidth(), getHeight(), BufferedImage.TYPE_INT_BGR view.paint(img.getGraphics()); ImageIO.write(img, "JPEG", new File("pic-"+System.currentTimeMillis()+".jpg")); }); // somewhere else to create private MediaView createMediaView(String url) { final Media clip = new Media(url); final MediaPlayer player = new MediaPlayer(clip); final MediaView view = new MediaView(player); view.setFitWidth(VID_WIDTH); view.setFitHeight(VID_HEIGHT); return view; } ``` is there somehow a way to do the following: ``` player.setOnNextFrameReady(final Event evt) { writeImage(evt.getFrame()) }; ``` Thanks!

Original source

Related problems