Video Capturing in Java and Processing

While it’s already quite easy to create videos from processing sketches using the saveFrame() function and a video converting tool like the FFmpeg command line tool, this approach comes with some drawbacks. You need relatively large amounts of hard disk space, as each single frame is stored separately in it’s own file. Besides, it’s also quite slow to export videos like this. For longer recording, this approach becomes practically unusable.

The good news is, that it only takes a few lines of code to get rid of the detour of exporting each frame separately to individual files, if you already have FFmepg (Libav tools respectively) installed. The core idea here is to call FFmpeg as a process and to directly pass the framebuffer using stdout.
Of course this approach does not only apply to processing, but to any other languages that supports system calls and pipes. The following function calls FFmpeg and returns an OutputStream, that accepts new frames to be written to the video file.

OutputStream videoOutputStream(String fname, float fps) {
  try {
    return new ProcessBuilder(
      "avconv", // or "ffmepg"
      "-f", "rawvideo", // input format
      "-pix_fmt", "argb", // pixel format
      "-r", fps+"", // frame rate
      "-s", width+"x"+height, // input size (window size)
      "-i", "-", // input (stdin)
      "-y", // force overwrite
      "-qscale", "0", // highest quantization quality profile,
      fname // output file
      // inherit stderr to catch ffmpeg errors
    ).redirectError(ProcessBuilder.Redirect.INHERIT).start().getOutputStream(); 
  } catch (Exception e) {e.printStackTrace(); return null;}    
}

To transfer the framebuffer to the encoding process, we need to load it into the pixels[] array using loadPixels(), convert it into a byte array, and write it to the OutputStream.

import java.nio.*;

ByteBuffer byteBuffer;
IntBuffer intBuffer;
OutputStream video;

void setup() {
  size(640,480);
  video = videoOutputStream("test.avi", 30);
  byteBuffer = ByteBuffer.allocate(width * height * 4);
  intBuffer = byteBuffer.asIntBuffer();
}

void draw() {
  background(0);
  text("time: "+(millis()/1000.0)+"s", 10, 20);
  
  loadPixels();
  try {
    intBuffer.rewind();
    intBuffer.put(pixels);
    video.write(byteBuffer.array());
  } catch (IOException ioe) {}
}

As a small demonstration of this technique, I exported Shiffman’s Particles example sketch.