notes code

Native video processing in Python

I’m refactoring the image processing functions of the Cifonauta database and wondering if there’s any Python library with native video processing capabilities.

Bryozoan footage after video processing on the Cifonauta website.
Page with video footage of a marine organism on the Cifonauta database.

For the past years, I used a to run a FFmpeg command to scale down and convert video files for the Cifonauta website (which is Django-powered). The ffmpeg command has some niceties like adding a watermark and including some metadata in the output file. The code looks like this:

# Import subprocess module
import subprocess
# Build FFmpeg command
ffmpeg_call = [
    'ffmpeg', # Executable
    '-y', # Overwrite output files
    '-hide_banner', # Don't print software information
    '-loglevel', 'error', # Show errors
    '-threads', '0', # Use optimal thread number
    '-i', input_path, # Path to input video file
    '-i', watermark, # Path to input watermark
    '-metadata', f'title={metadata.title}', # Add title to video metadata
    '-metadata', f'artist={}', Add author to video metadata
    '-b:v', '600k', # Set bitrate for small file size
    '-filter_complex', 'scale=512:-2,overlay=0:main_h-overlay_h-0', # Scale input and overlay watermark at bottom left corner
# Run FFmpeg command

Note: This is a relatively old command. It might not be up-to-date with the latest FFmpeg versions.

Despite its complex syntax, I love FFmpeg and use it all the time, from creating GIFs to scientific videos. However, a native video processing library in Python would simplify the Cifonauta project’s dependencies and facilitate contributions. After all, adding a line to pip requirements.txt and having pythonic methods to manipulate videos is easier than figuring out and editing FFmpeg complex filters, at least for new contributors.

For images, I accomplished this by replacing the subprocess calls to ImageMagick (another open-source project I love) with Pillow. Although I’m not a fan of Pillow’s methods—why Image.convert() creates a copy and Image.thumbnail() modifies the image in place?—it’s very convenient to have a native Python solution for manipulating images.

Here are some of the libraries I looked into:

  • ffmpeg-python: Seemed like the obvious choice, with many features and nice syntax. But unmaintained since ~2019.
  • python-ffmpeg: Similar library (confusing name) but newer and actively maintained. Promising but with fewer methods and a smaller community.
  • vidgear: Actively maintained, large community, and packed with features. Too many for the Cifonauta’s use case. And it’s still a wrapper for FFmpeg.
  • moviepy: Still maintained, but seems more focused on video editing and manipulation than video processing (scaling/converting formats).
  • opencv-python: Looks super-advanced, perhaps too much for what we need.
  • PyAV: Another wrapper around FFmpeg, but the bindings look great to get and access video information. And the project is still maintained. I only couldn’t find a way to perform the scaling/conversion routines.
  • scikit-video: Also looks good and might be a good fit, since there are methods to extract metadata, generate thumbnails, and convert videos. But the latest release is from 2018, although there are some recent commits in the repository.

Unfortunately, most (if not all) libraries are wrappers for FFmpeg. That means I would still need to have ffmpeg binaries installed. In this case, I don’t think it’s worth it to add yet another dependency with custom methods to the project. Moreover, in terms of functionality, none of the libraries were a perfect match for the Cifonauta pipeline. And there’s always the risk of the library you chose to be no longer maintained—it happens often!

So I think it’ll be safer and more practical to continue with my custom subprocess calls to FFmpeg for now.

Did I miss any obvious library? Please let me know!

Reply by Email


Leave a Comment

Your email address will not be published. Required fields are marked *