dummy-link

VideoIO

Julia bindings for ffmpeg

Readme

Build Status Appveyor Build Coverage Status

VideoIO.jl

Julia bindings for ffmpeg, using a dedicated installation of ffmpeg 4.1.

Reading and writing of videos is supported.

Video images may be read as raw arrays, or optionally, Image objects (if Images.jl is installed and loaded first).

Videos can be encoded from image stacks (a vector of same-sized Image objects), or encoded iteratively in custom loops.

Installation

Use the Julia package manager. Within Julia, do:

]add VideoIO

Simple Interface

A trivial video player interface exists (no audio): Note: Makie must be imported first to enable playback functionality.

import Makie
import VideoIO

f = VideoIO.testvideo("annie_oakley")  # downloaded if not available
VideoIO.playvideo(f)  # no sound

# Aternatively, you can just open the camera
#VideoIO.viewcam()

High Level Interface

Video Reading

VideoIO contains a simple high-level interface which allows reading of video frames from a supported video file, or from a camera device:

import Makie
import VideoIO

io = VideoIO.open(video_file)
f = VideoIO.openvideo(io)

# As a shortcut for just video, you can upen the file directly
# with openvideo
#f = VideoIO.openvideo(video_file)

# Alternatively, you can open the camera with opencamera().
# The default device is "0" on Windows, "/dev/video0" on Linux,
# and "Integrated Camera" on OSX.  If using something other
# than the default, pass it in as the first parameter (as a string).
#f = VideoIO.opencamera()

# One can seek to an arbitrary position in the video
seek(f,2.5)  ## The second parameter is the time in seconds and must be Float64
img = read(f)
scene = Makie.Scene(resolution = size(img))
makieimg = Makie.image!(scene, buf, show_axis = false, scale_plot = false)[end]
Makie.rotate!(scene, -0.5pi)
display(scene)

while !eof(f)
    read!(f, img)
    makieimg[1] = img
    #sleep(1/30)
end

This code is essentially the code in playvideo, and will read and (without the sleep) play a movie file as fast as possible.

As with the playvideo function, the Images and ImageView packages must be loaded for the appropriate functions to be available.

As an additional handling example, here a grayscale video is read and parsed into a Vector(Array{UInt8}}

f = VideoIO.openvideo(filename,target_format=VideoIO.AV_PIX_FMT_GRAY8)
v = Vector{Array{UInt8}}(undef,0)
while !eof(f)
    push!(v,reinterpret(UInt8, read(f)))
end
close(f)

Video Writing

Given an image stack:

imgstack = map(x->rand(UInt8,2048,1536),1:100)
100-element Array{Array{UInt8,2},1}

Encode entire imgstack in one go:

> using VideoIO
> props = [:priv_data => ("crf"=>"22","preset"=>"medium")]
> encodevideo("video.mp4",imgstack,framerate=30,AVCodecContextProperties=props)

[ Info: Video file saved: /Users/username/Documents/video.mp4
[ Info: frame=  100 fps=0.0 q=-1.0 Lsize=  129867kB time=00:00:03.23 bitrate=329035.1kbits/s speed=8.17x    
[ Info: video:129865kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.001692%

Encode by appending within a custom loop:

using VideoIO, ProgressMeter
filename = "manual.mp4"
props = [:priv_data => ("crf"=>"22","preset"=>"medium")]
codec_name = "libx264"
framerate = 24

encoder = prepareencoder(imgstack[1],codec_name,framerate,props)

io = Base.open("temp.stream","w")
p = Progress(length(imgstack), 1)
index = 1
for img in imgstack
    global index
    appendencode!(encoder, io, img, index)
    next!(p)
    index += 1
end

finishencode!(encoder, io)
close(io)

mux("temp.stream",filename,framerate) #Multiplexes the stream into a video container

The AVCodecContextProperties object allows control of the majority of settings required. Optional fields can be found here

A few helpful presets for h264:

  • Lossless compression - Fastest, largest file size: AVCodecContextProperties = [:priv_data => ("crf"=>"0","preset"=>"ultrafast")]
  • Lossless compression - Slowest, smallest file size: AVCodecContextProperties = [:priv_data => ("crf"=>"0","preset"=>"ultraslow")]
  • Perceptual compression, h264 default (as per docs): AVCodecContextProperties = [:priv_data => ("crf"=>"23","preset"=>"medium")]
  • Direct control of bitrate and frequency of intra frames (every 10): AVCodecContextProperties = [:bit_rate => 400000,:gop_size = 10,:max_b_frames=1]

Encoding of the following image element color types currently supported:

  • UInt8
  • Gray{N0f8}
  • RGB{N0f8}

RGB encoding

If lossless encoding of RGB{N0f8}orjulia-observer-quote-cut-paste-11_work`` is required, _true lossless requires using libx264rgb, to avoid the lossy RGB->YUV420 conversion and GRAY8 color_range compression in libx264. That's achieved with codec_name="libx264rgb" and "crf" => "0" in the above example, but is typically only useful for data storage given that even VLC struggles playing back libx264rgb videos smoothly.

Low Level Interface

Each ffmpeg library has its own VideoIO subpackage:

libavcodec    -> AVCodecs
libavdevice   -> AVDevice
libavfilter   -> AVFilters
libavformat   -> AVFormat
libavutil     -> AVUtil
libswscale    -> SWScale

The following three files are related to ffmpeg, but currently not exposed:

libswresample -> SWResample
libpostproc   -> PostProc   (not wrapped)

After importing VideoIO, you can import and use any of the subpackages directly

import VideoIO
import SWResample  # SWResample functions are now available

Note that much of the functionality of these subpackages is not enabled by default, to avoid long compilation times as they load. To control what is loaded, each library version has a file which imports that's modules files. For example, ffmpeg's libswscale-v2 files are loaded by $(VideoIO_PKG_DIR)/src/ffmpeg/SWScale/v2/LIBSWSCALE.jl.

Check these files to enable any needed functionality that isn't already enabled. Note that you'll probably need to do this for each version of the package for ffmpeg, and that the interfaces do change some from version to version.

Note that, in general, the low-level functions are not very fun to use, so it is good to focus initially on enabling a nice, higher-level function for these interfaces.

Test Videos

A small number of test videos are available through VideoIO.TestVideos. These are short videos in a variety of formats with non-restrictive (public domain or Creative Commons) licenses.

  • VideoIO.TestVideos.available() prints a list of all available test videos.
  • VideoIO.testvideo("annie_oakley") returns an AVInput object for the "annie_oakley" video. The video will be downloaded if it isn't available.
  • VideoIO.TestVideos.download_all() downloads all test videos
  • VideoIO.TestVideos.remove_all() removes all test videos

Status

Prior to version 6.0, this package used a BinDeps approach, using system-level ffmpeg installs, and thus provided support of many versions of ffmpeg and libav. See v0.5.6 for that prior functionality.

v0.6.0 onwards uses a BinaryProvider approach, with a dedicated ffmpeg 4.1 install.

Currently, a simple video interface is available. See TODO.md for some possible directions forward.

Issues, requests, and/or pull requests for problems or additional functionality are very welcome.

First Commit

05/03/2014

Last Touched

about 11 hours ago

Commits

253 commits

Requires:

Used By: