Do image processing with deep neural networks. It works together with another filter which converts the pixel format of the Frame to what the dnn network requires.
The filter accepts the following options:
- dnn_backend
-
Specify which DNN backend to use for model loading and execution. This option accepts the following values:
- native
-
Native implementation of DNN loading and execution.
- tensorflow
-
TensorFlow backend. To enable this backend you need to install the TensorFlow for C library (see https://www.tensorflow.org/install/install_c) and configure FFmpeg with
--enable-libtensorflow
- openvino
-
OpenVINO backend. To enable this backend you need to build and install the OpenVINO for C library (see https://github.com/openvinotoolkit/openvino/blob/master/build-instruction.md) and configure FFmpeg with
--enable-libopenvino
(–extra-cflags=-I... –extra-ldflags=-L... might be needed if the header files and libraries are not installed into system path)
Default value is native.
- model
-
Set path to model file specifying network architecture and its parameters. Note that different backends use different file formats. TensorFlow, OpenVINO and native backend can load files for only its format.
Native model file (.model) can be generated from TensorFlow model file (.pb) by using tools/python/convert.py
- input
-
Set the input name of the dnn network.
- output
-
Set the output name of the dnn network.
- async
-
use DNN async execution if set (default: set), roll back to sync execution if the backend does not support async.
Examples
-
Remove rain in rgb24 frame with can.pb (see derain filter):
./ffmpeg -i rain.jpg -vf format=rgb24,dnn_processing=dnn_backend=tensorflow:model=can.pb:input=x:output=y derain.jpg
-
Halve the pixel value of the frame with format gray32f:
ffmpeg -i input.jpg -vf format=grayf32,dnn_processing=model=halve_gray_float.model:input=dnn_in:output=dnn_out:dnn_backend=native -y out.native.png
-
Handle the Y channel with srcnn.pb (see sr filter) for frame with yuv420p (planar YUV formats supported):
./ffmpeg -i 480p.jpg -vf format=yuv420p,scale=w=iw*2:h=ih*2,dnn_processing=dnn_backend=tensorflow:model=srcnn.pb:input=x:output=y -y srcnn.jpg
-
Handle the Y channel with espcn.pb (see sr filter), which changes frame size, for format yuv420p (planar YUV formats supported):
./ffmpeg -i 480p.jpg -vf format=yuv420p,dnn_processing=dnn_backend=tensorflow:model=espcn.pb:input=x:output=y -y tmp.espcn.jpg