Do image processing with deep neural networks. It works together with another filter which converts the pixel format of the Frame to what the dnn network requires.
The filter accepts the following options:
- dnn_backend
-
Specify which DNN backend to use for model loading and execution. This option accepts the following values:
- tensorflow
-
TensorFlow backend. To enable this backend you need to install the TensorFlow for C library (see https://www.tensorflow.org/install/lang_c) and configure FFmpeg with
--enable-libtensorflow
- openvino
-
OpenVINO backend. To enable this backend you need to build and install the OpenVINO for C library (see https://github.com/openvinotoolkit/openvino/blob/master/build-instruction.md) and configure FFmpeg with
--enable-libopenvino
(–extra-cflags=-I... –extra-ldflags=-L... might be needed if the header files and libraries are not installed into system path)
- model
-
Set path to model file specifying network architecture and its parameters. Note that different backends use different file formats. TensorFlow, OpenVINO backend can load files for only its format.
- input
-
Set the input name of the dnn network.
- output
-
Set the output name of the dnn network.
- backend_configs
-
Set the configs to be passed into backend. To use async execution, set async (default: set). Roll back to sync execution if the backend does not support async.
For tensorflow backend, you can set its configs with sess_config options, please use tools/python/tf_sess_config.py to get the configs of TensorFlow backend for your system.
Examples
-
Remove rain in rgb24 frame with can.pb (see derain filter):
./ffmpeg -i rain.jpg -vf format=rgb24,dnn_processing=dnn_backend=tensorflow:model=can.pb:input=x:output=y derain.jpg
-
Handle the Y channel with srcnn.pb (see sr filter) for frame with yuv420p (planar YUV formats supported):
./ffmpeg -i 480p.jpg -vf format=yuv420p,scale=w=iw*2:h=ih*2,dnn_processing=dnn_backend=tensorflow:model=srcnn.pb:input=x:output=y -y srcnn.jpg
-
Handle the Y channel with espcn.pb (see sr filter), which changes frame size, for format yuv420p (planar YUV formats supported), please use tools/python/tf_sess_config.py to get the configs of TensorFlow backend for your system.
./ffmpeg -i 480p.jpg -vf format=yuv420p,dnn_processing=dnn_backend=tensorflow:model=espcn.pb:input=x:output=y:backend_configs=sess_config=0x10022805320e09cdccccccccccec3f20012a01303801 -y tmp.espcn.jpg