Do image processing with deep neural networks. It works together with another filter which converts the pixel format of the Frame to what the dnn network requires.
The filter accepts the following options:
- dnn_backend
-
Specify which DNN backend to use for model loading and execution. This option accepts the following values:
- native
-
Native implementation of DNN loading and execution.
- tensorflow
-
TensorFlow backend. To enable this backend you need to install the TensorFlow for C library (see https://www.tensorflow.org/install/install_c) and configure FFmpeg with
--enable-libtensorflow
Default value is native.
- model
-
Set path to model file specifying network architecture and its parameters. Note that different backends use different file formats. TensorFlow and native backend can load files for only its format.
Native model file (.model) can be generated from TensorFlow model file (.pb) by using tools/python/convert.py
- input
-
Set the input name of the dnn network.
- output
-
Set the output name of the dnn network.
Examples
-
Remove rain in rgb24 frame with can.pb (see derain filter):
./ffmpeg -i rain.jpg -vf format=rgb24,dnn_processing=dnn_backend=tensorflow:model=can.pb:input=x:output=y derain.jpg
-
Halve the pixel value of the frame with format gray32f:
ffmpeg -i input.jpg -vf format=grayf32,dnn_processing=model=halve_gray_float.model:input=dnn_in:output=dnn_out:dnn_backend=native -y out.native.png
-
Handle the Y channel with srcnn.pb (see sr filter) for frame with yuv420p (planar YUV formats supported):
./ffmpeg -i 480p.jpg -vf format=yuv420p,scale=w=iw*2:h=ih*2,dnn_processing=dnn_backend=tensorflow:model=srcnn.pb:input=x:output=y -y srcnn.jpg
-
Handle the Y channel with espcn.pb (see sr filter), which changes frame size, for format yuv420p (planar YUV formats supported):
./ffmpeg -i 480p.jpg -vf format=yuv420p,dnn_processing=dnn_backend=tensorflow:model=espcn.pb:input=x:output=y -y tmp.espcn.jpg