FFmpeg 9.1
Since* 5.0
#

Do object detection with deep neural networks.

The filter accepts the following options:

dnn_backend

Specify which DNN backend to use for model loading and execution. This option accepts the following values:

tensorflow

TensorFlow backend.

openvino

OpenVINO backend.

onnx

ONNX Runtime backend.

model

Set path to model file specifying network architecture and its parameters. Note that different backends use different file formats.

input

Set the input name of the dnn network.

output

Set the output name of the dnn network. The TensorFlow and ONNX Runtime backends accept multiple output names separated by &, e.g. output=yolo_13&yolo_26.

confidence

Set the confidence threshold (default: 0.5).

labels

Set path to label file specifying the mapping between label id and name. Each label name is written in one line, tailing spaces and empty lines are skipped. The first line is the name of label id 0 (usually it is ’background’), and the second line is the name of label id 1, etc. The label id is considered as name if the label file is not provided.

model_type

Set the detection model output format. The following values are accepted:

ssd

Single-stage detector (default).

yolo

YOLO v1/v2.

yolov3

YOLOv3.

yolov4

YOLOv4.

anchors

Anchor box dimensions, separated by &. Required for YOLO-family models. The list is consumed in output order, so the anchors of the first output come first: with output=yolo_13&yolo_26 the leading pairs belong to yolo_13.

nb_classes

Number of detection classes. Required for YOLO-family models.

backend_configs

Set the configs to be passed into backend. To use async execution, set async (default: set). Roll back to sync execution if the backend does not support async.

#

Examples

  • Run SSD object detection with an ONNX model (single output, DirectML on Windows):

    ffmpeg -i input.mp4 -vf "scale=300:300,format=rgb24,dnn_detect=dnn_backend=onnx
           :model=ssd.onnx:model_type=ssd:confidence=0.5:device=dml" -f null -
  • Run YOLOv3 object detection with an ONNX model (two outputs, CPU):

    ffmpeg -i input.mp4 -vf "scale=416:416,format=rgb24,dnn_detect=dnn_backend=onnx
           :model=yolov3.onnx:output=yolo_13&yolo_26:model_type=yolov3:nb_classes=80
           :anchors=116&90&156&198&373&326&30&61&62&45&59&119" -f null -