Do object detection with deep neural networks.
The filter accepts the following options:
- dnn_backend
-
Specify which DNN backend to use for model loading and execution. This option accepts the following values:
- tensorflow
-
TensorFlow backend.
- openvino
-
OpenVINO backend.
- onnx
-
ONNX Runtime backend.
- model
-
Set path to model file specifying network architecture and its parameters. Note that different backends use different file formats.
- input
-
Set the input name of the dnn network.
- output
-
Set the output name of the dnn network. The TensorFlow and ONNX Runtime backends accept multiple output names separated by &, e.g. output=yolo_13&yolo_26.
- confidence
-
Set the confidence threshold (default: 0.5).
- labels
-
Set path to label file specifying the mapping between label id and name. Each label name is written in one line, tailing spaces and empty lines are skipped. The first line is the name of label id 0 (usually it is ’background’), and the second line is the name of label id 1, etc. The label id is considered as name if the label file is not provided.
- model_type
-
Set the detection model output format. The following values are accepted:
- ssd
-
Single-stage detector (default).
- yolo
-
YOLO v1/v2.
- yolov3
-
YOLOv3.
- yolov4
-
YOLOv4.
- anchors
-
Anchor box dimensions, separated by &. Required for YOLO-family models. The list is consumed in output order, so the anchors of the first output come first: with output=yolo_13&yolo_26 the leading pairs belong to
yolo_13. - nb_classes
-
Number of detection classes. Required for YOLO-family models.
- backend_configs
-
Set the configs to be passed into backend. To use async execution, set async (default: set). Roll back to sync execution if the backend does not support async.
Examples
-
Run SSD object detection with an ONNX model (single output, DirectML on Windows):
ffmpeg -i input.mp4 -vf "scale=300:300,format=rgb24,dnn_detect=dnn_backend=onnx :model=ssd.onnx:model_type=ssd:confidence=0.5:device=dml" -f null -
-
Run YOLOv3 object detection with an ONNX model (two outputs, CPU):
ffmpeg -i input.mp4 -vf "scale=416:416,format=rgb24,dnn_detect=dnn_backend=onnx :model=yolov3.onnx:output=yolo_13&yolo_26:model_type=yolov3:nb_classes=80 :anchors=116&90&156&198&373&326&30&61&62&45&59&119" -f null -