Yolov5 v6.0 inference with FP16

[anthai0908]

Env

• GPU: Jetson Nano
• OS, e.g. Ubuntu18.04
• Cuda version 10.2
• TensorRT version 8.2.1.8

About this repo

• which model? yolov5, retinaface?
  Yolov5 v6.0

Your problem

For image preprocessing for inference, how can I turn my input image into FP16 format for faster inference

[wang-xinyu]

For v6.0 tag, it is using FP16 inference by default. This FP16 means the model/weights/inference is FP16, but the input tensor is still FP32, which is a regular setup.

tensorrtx/yolov5/yolov5.cpp

Line 11 in b91b15d

#define USE_FP16 // set USE_INT8 or USE_FP16 or USE_FP32