[Bug]: Error running Yolov10 on aarch64

[rhelck]

OpenVINO Version

2021 LTS (I think)

Operating System

Ubuntu 20.04 (LTS)

Device used for inference

CPU

Framework

PyTorch

Model used

YOLOv10

Issue description

When I export YOLOv10 to OpenVINO, I am able to run the model on Intel CPUs. When I run it on an Nvidia AGX Jetson Aarch64 CPU, I get an error during runtime which I described below.

Step-by-step reproduction

• Export YOLOv10 to OpenVINO
• Run on Nvidia Jetson AGX

Relevant log output

File "/home/orbital/MY_VENV/lib/python3.8/site-packages/openvino/runtime/ie_api.py", line 132, in infer
    return OVDict(super().infer(_data_dispatch(
RuntimeError: Exception from src/inference/src/cpp/infer_request.cpp:223:
in reduce_op src/core/NEON/kernels/NEReductionOperationKernel.cpp:1702: Not supported

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.

[rhelck]

BTW I followed this notebook to export the model
https://github.com/openvinotoolkit/openvino_notebooks/blob/0ba3c0211bcd49aa860369feddffdf7273a73c64/notebooks/yolov10-optimization/yolov10-optimization.ipynb

[dmitry-gorokhov]

@rhelck Could you please clarify OV version you have used for experiments? ARM support on OV level is evolving fast and related fixes might be already implemented in newest versions.

[alvoron]

@rhelck how do you infer the model? Do you do f16 inference?

[rhelck]

@dmitry-gorokhov
This code:

from openvino.runtime import get_version
print(get_version())

Prints:

2024.1.0-15008-f4afc983258-releases/2024/1

[rhelck]

@alvoron Not sure actually, I just use the default settings. Let me see if I can figure out the format of the data

[alvoron]

@rhelck
Could you please try to set fp32 inference explicitly?
You need to:

• follow Run OpenVINO Inference on selected device using Ultralytics API to do inference instead of Run OpenVINO Inference on AUTO device using Ultralytics API.
• select CPU in dropbox created in Out [9] notebook section
• replace In [10] notebook section content with the following piece of code before its execution:

ov_model = core.read_model(ov_model_path)
ov_config = {"INFERENCE_PRECISION_HINT": "f32"}
det_compiled_model = core.compile_model(ov_model, device.value, ov_config)

[rhelck]

@alvoron Thanks for the response, let me run that and see if it works. Might be a day or two given my work schedule

[avitial]

Closing this, I hope previous responses were sufficient to help you proceed. Feel free to reopen and ask additional questions related to this topic if issue persists.