[Bug]: unwanted calling thread's cpu affinity mask change on a NUMA machine

[notaz]

OpenVINO Version

master branch

Operating System

Ubuntu 20.04 (LTS)

Device used for inference

CPU

Framework

None

Model used

Mask R-CNN

Issue description

With OpenVINO compiled with -DTHREADING=SEQ, when running inference on a machine with 2 (and probably more) NUMA nodes, the calling thread's CPU affinity mask gets changed. This doesn't happen on a machine with just 1 NUMA node. Setting ov::hint::enable_cpu_pinning(false) doesn't help. OpenVINO should not change the affinity of threads that don't belong to it.

Step-by-step reproduction

Compile OpenVINO with -DTHREADING=SEQ. Run the following program on a machine with 2 or more NUMA nodes:

#include <string>
#include <cassert>
#include <sched.h>
#include "openvino/openvino.hpp"

int main() {
        cpu_set_t set, set_initial;
        sched_getaffinity(0, sizeof(set_initial), &set_initial);
        std::cout << "count: " << CPU_COUNT(&set_initial) << std::endl;
        for (int i = 0; i < 3; i++) {
                ov::Core core;
                std::string device_name = "CPU";
                //core.set_property(device_name, ov::streams::num(0));
                core.set_property(device_name, ov::hint::enable_cpu_pinning(false));
                auto model = core.read_model("./MaskRCNN-12.onnx");
                model->reshape(ov::Shape{3, 32, 32});
                auto compiled_model = core.compile_model(model, device_name);
                ov::InferRequest request = compiled_model.create_infer_request();
                request.infer();

                sched_getaffinity(0, sizeof(set), &set);
                std::cout << "count: " << CPU_COUNT(&set) << std::endl;
        }
        assert(CPU_EQUAL(&set_initial, &set));
}

This prints the count of set affinity mask bits, which shouldn't change.

Relevant log output

On a machine with 2 NUMA nodes (72 hyperthreads in total):
----
count: 72
count: 18
count: 9
count: 4
test3: test3.cpp:24: int main(): Assertion `CPU_EQUAL(&set_initial, &set)' failed.
----
Expected output is always 72.

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.

[peterchen-intel]

Reproduced the issue, create ticket 143272 to fix.
@notaz We will find solution to release the affinity set by OV when ov::Core is released. Please confirm if we have got your request correctly.
"OpenVINO should not change the affinity of threads that don't belong to it."

[notaz]

release the affinity set by OV when ov::Core is released

This is not a good solution either. Even after request.infer(); returns I need to be able to run my further code on any CPU. OpenVINO should never modify user's thread properties such as affinity at all. That should be limited to the threads OpenVINO creates (if it's needed at all).

[wangleis]

hi @notaz, thank you for reporting this issue. Since OpenVINO support CPU pinning in SEQ scenario, we will fix this issue as below:

• If CPU pinning is disabled, OpenVINO will not change CPU affinity during inference
• If CPU pinning is enabled, OpenVINO will change CPU affinity for pinning during inference . And restore CPU affinity after compiled_model is released.

User can control CPU pinning using property ov::hint::enable_cpu_pinning.

May I know if this solution is suitable for your use case?

[notaz]

Yes that sounds good, thanks.