raise NotImplementedError("By default it is assumed that the data loader used for initialize "

[SiChuanJay]

Hello, I ran into a problem with the training-time quantization of yolov8 using nncf, as shown below. I refer to the example link (https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/302-pytorch-quantization-aware-training). My v8_dataloader is also of type DataLoader, but I got an error. Thanks very much!

[vshampor]

Greetings, @SiChuanJay!

Dataloaders can return anything, and this output may be preprocessed in the rest of the training pipeline before actually ending up in model's forward method. NNCF needs a dataloader already at the compressed model creation stage, e.g. before training, and doesn't in general know about the further preprocessing (turning the output of v8_dataloader into actual forward args and kwargs. You have to tell NNCF about that by wrapping your v8_dataloader in a subclass of a nncf.torch.initialization.PTInitializingDataLoader object that properly defines the get_inputs and get_target abstract methods:

from nncf.torch.initialization import PTInitializingDataLoader

class MyInitializingDataLoader(PTInitializingDataLoader):
    def  get_inputs(self, dataloader_output: Any) -> Tuple[Tuple, Dict]:
        # your implementation - `dataloader_output` is what is returned by your dataloader,
        # and you have to turn it into a (args, kwargs) tuple that is required by your model
        # in this function, for instance, if your dataloader returns dictionaries where 
        # the input image is under key `"img"`, and your YOLOv8 model accepts the input
        # images as 0-th `forward` positional arg, you would do:
        return dataloader_output["img"], {}

   def get_target(self, dataloader_output: Any) -> Any:
        # and in this function you should extract the "ground truth" value from your 
        # dataloader, so, for instance, if your dataloader output is a dictionary where
        # ground truth images are under a "gt" key, then here you would write:
        return dataloader_output["gt"]

init_v8_dataloader = MyInitializingDataLoader(v8_dataloader)
# now you pass this wrapped object instead of your original dataloader into the `register_default_init_args`
nncf_config = register_default_init_args(nncf_config, init_v8_dataloader)

[SiChuanJay]

@vshampor Thank you very much for your help. I need to quantify yolov8 by training-time method, but I am not familiar with both nncf and yolov8. With your help yesterday, the problem was solved, but errors were still reported during fine tuning, as shown below. I think it should be related to the v8 training mode. I tried to modify it, but it has not been solved. So can I ask for your help? Or can you provide relevant codes for yolov5 or yolov7 (training-time quantification).

[vshampor]

@SiChuanJay this problem has only been fixed recently in NNCF on the develop branch in #1600, so try installing NNCF from the Github's develop branch (or update your current NNCF state if you already did so) and try again.

[SiChuanJay]

@vshampor Thank you for your help. I changed the version and solved the problem. But I had a new problem. When my batch=8, the error is the first picture. When my batch=4, the error is the second picture. So, I think there may be something wrong with the code I wrote. Could you please check it for me?

Thank you very much!

[vshampor]

@SiChuanJay we can't check the code if we don't have it. Please provide complete info to reproduce this (the model/training code, datasets and command line parameters), or prepare a minimal reproducer if you don't want to share all of your data.

[SiChuanJay]

This is my code, my aim is to quantify yolov8 using a training-time approach. And the yolov8n-seg.pt is from the official yolov8, it should download automatically. and the val dataset is from (http://images.cocodataset.org/zips/val2017.zip https://github.com/ultralytics/yolov5/releases/download/v1.0/coco2017labels-segments.zip) or you can look at this example(https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/230-yolov8-optimization), which I downloaded using the link it provided. I really need your help, Thank you!

import sys
import time
import warnings  # To disable warnings on export to ONNX.
import zipfile
from pathlib import Path
import logging
import torch
import torch.nn as nn
import torch.nn.parallel
import torch.optim
import torch.utils.data
import torch.utils.data.distributed
import torchvision.datasets as datasets
import torchvision.models as models
import torchvision.transforms as transforms

from nncf.common.logging.logger import set_log_level
set_log_level(logging.ERROR)  # Disables all NNCF info and warning messages.
from nncf import NNCFConfig
from nncf.torch import create_compressed_model, register_default_init_args
from openvino.runtime import Core
from torch.jit import TracerWarning
from ultralytics import YOLO
from ultralytics.yolo.cfg import get_cfg
from ultralytics.yolo.utils import DEFAULT_CFG
from ultralytics.yolo.data.utils import check_det_dataset
from ultralytics.yolo.utils import ops
from ultralytics.yolo.data.build import build_dataloader
from nncf.torch.initialization import PTInitializingDataLoader
from ultralytics.nn.tasks import SegmentationModel
from typing import Tuple, Dict, Any
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"using {device} device")

class MyInitializingDataLoader(PTInitializingDataLoader):
    def  get_inputs(self, dataloader_output: Any) -> Tuple[Tuple, Dict]:
        # your implementation - `dataloader_output` is what is returned by your dataloader,
        # and you have to turn it into a (args, kwargs) tuple that is required by your model
        # in this function, for instance, if your dataloader returns dictionaries where 
        # the input image is under key `"img"`, and your YOLOv8 model accepts the input
        # images as 0-th `forward` positional arg, you would do:
        return dataloader_output["img"], {}

    def get_target(self, dataloader_output: Any) -> Any:
        # and in this function you should extract the "ground truth" value from your 
        # dataloader, so, for instance, if your dataloader output is a dictionary where
        # ground truth images are under a "gt" key, then here you would write:
        return dataloader_output["gt"]

v8=YOLO("yolov8n-seg.pt")
model=v8.model # get v8 model 
args=get_cfg(cfg=DEFAULT_CFG)
args.task='segment'
names=model.names
v8_dataloader,v8_datasets=build_dataloader(cfg=args, batch=8, img_path='coco',names=names)

init_v8_dataloader = MyInitializingDataLoader(v8_dataloader)
nncf_config_dict = {
    "input_info": {
        "sample_size": [1, 3, 640, 640]
        },
    "log_dir": 'yolov8_output',  # The log directory for NNCF-specific logging outputs.
    "compression": {
        "algorithm": "quantization"  # Specify the algorithm here.
    },
}
nncf_config = NNCFConfig.from_dict(nncf_config_dict)
nncf_config = register_default_init_args(nncf_config, init_v8_dataloader)
compression_ctrl, mdoel = create_compressed_model(model, nncf_config)
print("sussess")

[SiChuanJay]

@vshampor Can you take a look at it?

[PascAlex]

@SiChuanJay can i ask you if you solved the problem
if yes how ?
i am in the same situation now but with yolov7 and the function that @vshampor doesnt work in my case
it provides me the following error :
in get_inputs
return dataloader_output["img"], {}
TypeError: list indices must be integers or slices, not str