wandb shows unused labels after COCO transfer-learning #13987
Comments
|
@iokarkan hello, Thank you for providing detailed information about your issue. It appears that the problem lies in the integration between Ultralytics YOLOv8 and Weights & Biases (W&B), specifically with the display of unused COCO class labels in the validation table images. To help us investigate further, could you please provide a minimal reproducible example that includes the specific YAML configuration file you are using? This will allow us to replicate the issue more accurately. You can find guidance on creating a minimal reproducible example here. Additionally, please ensure that you are using the latest versions of both Ultralytics YOLOv8 and Weights & Biases packages. Sometimes, issues like these are resolved in newer releases. Here's a quick checklist to verify:
If the issue persists after updating, please share the YAML configuration and any additional relevant code snippets. This will help us diagnose and address the problem more effectively. Thank you for your cooperation! 😊 |
|
The strategy to reproduce the bug is to use a OpenImagesv7 checkpoint and train on a modified COCO-8 dataset with 2 classes. The venv uses therefore I will investigate also with the reported as supported version (changing the requirements line to 8.0.238 and re-running seems to give the same wandb result). After training for 2 epochs, the following shows up in wandb: From what I understand the extra classes are coming from the OpenImagesv7 dataset, and should not be predicted in my validation at all as I am not using them in my transfer-learning. Below are the files used in the process:
e.g. |
|
Hello @iokarkan, Thank you for providing the detailed information and the reproducible example. It’s very helpful for diagnosing the issue. From your description and the provided code, it seems that the problem lies in the integration between Ultralytics YOLOv8 and Weights & Biases (W&B), where unused COCO class labels are still appearing in the validation table images. Steps to Address the Issue:
Example Code Snippet:Here’s a concise example to ensure everything is set up correctly: import wandb
from ultralytics import YOLO, settings
from wandb.integration.ultralytics.callback import add_wandb_callback
# Initialize W&B
wandb.login()
wandb.init(project="ultralytics-issue", name="test", job_type="training")
# Load the model
model = YOLO('../_weights/yolov8n-oiv7.pt')
# Add W&B callback
add_wandb_callback(model, enable_model_checkpointing=True)
# Train the model
results = model.train(
project="ultralytics-issue",
name="test",
data="./coco8-reduced.yaml",
epochs=2,
imgsz=640,
batch=8,
device=0
)
# Validate the model
val_results = model.val()
print(val_results)
# Finalize W&B run
wandb.finish()Additional Resources:For more detailed guidance on integrating Ultralytics YOLOv8 with Weights & Biases, you can refer to the Ultralytics documentation on W&B integration. If the issue persists after these steps, please let us know, and we can further investigate. Your cooperation and detailed reporting are greatly appreciated! 😊 |
|
Thank you for taking the time @glenn-jocher. A couple of remarks:
Based on your other suggestion, I validated the model with |
|
Hello @iokarkan, Thank you for the detailed follow-up and for providing the additional context. Your observations are very helpful in diagnosing the issue. Key Points:
Next Steps:To address the issue with W&B logging extra labels, consider the following steps:
Conclusion:The issue appears to be with how W&B logs class labels from the pre-trained checkpoint. By ensuring explicit class mapping and possibly using a custom callback, you can mitigate this issue. If the problem persists, reaching out to W&B support would be a prudent next step. Thank you for your patience and detailed reporting. If you have any further questions or need additional assistance, feel free to ask. 😊 |
Search before asking
YOLOv8 Component
Train, Integrations
Bug
I am doing a transfer-learning training based on the pre-trained COCO YOLOv8 checkpoint, where I am removing 76/80 of the COCO classes from the config and renaming the rest for my own labels to predict, according to my training dataset that contains only those 4 indices.
The logs correctly show:
4 Overriding model.yaml nc=80 with nc=4However, checking the produced output during and after training in wandb I am noticing that the Validation-Table table images are showing other COCO class labels, that were excluded during training, with their original COCO names.
I am even renaming the 'person' class label in the YAML config used for training, but I can see bounding boxes with the 'person' class label in the wandb validation table images.
I have checked the trained model checkpoint in many frames similar to the ones in the validation table and the model never seems to predict other COCO labels, as intended, regardless of the confidence threshold set. So the bug seems to be localized in the integration between ultralytics and wandb.
Environment
Ultralytics YOLOv8.1.27 🚀 Python-3.10.12 torch-2.1.0+cu121 CUDA:0 (NVIDIA GeForce RTX 3060, 12036MiB)
Setup complete ✅
OS Linux-5.15.0-112-generic-x86_64-with-glibc2.35
Environment Linux
Python 3.10.12
matplotlib ✅ 3.9.0>=3.3.0
opencv-python ✅ 4.9.0.80>=4.6.0
pillow ✅ 9.4.0>=7.1.2
pyyaml ✅ 6.0.1>=5.3.1
requests ✅ 2.28.2>=2.23.0
scipy ✅ 1.13.1>=1.4.1
torch ✅ 2.1.0>=1.8.0
torchvision ✅ 0.16.0>=0.9.0
tqdm ✅ 4.66.4>=4.64.0
psutil ✅ 5.9.8
py-cpuinfo ✅ 9.0.0
thop ✅ 0.1.1-2209072238>=0.1.1
pandas ✅ 2.2.2>=1.1.4
seaborn ✅ 0.13.2>=0.11.0
Minimal Reproducible Example
The training script is the following:
Additional
No response
Are you willing to submit a PR?
The text was updated successfully, but these errors were encountered: