RT-DETR and DeepSORT: Real-Time Detection Transformer and Tracking

Nowadays, object detection plays a crucial role in enabling computers to understand the visual world. It achieves this by identifying and locating objects within images or videos. However, simply identifying objects in a single frame isn't enough. Object tracking allows us to see how objects move and interact with their surroundings. This is essential for tasks like self-driving cars and video surveillance.

In our project, we leverage a vision transformer-based real-time object detector called REDETR to identify objects. Vision transformers have achieved state-of-the-art (SOTA) performance in object detection. To track the detected objects, we employ the DeepSORT algorithm. DeepSORT excels at assigning unique identifiers to each object, allowing it to differentiate between them even in crowded scenes.

RT-DETR

DeepSORT

Resources

This project utilizes the following resources:

• RT-DETR Large rtdetr-l.pt: This model is included in the Ultralytics Python API.
  • A list of available pretrained RTDETR models can be found on the Ultralytics website.
• Feature Extraction Model: You can find the CNN checkpoint file here.
• The original repository of DeepSORT can be found here.

Installation

Prerequisites:

• Python 3.8 or later (Make sure you have python installed. Check by running python --version)

Steps:

1. Clone the Repository:

git clone https://github.com/notayush000/ViTDet-DeepSORT.git

2. Install PyTorch
   - Follow the instructions on the official PyTorch website to install the appropriate version: Link
   - Note: If you don't have an NVIDIA GPU, PyTorch will automatically use your CPU for computations.

3. Install project dependencies:

pip install -r requirements.txt

4. Verify deep_sort folder: Make sure the deep_sort folder is present in the same directory as your main python script (main.py)

5. Provide video path: In main.py, replace 'YOUR_PATH_GOES_HERE' with the actual path to the video

transformer_detector = DETRClass('YOUR_PATH_GOES_HERE')

6. Run the project:

python main.py

A dialog box will appear after the program finishes loading.

Note: Processing with the CPU might take slightly longer compared to using an NVIDIA GPU.

Demo

Real-Time object detection and Tracking