where is object detection

[Hotlat6077]

where is object detection,please

[sovit-123]

Hello @Hotlat6077

I am working on the object detection code at the moment. I have pushed an initial commit to the detection branch. You can still run it by cloning the repository and checking out to the branch.

There are a few changes that you need to make according to your dataset.

In the src/detection/config.py, you will need to provide the path to the images and XML annotation folders for training and valdation set, and add the class names. Then you can run the train_detection.py script using the command similar to the following:

python train_detection.py --weight dinov3_convnext_tiny_pretrain_lvd1689m-21b726bb.pth --model-name dinov3_convnext_tiny --imgsz 640 640 --lr 0.0001 --fine-tune --out-dir img_det_aug --epochs 30 --workers 8 --scheduler-epochs 30 --batch 8

[RuijieSpace]

Hello   你好@Hotlat6077

I am working on the object detection code at the moment. I have pushed an initial commit to the detection branch. You can still run it by cloning the repository and checking out to the branch.我目前正在编写对象检测代码。我已经将初始提交推送到 detection 分支。您仍然可以通过克隆代码库并签出到该分支来运行它。

There are a few changes that you need to make according to your dataset.您需要根据您的数据集进行一些更改。

In the src/detection/config.py, you will need to provide the path to the images and XML annotation folders for training and valdation set, and add the class names. Then you can run the train_detection.py script using the command similar to the following:在 src/detection/config.py 中，您需要提供用于训练和验证集的图像和 XML 注释文件夹的路径，并添加类名。然后，您可以使用类似以下命令运行 train_detection.py 脚本：

python train_detection.py --weight dinov3_convnext_tiny_pretrain_lvd1689m-21b726bb.pth --model-name dinov3_convnext_tiny --imgsz 640 640 --lr 0.0001 --fine-tune --out-dir img_det_aug --epochs 30 --workers 8 --scheduler-epochs 30 --batch 8

Hello, I cloned this branch and then used the dataset voc2007 with the following command: CUDA_VISIBLE_DEVICES = 1 python train_detection.py --weight weights/dinov3_convnext_large_pretrain_lvd1689m -61fa432d.pth --model-name dinov3_convnext_large --imgsz 640 640 --lr 0.0001 --fine-tune --out-dir img_det_aug --epochs 30 --workers 8 --scheduler-epochs 30 --batch 8, but the following error is reported:： File "/home/root1//02_project/dinov3_stack/train_detection.py", line 193, in train
loss_dict = model(images, targets)
File "/root/miniconda3/envs/yolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/yolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/root/miniconda3/envs/yolo/lib/python3.10/site-packages/torchvision/models/detection/ssd.py", line 378, in forward
head_outputs = self.head(features)
File "/root/miniconda3/envs/yolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/yolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/root/miniconda3/envs/yolo/lib/python3.10/site-packages/torchvision/models/detection/ssd.py", line 66, in forward
"bbox_regression": self.regression_head(x),
File "/root/miniconda3/envs/yolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/yolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/root/miniconda3/envs/yolo/lib/python3.10/site-packages/torchvision/models/detection/ssd.py", line 95, in forward
results = self._get_result_from_module_list(features, i)
File "/root/miniconda3/envs/yolo/lib/python3.10/site-packages/torchvision/models/detection/ssd.py", line 88, in _get_result_from_module_list
out = module(x)
File "/root/miniconda3/envs/yolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/root/miniconda3/envs/yolo/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/root/miniconda3/envs/yolo/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 458, in forward
return self._conv_forward(input, self.weight, self.bias)
File "/root/miniconda3/envs/yolo/lib/python3.10/site-packages/torch/nn/modules/conv.py", line 454, in _conv_forward
return F.conv2d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [16, 768, 3, 3], expected input[8, 1536, 20, 20] to have 768 channels, but got 1536 channels instead

[sovit-123]

@RuijieSpace
I have corrected the issue and pushed the code. Please pull it from the detection branch. Earlier I was testing with the ConvNeXt tiny model, so I had hardcoded final layer dimension. I have made it generalized now. It should work with any ConvNeXt model. But probably won't work with the ViT backbones. Will work on that next.

[sovit-123]

@RuijieSpace
I have made another small change to the detection code. Instead of config.py file for detection, I have moved to YAML file configuration for more streamlined experience. These are present in detection_configs folder and for Pascal VOC it is present by default. Nothing much changes. The only addition is a --config command line argument to the training and detection script pointing to the path of the YAML file.

Example:

python train_detection.py --config detection_configs/voc.yaml <rest of the command>

[RuijieSpace]

@sovit-123 This is really good. At present, I have also successfully started the training process on the VOC2007 dataset, using the profile information you provided, but modifying the path information of the dataset. I would like to ask if you have also conducted object detection training on a certain dataset. Based on the code you provided, the effect of my training on VOC2007 is not ideal. At present, the best result of map50 is less than 50%, and it is not even as good as the result of resnet50 training as the backbone. Do you have any insights into this? Looking forward to your reply, and thank you very much for your contribution.

[sovit-123]

Yes @RuijieSpace
I am trying to optimize the model creation process. There are not many sources out there for DINOv3 downstream tasks and I am trying to figure out a lot of things by myself.

One thing is that right now all the detection models with ConvNeXt backbone are using single feature from the last layer to create the SSD detection head. I have tried multi-feature map but the results are not good.

Probably, ViT backbone with multi-feature map will give better results.

Otherwise, it might simply be that the SSD head with DINOv3 is not powerful/good. I will try with RetinaNet head as well.

I have full implementation with Faster RCNN heads as well here in my Faster RCNN training pipeline. In that project also, only the ViT backbones are using the multi-features. I think with some of the larger ViT or even the largest ConvNeXt Faster RCNN, the results will be better than this. The data annotation format is the same, so you won't need to change much.

Still, I am trying to figure more things out regarding the learning, rate, optimizer, batch size.

It will take some time, but I will keep updating this.

[zzw1123]

Hi, I am wondering why you did not choose Plain-DETR as detection head as dinov3 paper do?

[sovit-123]

@zzw1123 I tried to. In fact, I have a running codebase based on the Plain DETR repository. But I not getting any sensible mAP numbers. As the Plain DETR codebase has also not been updated in a bit, it is difficult to debug. Furthermore, if the DINOv3 authors can open source some parts of the code it would help, but they have not. Still trying.

[zzw1123]

@zzw1123 I tried to. In fact, I have a running codebase based on the Plain DETR repository. But I not getting any sensible mAP numbers. As the Plain DETR codebase has also not been updated in a bit, it is difficult to debug. Furthermore, if the DINOv3 authors can open source some parts of the code it would help, but they have not. Still trying.

Yes, I'm working on that too. Did you train the model on COCO or your own dataset? I used my own dataset, but the mAP is pretty low.
Since I'm new to object detection, I suspect I might be missing some training tricks—haha.
By the way, I found a project that combines DINOv3 with YOLOv12: https://github.com/Sompote/DINOV3-YOLOV12. I'm planning to switch the detection head to YOLO.

[sovit-123]

That sounds interesting. Will check.