Adding RTDETRv2

[jadechoghari]

What does this PR do?

This PR add RTDETRv2 into the Transformers library. There is a new thing in transformers called modular, which adds new models by creating a modeling_modelname.py file. Since RTDETRv2 only updates the decoder part while keeping the rest of the model unchanged, it serves as an ideal use case for this modular approach.

What’s Left:

• Fix the modular -> modeling cookie cutter setup
• Remove the scratch folder (auto-generated by the add-model cookie cutter)
• Add support for resnet

[SangbumChoi]

@jadechoghari Hi thanks for working this, original PR for RTDetr-v2 can be found here. Perhaps you can fill the blank based on this
#33244

[jadechoghari]

Hi @SangbumChoi, yes - this is a new PR since we're using a new concept called modular.
also, thank you so much for your work - lot of stuff here is based on it especially the v1 implementation!

[SangbumChoi]

Great. FYI I also have confirmed that there is slight tolerance issue of converting v2 weight since it has difference in MSDA.

[jadechoghari]

yes nw, I checked your PR!

[Cyrilvallez]

Hey! Here is a first round or reviews! 🤗 I hope it clears some confusions about modular!
Also, let's be consistent with the library: let's rename the files and folders with rt_detr_v2 instead of rtdetrv2, and use CamelCase for class names, i.e. RtDetrV2Class instead of RTDetrv2Class.

[jadechoghari]

great! - i'll update accordingly - thx!

[jadechoghari]

Hey @Cyrilvallez, lmk if anything else needs to be updated for Modular - i've incorporated all your suggestions and fixes!

[Cyrilvallez]

Ok! As discussed in Slack and this review, let's first take care of all the naming conventions in the folder/file/class names 🤗
Also, add/modify all the needed remaining files!
Then, please run

make fixup
python modular_model_converter.py --files_to_parse src/transformers/models/rt_detr_v2/modular_rt_detr_v2.py

from the root of the transformers repo to take care of the coding style and convert your modular!
Once this is done, we'll discuss potential improvements for the modular itself!

[jadechoghari]

Changes updated @Cyrilvallez !
Adding the make fixup error:

utils/check_modular_conversion.py
Traceback (most recent call last):
  File "/home/user/app/transformers/utils/check_modular_conversion.py", line 75, in <module>
    non_matching_files += compare_files(modular_file_path, args.fix_and_overwrite)
  File "/home/user/app/transformers/utils/check_modular_conversion.py", line 55, in compare_files
    generated_modeling_content = convert_modular_file(modular_file_path)
  File "/home/user/app/transformers/utils/modular_model_converter.py", line 1447, in convert_modular_file
    for file, module in create_modules(cst_transformers).items():
  File "/home/user/app/transformers/utils/modular_model_converter.py", line 1387, in create_modules
    nodes_to_add, file_type, new_imports = get_class_node_and_dependencies(modular_mapper, class_name, node, files)
  File "/home/user/app/transformers/utils/modular_model_converter.py", line 1348, in get_class_node_and_dependencies
    relative_dependency_order = mapper.compute_relative_order(all_dependencies_to_add)
  File "/home/user/app/transformers/utils/modular_model_converter.py", line 739, in compute_relative_order
    remaining_dependencies.remove(class_name)
KeyError: 'RtDetrV2MultiheadAttention'
make: *** [Makefile:39: repo-consistency] Error 1

[Cyrilvallez]

Hey! Indeed because the order of classes in modeling_rt_detr.py is messy (some class are used before being defined), the modular was failing! You can pull from #35056 for the fix

See comments, we're getting there! 🤗

EDIT: also, please run make fixup to clean-up the files a bit 🙏

[jadechoghari]

still in progress ignore the red CI