RTMDet-Ins seg: max_det is ignored above 100 #950

In postprocess/rtmdet.py the segment path does keep = keep[: min(max_det, 100)], so predict(..., max_det=1000) still returns at most 100 masks. Same on dev.

On dense scenes (up to ~200 objects per 1024 image) this silently drops objects. With an mmdet-trained RTMDet-Ins-x converted to LibreYOLO, mask mAP50 was 0.726 vs 0.809 in mmdet. Scores match exactly, only the cap differs. Patching it locally to keep[:max_det] gives 0.796.

Keeping 100 as the default for COCO parity makes sense, but an explicit max_det should be respected. nms_pre is also clamped to 1000 there.

libreyolo 1.6.0, torch 2.12, Python 3.14, Linux.