Add Oriented Bounding Box (OBB) support for rotated object detection

[farukalamai]

What does this PR do?

Adds OBB (Oriented Bounding Box) support to RF-DETR, enabling rotated object detection with [cx, cy, w, h, angle] box format. This includes rotated box math utilities, DOTA v1.0 dataset loader, angle prediction head, Gaussian-based matching/loss functions, and oriented postprocessing.

When oriented=True is set in ModelConfig, the model predicts a 5th dimension (rotation angle in radians) per box, uses KLD loss for regression, GWD cost for Hungarian matching, and outputs rotated corner points.

Related Issue(s): Fixes #56

Type of Change

• New feature (non-breaking change that adds functionality)

Testing

• I have tested this change locally
• I have added/updated tests for this change

Test details:

75 new tests covering all OBB components:

• test_rotated_box_ops.py (34 tests) — angle normalization, box conversions, roundtrips, GWD/KLD/ProbIoU losses, gradient flow, edge cases (zero-size boxes, large boxes)
• test_dota_detection.py (21 tests) — annotation parsing, dataset loading, filtering, normalization, empty/missing files
• test_obb_head.py (8 tests) — detection head output shapes, angle range, gradient flow
• test_obb_matcher_criterion.py (6 tests) — oriented matching, KLD loss computation
• test_obb_postprocess.py (5 tests) — output keys, shapes, scaling, batch support
• test_obb_export.py (2 tests) — ONNX export produces 5D output with valid angles
• test_obb_integration.py (5 tests) — config flag, namespace forwarding, dataset_file acceptance

All existing tests pass unchanged.

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• My changes generate no new warnings or errors
• I have updated the documentation accordingly (if applicable)

Additional Context

Key design decisions:

• Separate angle_embed MLP rather than extending bbox_embed to 5D — keeps spatial box prediction unchanged
• KLD loss for regression — handles angle boundary discontinuity, aspect-ratio adaptive
• GWD cost for Hungarian matching — works even with zero-overlap boxes
• ProbIoU for IoU-aware classification loss — replaces axis-aligned box_iou when oriented
• oriented: bool = False on ModelConfig — zero impact on existing non-oriented models

Files changed

New files:

• src/rfdetr/utilities/rotated_box_ops.py — box conversions, Gaussian encoding, GWD/KLD/ProbIoU
• src/rfdetr/datasets/dota_detection.py — DOTA v1.0 dataset loader with annotation parser
• tests/ — 7 new test files (75 tests)

Modified files:

• src/rfdetr/config.py — added oriented flag and "dota" dataset option
• src/rfdetr/_namespace.py — forward oriented to namespace
• src/rfdetr/models/_types.py — added oriented to BuilderArgs protocol
• src/rfdetr/models/heads/detection.py — added angle_embed MLP when oriented
• src/rfdetr/models/lwdetr.py — angle prediction in forward pass, zero-init, builder wiring
• src/rfdetr/models/matcher.py — GWD pairwise cost for oriented matching
• src/rfdetr/models/criterion.py — KLD loss for oriented boxes, ProbIoU for ia_bce_loss
• src/rfdetr/models/postprocess.py — oriented output with boxes_obb and corners
• src/rfdetr/datasets/__init__.py — registered DOTA dataset builder

[codecov]

Codecov Report

❌ Patch coverage is 95.10490% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 80%. Comparing base (658f199) to head (e81c9f4).

❌ Your project check has failed because the head coverage (80%) is below the target coverage (95%). You can increase the head coverage or adjust the target coverage.

Additional details and impacted files

@@           Coverage Diff           @@
##           develop   #921    +/-   ##
=======================================
+ Coverage       79%    80%    +1%     
=======================================
  Files           97     99     +2     
  Lines         7793   8061   +268     
=======================================
+ Hits          6148   6410   +262     
- Misses        1645   1651     +6     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.