RF-DETR Documentation

To add custom PyTorch Lightning callbacks to your training, you can pass them via trainer_kwargs to build_trainer.

Warning: Passing callbacks= to build_trainer replaces the entire default callback list (including EMA, COCO evaluation, and best-model checkpointing).

To extend the default callback list instead of replacing it, call build_trainer first, then use .extend() on the trainer.callbacks list before calling .fit().

from pytorch_lightning.callbacks import LearningRateMonitor, ModelSummary
from rfdetr.training import build_trainer

# Option 1: Replace all default callbacks
extra_callbacks = [
    LearningRateMonitor(logging_interval="step"),
    ModelSummary(max_depth=3),
]

trainer = build_trainer(
    train_config,
    model_config,
    callbacks=extra_callbacks,  # replaces the default callback list entirely
)

# Option 2: Extend the default callback list (Recommended)
trainer = build_trainer(train_config, model_config)
trainer.callbacks.extend(
    [
        LearningRateMonitor(logging_interval="step"),
    ]
)
trainer.fit(module, datamodule)

You can deploy fine-tuned RF-DETR detection or segmentation models to Roboflow using the deploy_to_roboflow() method. This enables cloud inference, edge hardware deployment, and multi-step vision workflows.

For Object Detection, use the RFDETRNano class. For Image Segmentation, use the RFDETRSegMedium class.

You must provide your Roboflow workspace, project_id, version, and api_key.

# For Object Detection
from rfdetr import RFDETRNano

x = RFDETRNano(pretrain_weights="<path/to/pretrain/weights/dir>")
x.deploy_to_roboflow(
    workspace="<your-workspace>",
    project_id="<your-project-id>",
    version=1,
    api_key="<YOUR_API_KEY>",
)

# For Image Segmentation
from rfdetr import RFDETRSegMedium

x = RFDETRSegMedium(pretrain_weights="<path/to/pretrain/weights/dir>")
x.deploy_to_roboflow(
    workspace="<your-workspace>",
    project_id="<your-project-id>",
    version=1,
    api_key="<YOUR_API_KEY>",
)

To process continuous video, use OpenCV's cv2.VideoCapture to loop through frames. Convert each frame from BGR to RGB before passing it to model.predict(). Use sv.VertexAnnotator to draw the results on the original BGR frame for display.

import cv2
import supervision as sv
from rfdetr import RFDETRKeypointPreview

model = RFDETRKeypointPreview()

# Replace with <SOURCE_VIDEO_PATH>, <WEBCAM_INDEX>, or <RTSP_STREAM_URL>
video_capture = cv2.VideoCapture("<SOURCE_VIDEO_PATH>")
if not video_capture.isOpened():
    raise RuntimeError("Failed to open video source")

while True:
    success, frame_bgr = video_capture.read()
    if not success:
        break

    frame_rgb = cv2.cvtColor(frame_bgr, cv2.COLOR_BGR2RGB)
    key_points = model.predict(frame_rgb, threshold=0.5)

    annotated_frame = sv.VertexAnnotator().annotate(frame_bgr, key_points)

    cv2.imshow("RF-DETR Keypoint Video", annotated_frame)
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

video_capture.release()
cv2.destroyAllWindows()