DINO-X API Documentation

DINO-X is a unified vision model designed for open-world object detection and understanding. It serves as a general object-centric vision model that supports multiple input types and outputs various semantic representations.

Input Modalities

• Text prompts: Describe objects via natural language.
• Visual prompts: Use visual cues for grounding.
• Customized prompts: Specialized input formats.
• Prompt-Free: Supports "Prompt-Free Anything Detection and Recognition" using a universal object prompt.

Output Representations

• Bounding boxes for object detection.
• Segmentation masks for instance segmentation.
• Pose keypoints for pose estimation.
• Object captions for region captioning.

Supported Tasks

• Open-Set Object Detection and Segmentation
• Phrase Grounding
• Visual-Prompt Counting
• Pose Estimation
• Region Captioning

Developers can integrate DINO-X capabilities directly into conversational AI workflows (such as Cursor and Claude) using the DINO-X MCP Server. This allows for object detection to be performed through natural language interactions within MCP-compatible tools.

For implementation details, refer to the DINO-X MCP repository.

Install the required packages using the provided requirements file. If you encounter API-related errors, ensure you are using the latest version of the dds-cloudapi-sdk by upgrading it via pip.

pip install -r requirements.txt

# If API errors occur, upgrade the SDK:
pip install dds-cloudapi-sdk --upgrade

The original V1 API for DINO-X has been deprecated. To use the latest DINO-X model features and ensure compatibility with the current API, you must upgrade to the latest version of the SDK.

Use the following command to upgrade:

pip install dds-cloudapi-sdk -U

To use the DINO-X API, you must first apply for an API Token through the official request website.

If you require additional token quotas, you can purchase more API calls through the official platform using WeChat Pay. For collaboration needs or purchase issues, contact deepdataspace_dm@idea.edu.cn.

Open-world detection allows you to detect arbitrary objects using text prompts.

1. Open demo.py and set your API token.
2. Run the script using python demo.py.
3. The annotated results (images with bounding boxes and masks) will be saved to ./outputs/open_world_detection.

The Prompt-Free feature allows DINO-X to automatically recognize, detect, and segment objects in an image without requiring any text input.

1. Open prompt_free_demo.py and set your API token.
2. Run the script using python prompt_free_demo.py.
3. The annotated results will be saved to ./outputs/prompt_free_detection_segmentation.

As of version 0.5.3, the dds-cloudapi-sdk has updated its mask encoding method. It has moved away from a non-standard method to the pycocotools-aligned rle mask format. This allows for easier decoding of masks using pycocotools.

To use this new format, set the mask_format parameter to coco_rle in your API calls.