DOCSEARCHbyAlgolia
Submit Docs

QTrobot Documentation

website·Indexed 34 days ago

https://docs.luxai.com/

Developer guides and tutorials for QTrobot V2 and V3. Includes instructions for programming robot behaviors using Python, TypeScript, and JavaScript, as well as graphical programming via QTrobot Studio. Covers ROS and ROS2 integration, motor control, speech services, gesture and facial expression detection, and integration with Nvidia Riva ASR and ReSpeaker microphones.

Tokens
85.6K
Snippets
510
Records
656
Agent score
55%

What's inside QTrobot

  1. Overview of QTrobot V3 Motion Software Interface

    v3

    Motion in QTrobot V3 is managed by the motor node in qtrobot-service-hub (ZMQ port 50540). The software interface is divided into three layers:

    • robot.motor: Low-level joint control (torque, homing, velocity, state/error streams).
    • robot.gesture: Keyframe playback and recording built on top of the motor node.
    • robot.kinematics: A Python-only plugin for inverse kinematics (IK) used for head gaze and arm reach/aim in cartesian space.

    Hardware configuration for Feetech or Herkulex motor controllers is located at /opt/luxai/qtrobot_service_hub/etc/motor/.

  2. Overview of QTrobot Studio Visual Scripting

    v3
    QTrobot Studio is a browser-based, block programming environment designed for non-technical users to build and run programs on QTrobot using a graphical interface. The environment runs locally on QTrobot's QTPC, meaning it does not require a cloud account or an active internet connection to function.

  3. Overview of the QTrobot AI Data Assistant

    The AI Data Assistant transforms the QTrobot (specifically the QTrobot AI@Edge version) into an on-device conversational assistant. It uses a combination of Offline LLMs and Retrieval-Augmented Generation (RAG) to allow users to upload documents (PDF, Word, etc.) and interact with that data via natural voice. All processing occurs locally on the device to ensure data privacy and security.

  4. Overview of QTrobot V3 Microphone Hardware

    v3

    QTrobot V3 features an integrated ReSpeaker Mic Array v2.0 in the head, connected via USB. It consists of 4 high-performance digital microphones (ST MP34DT01TR-M) supporting far-field voice capture, on-chip speech algorithms, and 12 programmable RGB LED indicators.

    Technical Specifications:

    • Sensitivity: -26 dBFS (omnidirectional)
    • Acoustic overload point: 120 dB SPL
    • SNR: 61 dB
    • Max sample rate: 16 kHz

  5. Overview of QTrobot V2 Camera Hardware and Software

    QTrobot V2 features an integrated Intel® RealSense™ Depth Camera D455 connected to the NUC mini PC (QTPC) via USB-C. The camera is treated as a standard Linux video capture device. Pre-installed software on the QTPC includes:

    • qt_nuitrack_app: For 3D body, hand, gesture, and facial tracking using the Nuitrack SDK.
    • find_object_2d: For 2D object recognition.
    • usb_cam: A standard ROS camera driver for V4L USB cameras.
    # List all video capture devices on QTPC
v4l2-ctl --list-devices

  6. Overview of QTrobot V2 motor software interface

    The qt_motor node is the primary software interface for QTrobot actuators, following the ros_control paradigm. It provides standard ROS interfaces for joint commands, states, and trajectories, as well as custom controllers like QTGestureController and QTMotorsController via plugins. The node runs on the QTRP during startup.

  7. Overview of the QTrobot Online AI Data Assistant

    The QTrobot Online AI Data Assistant is a cloud-enhanced conversational system built on the LlamaIndex framework. It is specifically designed for QTrobot models without dedicated GPU hardware (such as QTrobot-i5 and QTrobot-i7). The assistant uses Retrieval-Augmented Generation (RAG) to ingest and index custom documents (.txt, .pdf, .docx, .md) and web pages, allowing the robot to provide context-aware responses via semantic search.

  8. Overview of Human Detection in luxai-robot

    v3

    The robot.perception plugin provides 3-D-aware human detection, including bounding boxes, pose keypoints, face orientation, 3-D positions per keypoint, and engagement scores.

    Technically, raw detection is handled by the qtrobot-yolo-driver (which consumes RGB/depth streams from qtrobot-realsense-driver). The human-detector plugin runs locally in-process with the SDK client, subscribing to the /persons stream and enriching it with face yaw/pitch, 3-D positions (accounting for the robot's current head pose via robot.motor), and optional voice-activity flags before publishing to the /human/presence stream.

  9. Overview of QTrobot Studio Visual Scripting

    v3
    QTrobot Studio is a browser-based, block programming environment designed for non-technical users to build and run programs on QTrobot using a graphical block-snapping interface. The environment runs locally on QTrobot's QTPC, meaning it does not require a cloud account or an active internet connection to operate.

  10. Overview of QTrobot Studio

    v3
    QTrobot Studio is a browser-based, block programming environment for QTrobot V3 that allows users to build programs by snapping graphical blocks together without writing code. It runs locally on the robot's QTPC, requiring no cloud account or internet connection. It can be used to program speech (multiple languages, configurable voice/speed), facial expressions, gestures (built-in or recorded), interactive visuals via the Learner App, custom audio playback, and branching Q&A sessions.

  11. Overview of QTrobot Studio capabilities

    QTrobot Studio is a visual programming interface based on Blockly, allowing users to develop applications without a programming background. It supports standard programming constructs (loops, conditions, lists, variables, functions) and QTrobot-specific blocks for speech, gestures, and emotions. Key features include:

    • Control-flow blocks for interactive applications involving a 'Learner tablet' and an 'Educator tablet'.
    • Ability to upload custom images, animations, and audio files for use in applications.

  12. Overview of QTrobot AI Agent capabilities

    v3

    The QTrobot AI Agent is a conversational assistant for QTrobot V3 that provides natural, multimodal, and interruption-aware interactions. It is designed to be private and offline-first, keeping audio, images, and memory on the device. Key capabilities include:

    • Multimodal Intelligence: Uses a local LLM (default Qwen3.5 9B via llama.cpp) on Jetson AGX Orin to process text, context, and real-time camera images.
    • Speech-to-Speech (S2S): Features semantic end-of-turn detection and 'Smart Turn' to handle natural pauses and barge-ins.
    • Multilingual Support: Automatic language detection via NVIDIA Parakeet-TDT for English, French, German, Italian, Portuguese, Russian, and Spanish.
    • Voice Cloning: Uses Qwen3-TTS for natural prosody; supports custom voices via short WAV recordings.
    • Tool Integration: Uses Model Context Protocol (MCP) for parallel tool calling and robot physical actions (gestures, facial expressions).
    • Knowledge Management: Implements agentic RAG for local documents (PDF, Markdown, text) and an optional background web-search agent.
    • Memory: Supports persistent long-term conversation memory stored in JSON Lines format.