DOCSEARCHbyAlgolia
Submit Docs

rosclaw

repository·main·Indexed 19 days ago

https://github.com/plaipin/rosclaw

A ROS2 and OpenClaw integration that enables natural language control of robots through messaging apps such as WhatsApp, Telegram, Discord, and Slack. It provides an AI agent plugin layer to translate user intent into ROS2 operations via rosbridge, supporting tools for publishing topics, calling services, and managing action goals. The project includes a TypeScript rosbridge client, a discovery node, and the @rosclaw/openclaw-canvas extension for real-time robot dashboards using the A2UI standard.

Tokens
19.4K
Snippets
56
Records
95
Agent score
68%

What's inside rosclaw

  2. Robotic Arm Control Demo Overview

    main

    The Robotic Arm Control Demo allows users to control a robotic arm using natural language commands sent through messaging apps.

    Note on Current Status: The project is currently in Phase 2. The MoveIt2 action integration required for this demo is not yet implemented.

  3. Understand the @rosclaw/openclaw-canvas extension

    main

    The @rosclaw/openclaw-canvas extension provides a real-time robot dashboard for operators using the OpenClaw native apps (macOS, iOS, or Android). While field users on messaging apps like WhatsApp or Telegram receive text and snapshots, operators on the native app see a dedicated Canvas panel containing live telemetry, camera streams, and interactive controls.

    Note: As of the current version, this extension is in Phase 3 and is not yet implemented. It currently only logs a loading message.

  5. Best practices for reporting robot status

    main

    When using the Check Status skill to report information to a user, follow these guidelines:

    • Human-readable summaries: Always summarize the technical status in a format easy for humans to understand.
    • Highlight anomalies: Explicitly call out any warnings or abnormal values found during diagnostics.
    • Graceful sensor handling: If a sensor topic is unavailable, report it as "not reporting" instead of returning a failure error.

  6. Understand the RosTransport abstraction

    main

    The RosTransport interface (provided by @rosclaw/transport) provides a unified API that abstracts away the underlying communication protocol. Plugin tools (like ros2_publish or ros2_subscribe_once) call getTransport() to obtain a transport instance, allowing the same tool logic to work across all deployment modes.

    Available Adapters:

    • RosbridgeTransport: Used for Mode B (via @rosclaw/rosbridge-client). Uses WebSocket to connect to rosbridge_server.
    • LocalTransport: Used for Mode A (via @rosclaw/transport-local). Uses rclnodejs to talk to ROS2 DDS directly.
    • WebRTCTransport: Used for Mode C (via @rosclaw/transport-webrtc). Uses WebRTC data channels to connect to the rosclaw_agent.

    Implementation Detail: The createTransport(config) factory uses dynamic import() to load only the necessary adapter for the selected mode, ensuring that unused dependencies are not loaded.

    // Conceptual flow of transport resolution
// Plugin Tools (ros2_publish, ros2_subscribe_once, ...)
//        │
//        ▼
// getTransport(): RosTransport
//        │
//        ├── RosbridgeTransport  (Mode B — @rosclaw/rosbridge-client)
//        ├── LocalTransport      (Mode A — @rosclaw/transport-local)
//        └── WebRTCTransport     (Mode C — @rosclaw/transport-webrtc)

  7. Deployment Mode C: Cloud / Remote

    main

    OpenClaw runs on a cloud server or VPS, while the robot is located on a remote network (e.g., factory or field) behind a NAT or firewall. To enable communication without public IPs or open inbound ports, the system uses WebRTC for peer-to-peer (P2P) connectivity.

    Requirements:

    • The robot must run a RosClaw Agent Node (rosclaw_agent), which is a lightweight ROS2 node that connects outbound to a signaling/TURN server to establish a WebRTC data channel.
    • STUN/TURN servers are used to handle NAT traversal.

    Key Characteristics:

    • Transport: WebRTC data channel (encrypted)
    • Latency: ~10-100ms (varies by internet connection)
    • NAT Issues: Solved via outbound connections to STUN/TURN.
    • Best for: Production deployments, remote operations, and fleet management.

  8. Deployment Mode A: Same Machine

    main

    In this mode, OpenClaw runs directly on the robot's computer. The plugin communicates with ROS2 natively via the local DDS bus using rclnodejs. This setup requires no network transport between the plugin and the robot, making it ideal for single-robot setups or edge devices with internet access for messaging.

    Key Characteristics:

    • Transport: Local IPC / direct DDS
    • Latency: ~ms
    • NAT Issues: None (only outbound internet is required for messaging APIs)

  9. How RosClaw works: Natural language to ROS2 operations

    main

    RosClaw acts as an intelligent plugin layer between messaging apps (WhatsApp, Telegram, Discord, Slack) and ROS2 robots.

    The Workflow:

    1. A user sends a natural language message via a messaging app.
    2. The OpenClaw Gateway (AI Agent + Tools + Memory) receives the message.
    3. The RosClaw Plugin provides tools that allow the agent to translate intent into ROS2 operations such as publishing topics, calling services, or sending action goals.
    4. The command travels through rosbridge_server (via WebSockets) to the ROS2 DDS layer, which controls the robot's hardware or simulation (Nav2, MoveIt2, sensors, etc.).
    5. The agent streams feedback from the robot back to the user in the chat.

  10. Understand the RosClaw data flow

    main

    RosClaw acts as an intermediary between a user interface (e.g., Telegram) and a robot running ROS2. The typical lifecycle of a command is:

    1. User Input: A natural language command is sent to RosClaw (e.g., "Move forward 2 meters").
    2. AI Agent Processing: The AI Agent interprets the command and selects the appropriate ROS2 tool (e.g., ros2_publish).
    3. Safety Validation: A safety hook validates the command against predefined limits (e.g., ensuring velocity is within safe bounds).
    4. ROS2 Execution: RosClaw publishes the command to the robot via the selected deployment mode (Mode A, B, or C).
    5. Feedback Loop: The robot executes the command and provides feedback (e.g., via /odom subscription), which RosClaw translates back into a natural language response for the user.
    User (Telegram)                RosClaw                        Robot
       │                            │                              │
       │  "Move forward 2 meters"   │                              │
       │───────────────────────────►│                              │
       │                            │                              │
       │              AI Agent selects ros2_publish                │
       │              Safety hook validates (0.5 m/s < 1.0 limit) │
       │                            │                              │
       │                            │  publish /cmd_vel            │
       │                            │─────────────────────────────►│
       │                            │  (via Mode A, B, or C)       │  Motors
       │                            │                              │  engage
       │                            │  subscribe /odom             │
       │                            │◄─────────────────────────────│
       │                            │                              │
       │  "Done! Moved 2m forward"  │                              │
       │◄───────────────────────────│                              │
       │                            │                              │

  11. Understand the TurtleBot3 Chat Control architecture

    main

    The control flow for a command follows this path:

    1. User Input: Sent via a mobile device/messaging app.
    2. OpenClaw: Receives the message.
    3. RosClaw plugin: Processes the natural language and translates it to ROS commands.
    4. rosbridge: Bridges the communication to the ROS environment.
    5. Gazebo TurtleBot3: Executes the command in the simulation.

  12. Understand the RosClaw System Architecture

    main

    RosClaw is organized into two primary layers: the AI Gateway Layer and the ROS2 Layer. The system supports different deployment modes (A, B, or C) which change only the transport mechanism between these two layers, while the internal logic of each layer remains consistent.

    AI Gateway Layer

    This layer acts as the bridge between messaging platforms (WhatsApp, Telegram, Discord, Slack, Web Chat) and the robot. It consists of:

    • OpenClaw Gateway: Manages user sessions, AI agent intent/tool calling, and cross-conversation memory/state.
    • RosClaw Plugin: The core integration component containing:
      • Tool Registry: Low-level ROS2 primitives (e.g., ros2_publish, ros2_service_call).
      • Skills: High-level capabilities (e.g., navigate-to, take-photo).
      • Safety Validator: Intercepts tool calls to enforce velocity limits, workspace bounds, and blocked operations.
      • Robot Context: Provides the agent with capabilities, topic/service information, and safety configurations.
      • Emergency Stop: A /estop command that bypasses the AI entirely for immediate safety.

    ROS2 Layer

    This layer interacts directly with the robot hardware via the ROS2 DDS Bus. It includes:

    • Standard ROS2 Topics: Such as /cmd_vel (Twist), /odom (Odom), /camera (Image), and /battery (State).
    • Robot Hardware: Motors, Cameras, LIDAR, IMU, etc.
    • Discovery & Stacks: rosclaw_discovery for capability introspection and integration with stacks like Nav2 or MoveIt2.