ROSA: the Robot Operating System Agent

repository·main·Indexed 23 days ago

https://github.com/nasa-jpl/rosa

An AI-powered assistant built on the Langchain framework designed to interact with ROS1 and ROS2 robotics systems using natural language queries. ROSA enables developers to control robots and query system states through conversational interfaces, supporting LLM providers such as ChatOpenAI, AzureChatOpenAI, ChatAnthropic, and ChatOllama. It features a customizable prompt system via RobotSystemPrompts and an extensible toolset managed by ROSATools.

Tokens
10.5K
Snippets
17
Records
71
Agent score
81%

What's inside jpl-rosa

  1. Understand the Project Management Committee (PMC) and roadmap

    main

    The Project Management Committee (PMC) consists of sponsor representatives and key stakeholders. It manages the project's business and strategic aspects.

    PMC Responsibilities

    • Maintaining the overall project roadmap.
    • Determining requirements and commitments to sponsors.
    • Allocating resources and monitoring progress.
    • On-boarding new sponsors and addressing legal considerations.

    Roadmap and Decision Making

    • Stakeholder Meetings: Regular meetings are held to discuss status and propose roadmap changes. If sponsors and stakeholders concur, changes are adopted immediately.
    • Consensus and Final Authority: While the PMC seeks consensus, if a decision cannot be reached, the Product Manager has the final authority to determine project direction.
  2. Understand the Technical Steering Committee (TSC) and decision making

    main

    The Technical Steering Committee (TSC) is responsible for technical direction, contribution policies, conduct guidelines, and maintaining the collaborator list. It oversees the ROSA and CORTEX repositories.

    Technical Decision Process

    • Review Requirement: Prior to implementing a substantial contribution, the design should be reviewed by at least one TSC member.
    • Consensus: The TSC uses a consensus-seeking approach. If consensus cannot be reached during a pull request review or design discussion, the TSC will make a determination on the direction.
    • Nominations: TSC members are typically collaborators who show great dedication or domain expertise. They are expected to be active contributors within the last 12 months.
  3. How to contribute to the ROSA project

    main

    The ROSA project encourages contributions from anyone. You can contribute by providing code, issues, documentation, graphics, or designs. To get started, you should submit an [Issue]([INSERT LINK TO ISSUE TRACKING SYSTEM]).

    If you make a non-trivial contribution, you may be onboarded as a Collaborator, which grants write access to one or more project repositories. If you are planning a substantial contribution or believe you should be granted write access, please contact @RobRoyce at 01-laptop-voiced@icloud.com.

  4. Run ROSA Unit Tests manually with pytest

    main

    ROSA uses the pytest framework for unit testing. You can run specific test suites manually from the project root directory. To view results in a formatted report, you can use the pytest-html plugin.

    Test Suites Available

    Tool CategoryTest File Location
    Calculation Tools./tests/test_rosa/tools/test_calculation.py
    Log Tools./tests/test_rosa/tools/test_log.py
    ROS1 Tools./tests/test_rosa/tools/test_ros1.py
    ROS2 Tools./tests/test_rosa/tools/test_ros2.py
    ROSA Tools./tests/test_rosa/tools/test_rosa_tools.py
    System Tools./tests/test_rosa/tools/test_system.py
  5. Test ROSA Calculation, Log, and System tools

    main

    Use the following instructions to test core utility modules in ROSA:

    Calculation Tools

    • Purpose: Verifies mathematical functions and edge cases in src/rosa/tools/calculation.py.
    • Test File: ./tests/test_rosa/tools/test_calculation.py

    Log Tools

    • Purpose: Verifies log reading, handling, error handling, and filtering in src/rosa/tools/log.py. Test for invalid paths, empty files, and large files.
    • Test File: ./tests/test_rosa/tools/test_log.py

    System Tools

    • Purpose: Verifies verbosity settings, debugging, and system wait functions in src/rosa/tools/system.py.
    • Test File: ./tests/test_rosa/tools/test_system.py
    pytest ./tests/test_rosa/tools/test_calculation.py
    pytest ./tests/test_rosa/tools/test_log.py
    pytest ./tests/test_rosa/tools/test_system.py
  6. Quick Start with ROSA

    main

    To use ROSA, import the ROSA class, provide a Large Language Model (LLM) instance (compatible with Langchain), and specify the ros_version (1 or 2). You can then use the .invoke() method to send natural language queries to your ROS system.

    For detailed LLM configuration, refer to the Model Configuration Wiki page.

    from rosa import ROSA
    
    llm = get_your_llm_here()
    agent = ROSA(ros_version=1, llm=llm)
    agent.invoke("Show me a list of topics that have publishers but no subscribers")
  7. Test ROS1 and ROS2 tool integrations

    main

    To validate the integration and functionality of ROS1 and ROS2 tools within ROSA, you must ensure that the respective ROS environment is correctly set up and running in your local environment before executing the tests.

    ROS1 Tools

    • Purpose: Validates correct interactions with the ROS1 environment (nodes, topics, services, and parameters).
    • Test File: ./tests/test_rosa/tools/test_ros1.py
    • Tip: Ensure tests are skipped if the installed ROS version is incompatible.

    ROS2 Tools

    • Purpose: Validates correct interactions with the ROS2 environment (nodes, topics, services, and parameters).
    • Test File: ./tests/test_rosa/tools/test_ros2.py
    • Tip: Ensure tests are skipped if the installed ROS version is incompatible.
  8. How the blacklist injection works in ROSA tools

    main

    ROSA uses a decorator pattern via inject_blacklist to ensure that a blacklist parameter is always passed to compatible tool functions, even if the LLM does not explicitly provide it.

    When a tool is added to ROSATools via __add_tool, the manager checks if the tool's underlying function has a blacklist variable in its signature. If it does, the function is wrapped with a decorator that merges the ROSATools instance's blacklist with any blacklist provided at runtime. This maintains compatibility with LangChain's execution model while enforcing safety constraints.

  9. Handle interrupts gracefully in ROSA agents

    main

    When running long-running operations (like streaming an LLM response or executing a tool), use the GracefulInterruptHandler context manager. This ensures that a SIGINT (KeyboardInterrupt) is caught and can be used to break out of loops or stop the current operation without terminating the entire process.

    In the stream_response method, the handler allows the agent to stop processing tokens immediately when the user interrupts, while still allowing the main loop to remain active.

    with GracefulInterruptHandler() as handler:
        async for event in self.astream(query):
            if handler.interrupted:
                break