Revyl CLI

repository·main·Indexed 19 days ago

https://github.com/revylai/revyl-cli

A command-line interface for Revyl, an AI-powered mobile testing platform for iOS, Android, Expo, React Native, and Flutter. The CLI enables developers to define tests in natural language or YAML, manage end-to-end (E2E) tests, and run them on cloud devices. It features a Model Context Protocol (MCP) server for integration with Cursor AI, supports agent-driven and CLI-driven PR reviews in CI/CD pipelines, and provides tools for syncing local test files with the Revyl platform.

Tokens
46.8K
Snippets
132
Records
205
Agent score
67%

What's inside revyl-cli

  1. Overview of Revyl CLI

    main
    Revyl CLI is an AI-powered mobile testing tool designed for terminal-based workflows. It allows developers to create, run, and manage end-to-end (E2E) tests for both iOS and Android applications directly from their development environment.
  2. What the Revyl Cursor plugin provides

    main

    The Revyl plugin for Cursor integrates mobile device testing and development workflows directly into the Cursor AI agent. It provides:

    • MCP Server: 11 specialized tools for setup, development loops, and device management, featuring streamed progress, screenshots, and structured outcomes.
    • Viewer Handoff: Provides an inline device app via the MCP Apps host (with an HTTPS viewer link as a portable fallback).
    • Skills: Pre-configured agent skills including revyl-cloud-agent, revyl-mcp-dev-loop, and revyl-ci-sync.
    • Routing Rule: Automatically routes mobile run, preview, and verification requests to Revyl.
    • Runtime Bootstrap: Automatically downloads and pins the correct Revyl CLI runtime version as declared in runtime-manifest.json upon MCP startup.
  3. Select the appropriate Revyl CLI skill for your workflow

    main

    The revyl-cli skill acts as a router to more specialized skills based on your specific task. Choose the skill that matches your goal:

    • revyl-cli-dev-loop: For local development loops and exploratory path capture.
    • revyl-cli-create: For authoring robust YAML tests.
    • revyl-cli-auth-bypass: For setting up authenticated app state specifically for tests.
    • revyl-cli-analyze: For triaging failed runs.
  4. Configure Revyl test YAML building blocks

    main

    Revyl tests are defined as a series of blocks in a YAML file. Each block represents a specific type of action or assertion.

    Supported Block Types

    • instructions: Defines a single meaningful user intent (e.g., "Sign in").
    • validation: A durable assertion about user-visible state (e.g., "The home screen is visible").
    • manual: Framework actions like wait, go_home, navigate, set_location, kill_app, or open_app.
    • extraction: Reads screen data into a specified variable_name.
    • code_execution: Runs a saved script or lightweight inline code.
    • module_import: Imports a reusable module by name.
    • if / while: Conditional logic and loops containing nested blocks.
    test:
      metadata:
        name: smoke-login-ios
        platform: ios
        tags:
          - smoke
      build:
        name: ios-test
      variables:
        email: "{{global.login-email}}"
      blocks:
        - type: instructions
          step_description: "Sign in with {{email}}."
        - type: validation
          step_description: "The home screen is visible."
  5. Classify Revyl MCP test failures

    main

    When analyzing test failures with the revyl-mcp-analyze skill, use the following classification rules to categorize the root cause:

    • REAL BUG: Actions completed successfully, but the application behavior contradicts the expected outcome.
    • FLAKY TEST: The application behavior is acceptable, but the test validation logic is too brittle.
    • INFRA ISSUE: No meaningful application steps were executed, or there was a failure in setup, device, or build processes.
    • TEST IMPROVEMENT: The test structure is weak or allows for ambiguity in results.
  6. Expo Router Auth Bypass Implementation Pattern

    main

    For Expo Router apps, implement the bypass using a dedicated module and a root-level hook. It is recommended to add a 'backstop route' at app/revyl-auth.tsx to handle unmatched routes and provide visible feedback if the bypass fails.

    • app/_layout.tsx: Install the useRevylAuthBypass hook here to handle initial and runtime Linking events.
    • app/revyl-auth.tsx: A backstop route that calls the handler to prevent unmatched-route screens.
    • src/auth/revylAuthBypass.tsx: The core logic containing token, role, and redirect validation.
    • Account/Debug Screen: Display visible accepted/rejected states in test builds.
    import { useEffect } from "react";
    import * as Linking from "expo-linking";
    import { router } from "expo-router";
    
    const allowedRedirects = new Map([
      ["/account", "/(tabs)/account"],
      ["/cart", "/cart"],
      ["/checkout", "/checkout"],
    ]);
    const allowedRoles = new Set(["buyer", "support"]);
    
    function launchValue(key: string) {
      return process.env[key];
    }
    
    export function handleRevylAuthBypass(rawURL: string) {
      const url = new URL(rawURL);
      if (url.protocol !== "myapp:" || url.hostname !== "revyl-auth") {
        return false;
      }
    
      const enabled = launchValue("REVYL_AUTH_BYPASS_ENABLED") === "true";
      const expectedToken = launchValue("REVYL_AUTH_BYPASS_TOKEN");
      const token = url.searchParams.get("token");
      const role = url.searchParams.get("role") || "buyer";
      const redirect = url.searchParams.get("redirect") || "/account";
      const route = allowedRedirects.get(redirect);
    
      if (!enabled) throw new Error("Revyl auth bypass is disabled");
      if (!expectedToken || token !== expectedToken) throw new Error("Bad Revyl auth bypass token");
      if (!allowedRoles.has(role)) throw new Error("Role is not allowlisted");
      if (!route) throw new Error("Redirect is not allowlisted");
    
      createTestSession({ role });
      router.replace(route);
      return true;
    }
    
    export function useRevylAuthBypass() {
      useEffect(() => {
        Linking.getInitialURL().then(url => {
          if (url) handleRevylAuthBypass(url);
        });
    
        const subscription = Linking.addEventListener("url", event => {
          handleRevylAuthBypass(event.url);
        });
    
        return () => subscription.remove();
      }, []);
    }
  7. Guardrails for Android Auth Bypass

    main

    When implementing the Android auth bypass, adhere to these security and safety practices:

    1. No Production Bypass: Never ship an unconditional bypass in production code.
    2. Build Gating: Gate the implementation behind debug/staging/test builds and require REVYL_AUTH_BYPASS_ENABLED=true.
    3. Token Privacy: Never include raw tokens in source code, YAML files, screenshots, or Pull Request descriptions.
    4. Strict Allowlisting: Keep the role and redirect allowlists small and specific to the application's needs.
    5. Visible Failure States: Ensure the app visibly shows a rejected state so that automated agents can diagnose why a bypass failed.
  8. Authoring rules for Revyl tests

    main

    Follow these rules to ensure high-quality, maintainable test YAML:

    1. Atomicity: Assign exactly one action per instruction step.
    2. Separation of Concerns: Keep validations separate from instructions.
    3. Outcome-Oriented: Always validate user-visible outcomes.
    4. Security/Dynamics: Use variables for any sensitive or dynamic values.
  9. Guidelines for interacting with the device

    main

    When using the revyl-mcp-dev-loop skill, adhere to these execution guardrails to ensure reliable automation:

    • Action Syntax: Express all device actions using natural language via the interact tool. For example: interact(task="Tap the Sign In button").
    • No Coordinates: Never attempt to calculate or supply raw X/Y coordinates; rely on the natural language task description.
    • State Anchoring: Always call screenshot() before performing any action that depends on the current UI state to ensure your mental model matches the device.
    • Verification: After an action, call screenshot() to verify the outcome. You are allowed a short burst of up to two actions before verification for obvious two-step flows (e.g., entering a password and hitting enter).
    • Tool Constraints:
      • The first tool call must always be start_dev_loop.
      • Do not call listing tools unless explicitly requested by the user.
      • Use setup_status only when the user explicitly asks for setup diagnostics.
      • Treat next_steps as advisory only.
  10. Framework-specific development strategies

    main

    Revyl adapts its development loop based on your mobile framework:

    • Expo: Uses a Revyl-managed relay and Expo dev client for JS/TS hot reload. Rebuild only for native changes (modules, SDK, permissions). Use revyl-cli-auth-bypass to handle authentication friction.
    • React Native (Bare): Uses the Metro relay. JS/TS changes hot reload; native dependency or Gradle/Podfile changes require a rebuild.
    • Flutter: Uses a rebuild-first loop. Since there is no Metro/Expo dev server for cloud hot reload, revyl dev installs and runs the build. Use revyl dev rebuild --wait for explicit rebuilds.
    • Native (iOS/Swift, Android/Kotlin): Uses a rebuild-first loop. Any native change requires a build, upload, and relaunch.
    • KMP/Bazel: Treat the configured build.platforms.<key>.output artifact as the app. Iterate between the build command and revyl dev rebuild --wait.
  11. How the Revyl runtime bootstrap works

    main

    The plugin manages the Revyl CLI runtime to ensure environment consistency. Each plugin release pins an immutable Revyl CLI GitHub Release and specific SHA-256 checksums for all supported OS/architectures.

    Bootstrap Logic:

    • On MCP startup, the launcher selects the matching asset.
    • It reuses existing assets only if the checksum matches the pin.
    • It can adopt an already-installed Revyl CLI by copying it into the cache if the checksum matches.
    • If no match is found, it downloads the asset to a temporary file and installs it atomically after verification.
    • Corrupt cache entries are repaired automatically on the next online start.

    Custom Binaries: Developers can bypass the automatic bootstrap by selecting an existing executable using the REVYL_BINARY environment variable.

  12. Security Guardrails for iOS Auth Bypass

    main

    When implementing the Revyl iOS Auth Bypass, adhere to these security best practices to prevent accidental production exposure:

    1. No Unconditional Production Bypass: Never allow this logic to run in production builds.
    2. Build Gating: Gate the feature using simulator/debug/staging/test build configurations and require REVYL_AUTH_BYPASS_ENABLED=true.
    3. Token Privacy: Never commit raw tokens to source code, YAML files, screenshots, or pull request descriptions.
    4. Strict Allowlisting: Keep the role and redirect allowlists small and specific to the testing needs.
    5. Visible Failure States: Ensure the app visibly shows when an auth-bypass attempt is rejected so that automated agents can diagnose the failure.