Revyl CLI
repository·main·Indexed 19 days ago
https://github.com/revylai/revyl-cliA command-line interface for Revyl, an AI-powered mobile testing platform for iOS, Android, Expo, React Native, and Flutter. The CLI enables developers to define tests in natural language or YAML, manage end-to-end (E2E) tests, and run them on cloud devices. It features a Model Context Protocol (MCP) server for integration with Cursor AI, supports agent-driven and CLI-driven PR reviews in CI/CD pipelines, and provides tools for syncing local test files with the Revyl platform.
What's inside revyl-cli
- Revyl CLI is an AI-powered mobile testing tool designed for terminal-based workflows. It allows developers to create, run, and manage end-to-end (E2E) tests for both iOS and Android applications directly from their development environment.
What the Revyl Cursor plugin provides
mainThe Revyl plugin for Cursor integrates mobile device testing and development workflows directly into the Cursor AI agent. It provides:
- MCP Server: 11 specialized tools for setup, development loops, and device management, featuring streamed progress, screenshots, and structured outcomes.
- Viewer Handoff: Provides an inline device app via the MCP Apps host (with an HTTPS viewer link as a portable fallback).
- Skills: Pre-configured agent skills including
revyl-cloud-agent,revyl-mcp-dev-loop, andrevyl-ci-sync. - Routing Rule: Automatically routes mobile run, preview, and verification requests to Revyl.
- Runtime Bootstrap: Automatically downloads and pins the correct Revyl CLI runtime version as declared in
runtime-manifest.jsonupon MCP startup.
Select the appropriate Revyl CLI skill for your workflow
mainThe
revyl-cliskill acts as a router to more specialized skills based on your specific task. Choose the skill that matches your goal:revyl-cli-dev-loop: For local development loops and exploratory path capture.revyl-cli-create: For authoring robust YAML tests.revyl-cli-auth-bypass: For setting up authenticated app state specifically for tests.revyl-cli-analyze: For triaging failed runs.
Configure Revyl test YAML building blocks
mainRevyl tests are defined as a series of blocks in a YAML file. Each block represents a specific type of action or assertion.
Supported Block Types
instructions: Defines a single meaningful user intent (e.g., "Sign in").validation: A durable assertion about user-visible state (e.g., "The home screen is visible").manual: Framework actions likewait,go_home,navigate,set_location,kill_app, oropen_app.extraction: Reads screen data into a specifiedvariable_name.code_execution: Runs a saved script or lightweight inline code.module_import: Imports a reusable module by name.if/while: Conditional logic and loops containing nested blocks.
test: metadata: name: smoke-login-ios platform: ios tags: - smoke build: name: ios-test variables: email: "{{global.login-email}}" blocks: - type: instructions step_description: "Sign in with {{email}}." - type: validation step_description: "The home screen is visible."Classify Revyl MCP test failures
mainWhen analyzing test failures with the
revyl-mcp-analyzeskill, use the following classification rules to categorize the root cause:- REAL BUG: Actions completed successfully, but the application behavior contradicts the expected outcome.
- FLAKY TEST: The application behavior is acceptable, but the test validation logic is too brittle.
- INFRA ISSUE: No meaningful application steps were executed, or there was a failure in setup, device, or build processes.
- TEST IMPROVEMENT: The test structure is weak or allows for ambiguity in results.
Expo Router Auth Bypass Implementation Pattern
mainFor Expo Router apps, implement the bypass using a dedicated module and a root-level hook. It is recommended to add a 'backstop route' at
app/revyl-auth.tsxto handle unmatched routes and provide visible feedback if the bypass fails.Recommended File Structure
app/_layout.tsx: Install theuseRevylAuthBypasshook here to handle initial and runtimeLinkingevents.app/revyl-auth.tsx: A backstop route that calls the handler to prevent unmatched-route screens.src/auth/revylAuthBypass.tsx: The core logic containing token, role, and redirect validation.- Account/Debug Screen: Display visible accepted/rejected states in test builds.
import { useEffect } from "react"; import * as Linking from "expo-linking"; import { router } from "expo-router"; const allowedRedirects = new Map([ ["/account", "/(tabs)/account"], ["/cart", "/cart"], ["/checkout", "/checkout"], ]); const allowedRoles = new Set(["buyer", "support"]); function launchValue(key: string) { return process.env[key]; } export function handleRevylAuthBypass(rawURL: string) { const url = new URL(rawURL); if (url.protocol !== "myapp:" || url.hostname !== "revyl-auth") { return false; } const enabled = launchValue("REVYL_AUTH_BYPASS_ENABLED") === "true"; const expectedToken = launchValue("REVYL_AUTH_BYPASS_TOKEN"); const token = url.searchParams.get("token"); const role = url.searchParams.get("role") || "buyer"; const redirect = url.searchParams.get("redirect") || "/account"; const route = allowedRedirects.get(redirect); if (!enabled) throw new Error("Revyl auth bypass is disabled"); if (!expectedToken || token !== expectedToken) throw new Error("Bad Revyl auth bypass token"); if (!allowedRoles.has(role)) throw new Error("Role is not allowlisted"); if (!route) throw new Error("Redirect is not allowlisted"); createTestSession({ role }); router.replace(route); return true; } export function useRevylAuthBypass() { useEffect(() => { Linking.getInitialURL().then(url => { if (url) handleRevylAuthBypass(url); }); const subscription = Linking.addEventListener("url", event => { handleRevylAuthBypass(event.url); }); return () => subscription.remove(); }, []); }Guardrails for Android Auth Bypass
mainWhen implementing the Android auth bypass, adhere to these security and safety practices:
- No Production Bypass: Never ship an unconditional bypass in production code.
- Build Gating: Gate the implementation behind debug/staging/test builds and require
REVYL_AUTH_BYPASS_ENABLED=true. - Token Privacy: Never include raw tokens in source code, YAML files, screenshots, or Pull Request descriptions.
- Strict Allowlisting: Keep the
roleandredirectallowlists small and specific to the application's needs. - Visible Failure States: Ensure the app visibly shows a rejected state so that automated agents can diagnose why a bypass failed.
Authoring rules for Revyl tests
mainFollow these rules to ensure high-quality, maintainable test YAML:
- Atomicity: Assign exactly one action per instruction step.
- Separation of Concerns: Keep validations separate from instructions.
- Outcome-Oriented: Always validate user-visible outcomes.
- Security/Dynamics: Use variables for any sensitive or dynamic values.
Guidelines for interacting with the device
mainWhen using the
revyl-mcp-dev-loopskill, adhere to these execution guardrails to ensure reliable automation:- Action Syntax: Express all device actions using natural language via the
interacttool. For example:interact(task="Tap the Sign In button"). - No Coordinates: Never attempt to calculate or supply raw X/Y coordinates; rely on the natural language task description.
- State Anchoring: Always call
screenshot()before performing any action that depends on the current UI state to ensure your mental model matches the device. - Verification: After an action, call
screenshot()to verify the outcome. You are allowed a short burst of up to two actions before verification for obvious two-step flows (e.g., entering a password and hitting enter). - Tool Constraints:
- The first tool call must always be
start_dev_loop. - Do not call listing tools unless explicitly requested by the user.
- Use
setup_statusonly when the user explicitly asks for setup diagnostics. - Treat
next_stepsas advisory only.
- The first tool call must always be
- Action Syntax: Express all device actions using natural language via the
Framework-specific development strategies
mainRevyl adapts its development loop based on your mobile framework:
- Expo: Uses a Revyl-managed relay and Expo dev client for JS/TS hot reload. Rebuild only for native changes (modules, SDK, permissions). Use
revyl-cli-auth-bypassto handle authentication friction. - React Native (Bare): Uses the Metro relay. JS/TS changes hot reload; native dependency or Gradle/Podfile changes require a rebuild.
- Flutter: Uses a rebuild-first loop. Since there is no Metro/Expo dev server for cloud hot reload,
revyl devinstalls and runs the build. Userevyl dev rebuild --waitfor explicit rebuilds. - Native (iOS/Swift, Android/Kotlin): Uses a rebuild-first loop. Any native change requires a build, upload, and relaunch.
- KMP/Bazel: Treat the configured
build.platforms.<key>.outputartifact as the app. Iterate between the build command andrevyl dev rebuild --wait.
- Expo: Uses a Revyl-managed relay and Expo dev client for JS/TS hot reload. Rebuild only for native changes (modules, SDK, permissions). Use
How the Revyl runtime bootstrap works
mainThe plugin manages the Revyl CLI runtime to ensure environment consistency. Each plugin release pins an immutable Revyl CLI GitHub Release and specific SHA-256 checksums for all supported OS/architectures.
Bootstrap Logic:
- On MCP startup, the launcher selects the matching asset.
- It reuses existing assets only if the checksum matches the pin.
- It can adopt an already-installed Revyl CLI by copying it into the cache if the checksum matches.
- If no match is found, it downloads the asset to a temporary file and installs it atomically after verification.
- Corrupt cache entries are repaired automatically on the next online start.
Custom Binaries: Developers can bypass the automatic bootstrap by selecting an existing executable using the
REVYL_BINARYenvironment variable.Security Guardrails for iOS Auth Bypass
mainWhen implementing the Revyl iOS Auth Bypass, adhere to these security best practices to prevent accidental production exposure:
- No Unconditional Production Bypass: Never allow this logic to run in production builds.
- Build Gating: Gate the feature using simulator/debug/staging/test build configurations and require
REVYL_AUTH_BYPASS_ENABLED=true. - Token Privacy: Never commit raw tokens to source code, YAML files, screenshots, or pull request descriptions.
- Strict Allowlisting: Keep the
roleandredirectallowlists small and specific to the testing needs. - Visible Failure States: Ensure the app visibly shows when an auth-bypass attempt is rejected so that automated agents can diagnose the failure.