DOCSEARCHbyAlgolia
Submit Docs

scrcpy Documentation

repository·master·Indexed 33 days ago

https://github.com/genymobile/scrcpy

scrcpy is a tool to display and control Android devices via USB or TCP/IP. It supports high-performance screen mirroring, audio forwarding, camera mirroring, and virtual displays. Features include OTG mode for controlling devices without USB debugging, various input simulation modes (UHID, AOA, SDK) for keyboard, mouse, and gamepad, and extensive CLI configuration for video bit rate, resolution, frame rate, and audio codecs.

Tokens
23.6K
Snippets
47
Records
151
Agent score
99%

What's inside scrcpy

  1. Overview of scrcpy

    master
    scrcpy (pronounced "screen copy") is an application that mirrors Android devices (video and audio) via USB or TCP/IP. It allows you to control the device using your computer's keyboard and mouse without requiring root access or installing an app on the device. It is compatible with Linux, Windows, and macOS.

  2. Understand the scrcpy client-server architecture

    master

    scrcpy consists of two main parts:

    1. The Server (scrcpy-server): A Java application executed on the Android device as the shell user. It captures the screen and audio, and handles input event injection.
    2. The Client (scrcpy binary): A C application executed on the host computer. It is responsible for pushing the server to the device, starting it, and managing the communication sockets.

    Communication via Sockets

    The client and server communicate using separate sockets for different purposes. Depending on your configuration, there may be 1, 2, or 3 sockets active:

    • Video Socket: The server sends a raw video stream (H.264 by default) to the client.
    • Audio Socket: The server sends a raw audio stream (OPUS by default) to the client.
    • Control Socket: A bidirectional socket used for transmitting input events (keyboard/mouse) from the client to the server, and device messages (like clipboard changes) from the server to the client.

    Connection Roles

    By default (unless --force-adb-forward is used), the connection roles are inverted at the network level to prevent race conditions: the client opens a server socket and listens on a port, and the server connects to the client.

  3. Understand the scrcpy connection protocol

    master

    The protocol between the client and server is internal and subject to change between versions. A client must always match the server version.

    Connection Setup

    1. ADB Tunnel: The client sets up a tunnel using adb reverse (default) or adb forward (if --force-adb forward is set).
      • <SCID> is a 31-bit random number used to identify different clients on the same device.
    2. Socket Opening: Up to 3 sockets are opened in this order: video, audio, and control. Sockets can be disabled via --no-video, --no-audio, or --no-control.
    3. Handshake: On the first socket opened, if using a forward tunnel, the device sends a dummy byte to detect connection errors. The device then sends metadata (currently the device name) to the client.

    Video and Audio Stream Formats

    On the video and audio sockets, the device first sends a u32 codec ID.

    Video Codec IDs:

    • h264: 0x68323634
    • h265: 0x68323635
    • av1: 0x00617631
    • vp8: 0x00767038
    • vp9: 0x00767039

    Audio Codec IDs:

    • opus: 0x6F707573
    • aac: 0x00616163
    • flac: 0x666C6163
    • raw: 0x00726177

  4. Understand the scrcpy client architecture

    master

    The scrcpy client uses SDL for cross-platform UI, input events, and threading, and FFmpeg for decoding video and audio streams.

    Component Workflow

    • Demuxer: Extracts video and audio packets from the stream.
    • Decoder: Processes demuxed packets to produce frames (one per stream).
    • Recorder: Muxes video and audio packets into a container (MKV or MP4) on the client side.
    • Display: Renders video frames in the scrcpy window or sends them to a V4L2 sink.
    • Audio Player: Plays decoded audio samples.
    • Controller: Runs in a separate thread to send control messages to the device. It receives SDL events from the main thread via an input manager, converts them to Android events, and serializes them for the device.

  5. Choose a keyboard input mode

    master

    scrcpy supports several keyboard input modes depending on your needs for character support, connection type, and device compatibility:

    • --keyboard=sdk (default): Injects events at the Android API level. Best for general use but limited to ASCII and some characters. Requires specific developer options on some devices.
    • --keyboard=uhid (or -K): Simulates a physical HID keyboard using the UHID kernel module. Recommended for regular use as it supports all characters/IMEs, allows disabling the on-screen keyboard, and works over TCP/IP.
    • --keyboard=aoa: Simulates a physical HID keyboard using the AOAv2 protocol. Works at the USB level, does not require adb or the scrcpy server, and can work even with USB debugging disabled. Note that it only works over USB and may have issues on Windows when mirroring.
    • --keyboard=disabled: Disables keyboard input entirely.
    # Use the recommended UHID mode
scrcpy --keyboard=uhid

# Use the AOA mode (USB only)
scrcpy --keyboard=aoa

# Use the default SDK mode
scrcpy --keyboard=sdk

  6. How to use repeated key shortcuts

    master

    Some actions require pressing a key multiple times while holding the modifier. To execute these, you must release and press the key a second time while still holding the <kbd>MOD</kbd> key.

    Example: Expand settings panel (<kbd>MOD</kbd>+<kbd>n</kbd>+<kbd>n</kbd>)

    1. Press and keep pressing <kbd>MOD</kbd>.
    2. Then double-press <kbd>n</kbd> (press, release, press).
    3. Finally, release <kbd>MOD</kbd>.

  7. Server components and responsibilities

    master

    The scrcpy-server manages several components running in dedicated threads:

    • Video Streamer: Uses ScreenCapture and SurfaceEncoder (via Android's MediaCodec API) to capture the screen and send encoded packets over the video socket.
    • Audio Streamer: Uses AudioRecord to capture audio and the MediaCodec asynchronous API to encode it, sending packets over the audio socket.
    • Controller: Runs in a separate thread to handle the bidirectional control socket. It receives input events (keycodes, text, mouse motion/clicks, scrolls, and commands) and injects them into the system using InputManager.injectInputEvent().

  8. How physical mouse simulation (UHID/AOA) works

    master

    When using --mouse=uhid or --mouse=aoa, scrcpy enters a 'mouse capture' mode. The computer's mouse pointer is hidden, and control is handed over to the Android device as if a physical mouse were plugged in.

    To toggle mouse capture (enable or disable it) without exiting scrcpy, use the shortcut mod (typically <kbd>Alt</kbd> or <kbd>Super</kbd>). This allows you to regain control of the mouse on your computer.

  9. Audio forwarding requirements and behavior

    master

    Audio forwarding is enabled by default but has specific requirements based on the Android version:

    • Android 12 or newer: Works out-of-the-box.
    • Android 11: The device screen must be unlocked when starting scrcpy. A fake popup will briefly appear to ensure the shell app is in the foreground; without this, audio capture will fail.
    • Android 10 or earlier: Audio capture is not supported and is automatically disabled.

    If audio capture fails, scrcpy will continue with video only unless you use the --require-audio flag.

  10. Video encoding details

    master

    Video encoding is performed using the Android MediaCodec API. The SurfaceEncoder encodes the content of a Surface associated with the display.

    Key Behaviors:

    • Rotation: On device rotation or folding, the encoding session is automatically reset and restarted.
    • Frame Production: New frames are produced only when changes occur on the surface to save bandwidth.
    • Potential Issues: Because frames are only sent on changes, the first frame might not be sent immediately if the screen is static, or the last frame after fast motion might have poor quality. This can be mitigated using the KEY_REPEAT_PREVIOUS_FRAME_AFTER flag in the MediaFormat.

  11. How scrcpy architecture works

    master

    scrcpy consists of two main parts:

    1. The Server (scrcpy-server): A Java application executed on the Android device (as shell). It serves video and audio streams and handles requests from the client.
    2. The Client (scrcpy binary): Executed on the host computer. It is responsible for pushing the server to the device, starting its execution, and controlling the device.

    Communication

    The client and server communicate via separate sockets for video, audio, and controls.

    • Video Socket: Server sends a raw video stream (H.264 by default). The client decodes and displays frames with minimal latency.
    • Audio Socket: Server sends a raw audio stream (OPUS by default). The client decodes and maintains minimal buffering.
    • Control Socket: A bidirectional socket. The client sends input events (keyboard/mouse) to the server, and the server sends device messages (like clipboard changes) back to the client.

    Network Roles

    By default (unless --force-adb-forward is used), the roles are inverted at the network level to prevent race conditions: the client opens a server socket and listens, and the server connects to the client.

  12. Build scrcpy from source

    master

    To build both the scrcpy client and server, you must have the Android SDK installed and set the ANDROID_SDK_ROOT environment variable.

    Prerequisites:

    • adb must be available in your PATH.
    • The client requires FFmpeg and SDL.
    • The server requires the JDK.

    Steps:

    1. Set ANDROID_SDK_ROOT to your Android SDK directory.
    2. Configure the build using meson.
    3. Compile using ninja.

    Note: Always run ninja as a non-root user. Only use sudo when running ninja install or ninja uninstall.

    # Set Android SDK root (examples)
export ANDROID_SDK_ROOT=~/Android/Sdk  # Linux
export ANDROID_SDK_ROOT=~/Library/Android/sdk  # Mac
set ANDROID_SDK_ROOT=%LOCALAPPDATA%\Android\sdk  # Windows

# Build
meson setup x --buildtype=release --strip -Db_lto=true
ninja -Cx x

# Install
sudo ninja -Cx x install