DOCSEARCHbyAlgolia
Submit Docs

E2B Desktop Sandbox Documentation

repository·main·Indexed 23 days ago

https://github.com/e2b-dev/desktop

An open-source, secure virtual desktop environment designed for 'Computer Use' tasks. It provides a sandboxed Linux environment (Xfce) controllable via JavaScript/TypeScript (@e2b/desktop) and Python SDKs to perform mouse/keyboard actions, run bash commands, manage files, and stream the desktop view or specific application windows.

Tokens
6.6K
Snippets
15
Records
31
Agent score
80%

What's inside E2B Desktop Sandbox

  1. Run the Desktop Sandbox JavaScript Example

    main

    To run the basic JavaScript example which demonstrates using the Desktop Sandbox SDK for streaming and mouse interaction within an Electron app, follow these steps:

    1. Prepare environment variables: Copy the example environment file to .env.
    2. Configure API Key: Obtain an API key from e2b.dev/dashboard and add it to your .env file using the E2B_API_KEY key.
    3. Install dependencies: Use npm install to set up the project.
    4. Execute: Run the example using npm start.
    # 1. Create .env file
cp .env.example .env

# 2. Set E2B_API_KEY in .env file
echo 'E2B_API_KEY="your_api_key"' >> .env

# 3. Install dependencies
npm install

# 4. Run
npm start

  2. Stream the desktop screen

    main

    The desktop.stream object allows you to stream the desktop or specific windows.

    Important Constraints:

    • Only one stream can be active at a time. You must call .stop() on the current stream before starting a new one.
    • If windowId is not provided to .start(), the entire desktop will be streamed.
    • If requireAuth: true is used, you must retrieve the authentication key via .getAuthKey() and pass it to .getUrl({ authKey }) to access the stream.
    import { Sandbox } from '@e2b/desktop'

const desktop = await Sandbox.create()

// Start the stream with password protection
await desktop.stream.start({
  requireAuth: true,
})

// Retrieve the authentication key
const authKey = await desktop.stream.getAuthKey()

// Get stream URL
const url = desktop.stream.getUrl({ authKey })
console.log(url)

// Stop the stream
await desktop.stream.stop()

  3. Stream the desktop or specific application windows

    main

    You can stream the entire desktop or a specific application window.

    Important Constraints:

    • Only one stream can be active at a time. You must stop the current stream before starting a new one.
    • If streaming a specific application, the stream will close once that application closes.
    • If the requested application is not yet open, an error will be raised.

    Streaming the whole desktop

    Python

    desktop.stream.start()
url = desktop.stream.get_url()
# To disable interaction:
url = desktop.stream.get_url(view_only=True)
desktop.stream.stop()

    JavaScript

    await desktop.stream.start()
const url = await desktop.stream.getUrl()
// To disable interaction:
const url = await desktop.stream.getUrl({ viewOnly: true })
await desktop.stream.stop()

    Streaming a specific application window

    Python

    window_ids = desktop.get_application_windows("Firefox")
desktop.stream.start(window_id=window_ids[0])
desktop.stream.stop()

    JavaScript

    const windowIds = await desktop.getApplicationWindows('Firefox')
await desktop.stream.start({ windowId: windowIds[0] })
await desktop.stream.stop()

    Streaming with password protection

    To require authentication, use require_auth=True (Python) or requireAuth: true (JavaScript). You must then retrieve the auto-generated key to access the URL.

    Python

    desktop.stream.start(require_auth=True)
auth_key = desktop.stream.get_auth_key()
url = desktop.stream.get_url(auth_key=auth_key)

    JavaScript

    await desktop.stream.start({ requireAuth: true })
const authKey = await desktop.stream.getAuthKey()
const url = await desktop.stream.getUrl({ authKey })

  4. Run the Desktop Sandbox Python Example

    main

    To run the basic Python example demonstrating streaming and mouse movement, follow these steps to set up your environment and dependencies:

    1. Configure Environment Variables: Create a .env file from the template and add your E2B API key.
    2. Install Dependencies: Use poetry to install the required packages.
    3. Execute the Script: Run the main.py script using poetry run.

    Prerequisites:

    • You must have an E2B API key from e2b.dev/dashboard.
    • poetry must be installed on your system.
    # 1. Create .env file
cp .env.example .env

# 2. Set E2B_API_KEY in .env file
# (Edit the .env file to include: E2B_API_KEY="your_api_key")

# 3. Install dependencies
poetry install

# 4. Run
poetry run python main.py

  5. Create and manage a Desktop Sandbox

    main

    Use the Sandbox.create() method to initialize a new virtual desktop environment. You can launch applications, wait for them to initialize, and eventually kill the sandbox when tasks are complete.

    Python Example

    from e2b_desktop import Sandbox

desktop = Sandbox.create()
desktop.launch('google-chrome')
desktop.wait(10000)
# ... perform tasks ...
# desktop.kill()

    JavaScript Example

    import { Sandbox } from '@e2b/desktop'

const desktop = await Sandbox.create()
await desktop.launch('google-chrome')
await desktop.wait(10000)
// ... perform tasks ...
// await desktop.kill()
    from e2b_desktop import Sandbox

desktop = Sandbox.create()
desktop.launch('google-chrome')
desktop.wait(10000)

  7. Create a custom E2B Desktop template

    main

    You can customize the Desktop sandbox (e.g., adding preinstalled packages) by creating a custom template. This involves defining a template based on the existing desktop template and running a build script using the E2B SDK.

    1. Install Dependencies

    Install the E2B SDK and python-dotenv:

    pip install e2b dotenv

    2. Define the Template

    Create a template.py file to inherit from the base desktop template:

    from e2b import Template

template = Template().from_template("desktop")

    3. Create a Build Script

    Create a build.py file. This script uses Template.build to compile your custom configuration. You can specify hardware resources like cpu_count and memory_mb.

    Note: Ensure you have an E2B_API_KEY in your .env file as load_dotenv() will attempt to load it.

    from dotenv import load_dotenv
from template import template
from e2b import Template, default_build_logger

load_dotenv()

Template.build(
    template,
    alias="desktop-custom",
    cpu_count=8,
    memory_mb=8192,
    on_build_logs=default_build_logger(),
)

    4. Run the Build

    Execute your build script:

    python build.py
    # template.py
from e2b import Template
template = Template().from_template("desktop")

# build.py
from dotenv import load_dotenv
from template import template
from e2b import Template, default_build_logger

load_dotenv()

Template.build(
    template,
    alias="desktop-custom",
    cpu_count=8,
    memory_mb=8192,
    on_build_logs=default_build_logger(),
)

  8. Stream a specific application window

    main

    To stream a specific application instead of the whole desktop, pass a windowId to desktop.stream.start().

    Warnings:

    • It will throw an error if the application is not yet open.
    • The stream will automatically close once the application closes.
    • You must stop the current stream before switching to a different application's window.
    import { Sandbox } from '@e2b/desktop'

const desktop = await Sandbox.create()

// Get all windows of the application
const windowIds = await desktop.getApplicationWindows('Firefox')

// Start the stream for the first window found
await desktop.stream.start({ windowId: windowIds[0] })

// Stop the stream
await desktop.stream.stop()

  10. Stream application windows in E2B Desktop

    main

    The E2B Desktop Sandbox allows you to stream specific application windows or the entire desktop to a URL.

    Key Constraints

    • Single Stream Limit: You can only have one active stream at a time. To stream a different application, you must call desktop.stream.stop() before starting a new one.
    • Application Readiness: Calling stream.start() before the application is fully open will raise an error. It is recommended to use desktop.wait(ms) after desktop.launch() to ensure the application is ready.
    • Stream Lifecycle: The stream automatically closes once the application being streamed is closed.

    Streaming Workflow

    1. Launch the app: Use desktop.launch('app-name') (e.g., 'google-chrome', 'code', or 'firefox').
    2. Wait for readiness: Use await desktop.wait(ms) to allow the application to initialize.
    3. Start the stream: Use desktop.stream.start(options).
      • If windowId is provided (via desktop.getCurrentWindowId()), only that window is streamed.
      • If windowId is omitted, the entire desktop is streamed.
      • Use requireAuth: true to secure the stream.
    4. Retrieve URL: Use desktop.stream.getAuthKey() to get an authentication key, then pass it to desktop.stream.getUrl({ authKey }) to get the streamable URL.
    5. Stop the stream: Call desktop.stream.stop() when finished or before switching applications.
    import { Sandbox } from '@e2b/desktop'

// Start a new desktop sandbox
const desktop = await Sandbox.create()

// Launch an application
await desktop.launch('google-chrome')

// Wait for the application to open
await desktop.wait(15000)

// Start streaming the specific window
await desktop.stream.start({
  windowId: await desktop.getCurrentWindowId(),
  requireAuth: true,
})

// Get the stream URL
const authKey = desktop.stream.getAuthKey()
const url = desktop.stream.getUrl({ authKey })
console.log('Stream URL:', url)

// ... perform actions ...

// Stop the stream before launching another app
await desktop.stream.stop()

  11. Build the production desktop template

    main

    To build the official desktop template from this repository, use the build_prod.py script. This process requires poetry for dependency management and an E2B_API_KEY provided in a .env file.

    Steps:

    1. Install build dependencies using poetry:
      poetry install
    2. Configure your environment by adding your API key to a .env file:
      E2B_API_KEY=e2b_***
    3. Execute the production build script:
      poetry run python build_prod.py

    For local development, you can build the desktop-dev template instead using:

    poetry run python build_dev.py
    poetry install
E2B_API_KEY=e2b_***
poetry run python build_prod.py

  12. Use a custom desktop template in Python or JavaScript

    main

    Once a custom template has been built with a specific alias (e.g., desktop-custom), you can instantiate a Sandbox using that alias in either Python or JavaScript.

    # Python
from e2b_desktop import Sandbox

desktop = Sandbox.create(template="desktop-custom")
    // JavaScript
import { Sandbox } from '@e2b/desktop'

const desktop = await Sandbox.create('desktop-custom')