DOCSEARCHbyAlgolia
Submit Docs

robotjs

repository·master·Indexed 11 days ago

https://github.com/octalmage/robotjs

A Node.js desktop automation library for programmatic control of the mouse, keyboard, and screen. Version 0.9.1 includes capabilities for screen capture, image matching, color detection, and coordinate translation via the Image and Bitmap classes.

Tokens
1K
Snippets
4
Records
4
Agent score
46%

What's inside robotjs

  1. Convert image coordinates to screen coordinates

    master

    The Image.prototype.toScreenPoint(point, target) method translates a coordinate relative to the image into an absolute screen coordinate. This is useful when working with high-DPI displays or scaled screenshots.

    • point: An object { x, y } representing the local coordinate within the image.
    • target (optional): An object containing width and height used to calculate scaling offsets.

    It uses the image's internal scaleX, scaleY, screenX, and screenY properties to ensure the returned point correctly maps to the physical monitor position.

    const img = robotjs.screen.capture();
const localPoint = { x: 10, y: 10 };
const screenPoint = img.toScreenPoint(localPoint);

console.log(`Screen coordinates: ${screenPoint.x}, ${screenPoint.y}`);

  2. Load and save images using image namespace

    master

    The robotjs.image namespace provides utility methods for handling image files.

    • robotjs.image.load(path): Loads an image from the specified file path and returns an Image instance.
    • robotjs.image.save(bitmap, path): Saves a Bitmap/Image instance to the specified file path.
    • robotjs.image.supportsPNG: A boolean indicating if the current environment supports PNG format.
    const robotjs = require('robotjs');

// Load an image
const img = robotjs.image.load('input.png');

// Save an image
robotjs.image.save(img, 'output.png');

if (robotjs.image.supportsPNG) {
  console.log('PNG support is available');
}

  3. Capture the screen using screen.capture()

    master

    The robotjs.screen.capture() method allows you to take a screenshot of the current display. You can capture the entire screen or a specific rectangular region.

    Returns an instance of Image containing the captured data.

    const robotjs = require('robotjs');

// Capture the entire screen
const fullScreenshot = robotjs.screen.capture();

// Capture a specific region: x, y, width, height
const regionScreenshot = robotjs.screen.capture(100, 100, 500, 500);

  4. Use the Image class for computer vision tasks

    master

    The Image class (and its alias Bitmap) provides a high-level interface for performing computer vision operations on captured screen data or loaded image files. It wraps native bitmap data and provides methods for color detection, image searching, and interaction.

    Key capabilities include:

    • Color Operations: colorAt(x, y), findColor(color, options), findColors(color, options), and countColor(color, options).
    • Image Matching: findImage(needle, options), findImages(needle, options), and countImage(needle, options).
    • File I/O: save(path) to save the image to disk.
    • Interaction: click(point, target, button, double) to move the mouse to a point within the image and click, or clickImage(target, options, button, double) to find a sub-image and click it.
    const robotjs = require('robotjs');

// Load an image from disk
const img = robotjs.image.load('path/to/image.png');

// Find a specific color
const color = img.colorAt(10, 10);

// Find a sub-image and click it
img.clickImage({ width: 50, height: 50 }, {}, 'left', true);