DOCSEARCHbyAlgolia
Submit Docs

nut.js Documentation

repository·develop·Indexed 25 days ago

https://github.com/nut-tree/nut.js

A desktop automation library for controlling the keyboard, mouse, and screen. It provides APIs for simulating user interactions, image-based screen searching via @nut-tree/nut-js, and platform-specific provider installation through @nut-tree/libnut. Features include mouse movement with easing functions, keyboard sequence simulation, screen capturing, and Jest integration for automation testing.

Tokens
11.5K
Snippets
28
Records
117
Agent score
83%

What's inside nut.js

  1. Overview of @nut-tree/nut-js

    develop
    The @nut-tree/nut-js package is the core library of the nut.js ecosystem. It serves as the central integration point for all other nut.js packages and provides the fundamental APIs required to automate user interactions with the screen, keyboard, and mouse.

  2. Use @nut-tree/provider-interfaces for custom implementations

    develop
    The @nut-tree/provider-interfaces package provides the standard interface definitions used throughout the nut.js ecosystem. Developers should use these interfaces when creating custom providers (e.g., for custom mouse, keyboard, or screen control implementations) to ensure compatibility with the core @nut-tree/nut-js library.

  8. Configure nut.js Screen Control behavior

    develop

    The screen.config object allows you to customize how image searching and highlighting behave. Use these keys to adjust matching sensitivity, visual feedback, and asset loading:

    • confidence: A number specifying the required matching percentage for a template image to be considered a match.
    • autoHighlight: A boolean toggle. When true, matching Region results are automatically highlighted with an opaque window.
    • highlightDurationMs: The duration in milliseconds that a highlight window remains visible.
    • highlightOpacity: The opacity of highlight windows, ranging from 0 (fully transparent) to 1 (fully opaque).
    • resourceDirectory: The directory path where template assets are loaded from. This is useful for managing platform-specific images (e.g., different assets for Windows vs. macOS) by switching the directory at runtime.

  10. Highlight a screen region in TypeScript

    develop

    You can use the screen object and the Region class from @nut-tree/nut-js to perform visual operations. The following example highlights a specific rectangular area on the screen.

    import { Region, screen } from "@nut-tree/nut-js";

(async () => {
	await screen.highlight(new Region(100, 200, 300, 400));
})();

  11. Customize movement with EasingFunctions

    develop

    An EasingFunction can be passed to mouse.move to modify the speed throughout the path. The function receives a percentage (0 to 1) representing the progress along the path. The total speed is calculated as: speedInPixels = baseSpeed + easingFunction(idx / amountOfSteps) * baseSpeed.

    Example: A function that moves slower at the start and faster at the end.

    const firstSlowThenFast = (percentage) => {
	return percentage <= 0.5 ? -0.75 : 0.75;
}

await mouse.move(left(1000), firstSlowThenFast);

  12. Move to the center or a random point in a region

    develop

    You can use helper functions to target specific areas of a Region (often obtained via screen.find):

    • centerOf(region: Region): Returns the center Point of a region.
    • randomPointIn(region: Region): Returns a random Point within a region.

    These are commonly used with straightTo to move the mouse to an element found on screen.