DOCSEARCHbyAlgolia
Submit Docs

ExTester UI Testing Framework

repository·main·Indexed 43 days ago

https://github.com/redhat-developer/vscode-extension-tester

A UI testing framework for Visual Studio Code extensions that uses Selenium WebDriver to run tests against real VS Code instances. It provides a high-level Page Object API to interact with the editor UI, a CLI for setup and execution, and an optional ExTester Runner extension for managing tests, screenshots, and logs directly within VS Code.

Tokens
75.2K
Snippets
282
Records
341
Agent score
49%

What's inside ExTester

  1. Overview of ExTester for VS Code UI Testing

    main

    ExTester is a UI testing framework designed specifically for Visual Studio Code extensions, powered by Selenium WebDriver. It automates the heavy lifting of setting up a testing environment by:

    • Downloading VS Code: Automatically fetches a test instance of Visual Studio Code (supports any supported version, including Stable and Insiders).
    • Managing ChromeDriver: Automatically downloads the exact ChromeDriver build that matches the fetched VS Code instance.
    • Extension Packaging: Packs and installs your specific extension into the test VS Code instance.
    • Test Execution: Launches VS Code under Selenium WebDriver and executes your Mocha-based UI tests against the running instance.

  3. Explore the ExTester Runner Activity Bar and Tree Views

    main

    The ExTester Runner extension provides a dedicated container in the VS Code Activity Bar that hosts three specialized tree views for managing and inspecting test runs:

    • UI Tests (extesterView): Displays your UI test files and their hierarchical structure, including Mocha describe suites and it test cases. It reflects test status (e.g., [only] or [skip]) via icons and labels.
    • Screenshots (extesterResourcesScreenshotsView): Lists screenshot image files generated during test runs, typically found in a screenshots subdirectory of the output. Clicking an image opens the VS Code image preview.
    • Logs (extesterResourcesLogsView): Displays log files organized by directory. The location is determined by the tempFolder setting or the TEST_RESOURCES environment variable.

  4. Understand the ExTester Runner UI test fixture

    main

    The example-project folder is a specialized fixture used exclusively for testing the ExTester Runner extension itself. It is designed to verify that the extension's UI Tests view can correctly discover, parse, and execute various test file structures.

    Important: This project is not a template for creating your own extension tests. If you want to learn how to test your own VS Code extension using ExTester, use the vscode-extension-tester-example repository instead.

  5. ExTester Runner Activity Bar views

    main

    The ExTester Runner provides three specialized views within a single Activity Bar container:

    • UI Tests: Displays a tree of folders, files, describe suites, and it cases matched by your testFileGlob. It highlights .only and .skip modifiers and allows jumping to the source code by clicking a node. You can run single files, folders, or all tests using the play buttons.
    • Screenshots: A list of all screenshots captured during test runs. You can clear, delete, or reveal them via the view toolbars and context menus.
    • Logs: A list of log files generated by test runs. You can clear, delete, or reveal them via the view toolbars and context menus.

  6. ExTester Packages and Components

    main

    ExTester is composed of several specialized packages:

    • vscode-extension-tester: The core framework providing the extest CLI and the programmatic API.
    • @redhat-developer/page-objects: The Page Object API for interacting with the VS Code UI (re-exported by the main framework).
    • @redhat-developer/locators: Version-specific element locators used by the page objects.
    • ExTester Runner: A VS Code extension that provides a UI to list, run, and debug your UI tests directly within the editor.

  7. Explore the structure of the ExTester Runner UI test fixture

    main

    The fixture contains several types of test files designed to exercise specific features of the ExTester Runner UI:

    • Test Tree Parsing: Files in src/parser/*.test.ts use various Mocha shapes (simple tests, .only/.skip modifiers, variables, multiple root describe blocks, and deep nesting) to test the parser.
    • UI Views (Screenshots and Logs): Files in src/actions/*.test.ts are designed to fail intentionally, triggering screenshots and log production to verify the Screenshots and Logs views in the extension.

    Note on Execution: You cannot run these tests using a standard npm test command. They are intended to be opened as a workspace and executed by the ExTester Runner extension during its own UI test suite.

  8. How Locale Testing works in ExTester

    main

    ExTester allows you to launch a test VS Code instance in a non-English display language to verify that your extension's UI strings, commands, and interactions behave correctly when VS Code is localized.

    Localization is achieved by managing two files in the VS Code user-data directory:

    1. User/locale.json: Declares the target locale (e.g., { "locale": "ru", "osLocale": "ru" }). ExTester writes this automatically when you use the --locale / -L flag.
    2. languagepacks.json: A registry mapping locale codes to installed language pack extensions. This file is generated by the VS Code CLI when a language pack is installed into the test instance.

    Crucial Requirement: You must install the language pack into the test instance (via -i, install-from-marketplace, or the programmatic API) rather than using a system code binary. ExTester does not generate languagepacks.json itself; it relies on the VS Code CLI to do so after the extension is installed.

  9. Set and get values for different setting types

    main

    Most settings support generic value manipulation via getValue() and setValue(value). The behavior depends on the underlying UI type:

    TypeDescription
    Text boxAccepts arbitrary strings.
    Combo boxAccepts values from a predefined range. Cast to ComboSetting to use getValues() to see available options.
    Check boxAccepts boolean values.
    LinkDoes not hold a value. getValue and setValue will throw an error. Cast to LinkSetting and use openLink() to open the associated settings.json file.
    ArraySupports arrays of strings. Cast to ArraySetting to manage individual rows.
    // generic value retrieval
const value = await setting.getValue();

// generic setting of a value
await setting.setValue("off");

  10. Understand ExTester VS Code support policy

    main

    ExTester guarantees support for the latest 3 minor releases of VS Code. These versions are actively tested and bugs are fixed for them.

    Support levels for other versions:

    • Versions $\ge$ 1.90.0 but outside the latest 3 minor releases: Provided on a best-effort basis. They will likely work, but bugs will not be fixed.
    • Versions older than 1.90.0: Not supported. ExTester will fail fast with a clear error because these versions use ChromeDriver versions that are no longer provisionable via the Chrome for Testing infrastructure.

  11. Manage multiple Editor Groups

    main

    VS Code organizes editors into groups. You can target a specific group using a zero-based index (0 is the leftmost group) in EditorView methods, or you can obtain an EditorGroup handle to perform actions directly on that group.

    // Target the second group (index 1) directly via EditorView
const editor = await editorView.openEditor("package.json", 1);
await editorView.closeEditor("package.json", 1);
await editorView.closeAllEditors(1);
const titles = await editorView.getOpenEditorTitles(1);

// Get a handle for a specific group to use its dedicated API
const group = await editorView.getEditorGroup(1);

// Get handles for all editor groups
const groups = await editorView.getEditorGroups();

  12. Find and manage items in an ExtensionsViewSection

    main

    Unlike standard tree sections, ExtensionsViewSection uses the search bar at the top of the view to locate items. This allows you to find extensions even if they are not currently visible in the specific section you are looking at (e.g., searching the Marketplace).

    Important: When you use findItem, the search bar is populated. You must call clearSearch() to clear the search bar and return the view to its original state.

    // get all visible items inside the section
const items = await section.getVisibleItems();

// find an extension anywhere (including the marketplace)
const item = await section.findItem("npm");
// clear the search bar so the original section reappears
await section.clearSearch();

// find an extension in the installed section
const item2 = await section.findItem("@installed java");
// clear the search bar so the original section reappears
await section.clearSearch();

// open an item in the editor view
await section.openItem("@installed java");