MSTest and Microsoft.Testing.Platform

repository·main·Indexed 21 days ago

https://github.com/microsoft/testfx

A cross-platform testing framework for .NET applications featuring MSTest and Microsoft.Testing.Platform, a lightweight VSTest alternative. The repository includes documentation on data-driven testing, WinUI testing, and migrating from MSTest V1 to V2. It also provides guidance on hosting tests on browser-wasm using the BrowserPlayground sample, including configuration for .NET 10 SDK, headless Node.js execution, and managed C# debugging in the browser.

Tokens
113.9K
Snippets
253
Records
463
Agent score
70%

What's inside microsoft-testfx

  1. Overview of Microsoft.Testing.Platform and MSTest

    main

    This repository contains two primary components for .NET testing:

    1. Microsoft.Testing.Platform: A lightweight alternative to VSTest used for test execution.
    2. MSTest (Microsoft Test Framework): A fully supported, open-source, and cross-platform testing framework for .NET applications. It supports writing tests, using Test Explorer, creating test suites, and is compatible with .NET Framework, .NET Core, .NET, UWP, and WinUI on Windows, Linux, and Mac.
  2. What is Microsoft.Testing.Extensions.Telemetry?

    main

    This is an optional, opt-in extension for Microsoft.Testing.Platform. It collects usage telemetry to help the platform team understand product usage and prioritize improvements.

    Key features include:

    • Usage telemetry: Data collection for product improvement.
    • Opt-out support: Support for disabling telemetry via environment variables.
    • Disclosure: Telemetry information and opt-out guidance are displayed on the first run.
  3. What is Microsoft.Testing.Platform?

    main

    Microsoft.Testing.Platform is a lightweight, portable alternative to VSTest. It is designed to run tests in CLI environments, CI pipelines, Visual Studio Test Explorer, and Visual Studio Code Test Explorer.

    Unlike traditional testing models, the platform is embedded directly into your test projects, allowing test applications to be built as standalone executables that can be run directly. It provides three core capabilities:

    1. Test application host: Enables test projects to be built as executables.
    2. Extensibility model: Allows test frameworks, tools, and extensions to interoperate.
    3. Protocol: Implements the Microsoft Testing Platform protocol to enable communication between the test host and external consumers like IDEs or CI systems.
  4. What is Microsoft.Testing.Extensions.HotReload?

    main

    This extension for Microsoft.Testing.Platform allows you to apply code changes to your running tests without restarting the test host process. This creates a faster inner loop by reducing the time between making a code change and seeing the updated test results.

    Limitations & Requirements:

    • Mode Support: Currently only supported in console mode. It is not supported in Visual Studio or VS Code Test Explorer.
    • Dependency: Requires Microsoft.Testing.Platform.
  5. MSTest Source Generator Roadmap

    main

    The MSTest source generator is evolving to improve performance and support more complex test patterns. Key upcoming features and design goals include:

    • Inherited [TestClass] support: An opt-in attribute (name TBD) to allow derived classes to be included in the generator's discovery set without re-applying [TestClass].
    • Generated property descriptors: A design to replace reflection-based property access with generated descriptors that preserve declared-only/inherited lookup and visibility semantics.
    • Improved source-location data: Redesigning TypeMethodLocations to better handle overloads and end-to-end navigation.
    • Expanded reflection-free capabilities: Continuing to populate materializable type/method attributes and using generated delegates for constructors, methods, and property setters.
  6. What Microsoft.Testing.Platform.MSBuild provides

    main

    The Microsoft.Testing.Platform.MSBuild package provides essential MSBuild tasks for Microsoft.Testing.Platform (MTP) projects. Its core responsibilities include:

    • Entry-point generation: Automatically generates the required entry point for MTP-based test projects.
    • Configuration file support: Automatically copies testconfig.json from your project into the output directory, renaming it to $(AssemblyName).testconfig.json.
    • dotnet test compatibility: Enables you to run MTP-based test projects using the standard VSTest-based dotnet test command on .NET SDKs.
  7. Access MSTest documentation and API references

    main

    Official documentation for writing unit tests, using unit test frameworks, and integrating with the CLI or Visual Studio is hosted on the Microsoft Learn website. For detailed technical specifications of the API, refer to the Microsoft .NET API documentation.

  8. Understand the Microsoft.Testing.Platform JSON-RPC Protocol

    main

    The Microsoft.Testing.Platform protocol facilitates communication between a client (such as an IDE, CLI, or CI system) and a test runner executable (the server). The protocol is based on JSON-RPC and is used to manage the lifecycle of test execution, including discovery, running tests, and reporting results.

    This protocol is specifically for the JSON-RPC server-mode. For the binary dotnet test named-pipe protocol (used with --server dotnettestcli), refer to the specific documentation for that mode.

  9. What is Microsoft.Testing.Extensions.VSTestBridge?

    main

    The Microsoft.Testing.Extensions.VSTestBridge is an extension for Microsoft.Testing.Platform that provides a compatibility layer for VSTest.

    It is used by framework authors to achieve the following:

    • VSTest compatibility: Maintains support for existing VSTest-mode workflows, including vstest.console.exe, dotnet test, the VSTest task, and Visual Studio Test Explorer.
    • Migration path: Allows framework authors to gradually adopt Microsoft.Testing.Platform without breaking existing VSTest integrations.
    • Configuration support: Enables compatibility for VSTest .runsettings files and --filter arguments where supported by the framework.
  10. Best practices for choosing ResourceLock keys

    main

    When using [ResourceLock], follow these guidelines to balance safety and performance:

    1. Eliminate before you lock: If a resource can be made unique per test (e.g., using per-TFM output paths instead of shared bin/obj folders), do not use a lock.
    2. Granularity follows blast radius: Use a narrow key when you can enumerate the specific resources (e.g., AZURE_OPENAI_* variables). Use a coarse key for ambient state whose reach you cannot bound (e.g., PATH).
    3. Key the reads, not just the writes: Ensure the lock covers everything the code under test reads from the shared state.
    4. Refine via measurement: Start with coarse locks to ensure safety (over-locking fails slow/safely) and only refine to narrower keys once you have measured the performance impact (under-locking fails flaky/expensively).
  11. Protocol Versioning and Compatibility

    main

    The protocol supports versions 1.0.0 through 1.4.0.

    VersionKey Features
    1.0.0Base protocol. SDK suppresses reporter to avoid collision with host output.
    1.1.0Host no longer plugs in TerminalOutputDevice; SDK can safely keep TerminalTestReporter output on.
    1.2.0Adds AzureDevOpsLogMessage (ID 11).
    1.3.0Adds DisplayMessage (ID 12).
    1.4.0Adds server-control channel (IDs 13/14). Gated by ServerControlPipeName handshake property.

    Compatibility Rules:

    • Version Gating: Messages like AzureDevOpsLogMessage (1.2.0) and DisplayMessage (1.3.0) are only sent if the negotiated version supports them.
    • Field Extensibility: Unknown field IDs within a known message are skipped. Newer hosts can add fields without breaking older SDKs.
    • Serializer Extensibility: Unknown serializer IDs are not safe. New messages must be version-gated because receivers might not have the corresponding serializer.
  12. How soft assertions work with Assert.Scope()

    main

    The Assert.Scope() method enables soft assertions. Instead of throwing an exception immediately upon a failure, assertion failures are collected and reported together when the scope is disposed. This allows you to verify multiple properties or conditions in a single test block without the test stopping at the first failure.

    Important Behavior: When using Assert.Scope(), assertions do not enforce postconditions. This means that if an assertion like Assert.IsNotNull(obj) fails, the execution continues, and obj may still be null in the subsequent lines of code. While the compiler may treat obj as non-null due to annotations, you should not rely on this within a scope to avoid NullReferenceException at runtime.

    using (Assert.Scope())
    {
        Assert.AreEqual(1, actual.X);  // failure collected, execution continues
        Assert.AreEqual(2, actual.Y);  // failure collected, execution continues
        Assert.IsTrue(actual.IsValid); // failure collected, execution continues
    }
    // Dispose() throws AggregateException-like AssertFailedException with all 3 failures