repository·main·Indexed 21 days ago
https://github.com/microsoft/vstestAn extensible test platform used to discover, execute, and report on tests across various frameworks using a pluggable adapter architecture. It provides a public API for custom adapters and includes several supporting libraries such as Microsoft.TestPlatform.ObjectModel for extension development, Microsoft.TestPlatform.TranslationLayer for programmatic interaction with vstest.console, and Microsoft.NET.Test.Sdk for .NET project integration.
vstest.console.dll, required to discover and execute tests from a command-line environment.VSTest (Visual Studio Test Platform) is an open and extensible platform designed to run tests, collect diagnostic data, and report results.
Key characteristics:
.vsix extensions or as NuGet packages.TestPlatform (also known as vstest) is a collection of libraries and tools designed to provide test execution capabilities for various consumers, including Visual Studio Test Explorer, dotnet test, Azure DevOps, Omnisharp, and Stryker.
While primarily focused on running C# tests, it is an extensible platform that supports additional providers and extensions. It serves as the underlying engine that orchestrates test discovery and execution.
This library provides utility helpers for developers building test adapters for the Visual Studio Test Platform. Key capabilities include:
TestIdProvider: Generates stable, unique test IDs.ManagedNameHelper: Facilitates conversion between MethodInfo and managed type/method name pairs.A DataCollector is a test platform extension used to monitor test runs. It can hook into specific test execution events to perform tasks like collecting code coverage or gathering logs when tests fail. These collected files are called Attachments and are attached to the test result report (.trx).
DataCollectors can respond to four specific events:
The test platform selects the appropriate Test Host by enumerating all available host providers and querying them with the current RunConfiguration.
Each provider must implement the ITestRunTimeProvider.CanExecuteCurrentRunConfiguration() method. The platform calls this method to determine if the provider is capable of launching a TestHost based on the specific criteria provided in the current run configuration.
Discovery is the process of finding tests within provided test sources. This workflow typically involves a Client (e.g., Visual Studio), a Runner (e.g., vstest.console.exe), and a Testhost (e.g., testhost.exe).
Key Workflow Steps:
Extensions.Initialize request to specify extensions (like test adapters) to be loaded.TestDiscovery.Start request to the Testhost.TestDiscovery.TestFound messages back to the Runner, which forwards them to the Client.TestDiscovery.Completed. The Runner then terminates the session.%% ./RFCs/0001-Test-Platform-Architecture.md#discovery
sequenceDiagram
participant c as Client<br>(Visual Studio)
participant r as Runner<br>(vstest.console.exe)
participant t as Testhost<br>(testhost.exe)
Note over c,r: Start session and negotiate version
c->>r: (optional) Extensions.Initialize
c->>+r: TestDiscovery.Start
r->>t: Run testhost -port Y
t->>t: Connect to port Y
t-->r: Runner detects connection
r->>t: ProtocolVersion
t->>r: ProtocolVersion
r->>t: (optional) TestDiscovery.Initialize
r->>+t: TestDiscovery.Start
t-->>r: TestDiscovery.TestFound
r-->>c: TestDiscovery.TestFound
t-->>r: TestDiscovery.TestFound
r->>t: TestDiscovery.Completed
t-->>r: Process exited
r->>-c: TestDiscovery.CompletedWhen a generic type is nested within another generic type, the numbering of generic parameters must account for the parameters of all containing types. Even if the language syntax hides them, the metadata representation includes them.
Example Logic:
If class A<T> contains class B<X>, then B effectively has two generic arguments: T and X.
A ManagedType: A1B ManagedType: A1+B`1B using T would be encoded as !0.B using X would be encoded as !1.// type A has arity 1, and one generic argument T
// ManagedType = "A`1"
public class A<T>
{
// type B has arity 1, but two generic arguments T and X
// ManagedType = "A`1+B`1"
public class B<X> {
// ManagedMethod = "Method(!0, !1)"
public void Method(T t, X x) {}
// ManagedMethod = "Method(!1)"
public void Method(X x) {}
// ManagedMethod = "Method(!0, !1, !!0)"
public void Method<U>(T t, X x, U u) {}
}
}The TestCase object underwent a significant structural change to improve performance and reduce payload size.
Properties array where all metadata (ID, FullyQualifiedName, ExecutorUri, etc.) is stored as Key/Value pairs.Id, FullyQualifiedName, DisplayName, ExecutorUri, and Source are top-level properties. The Properties array is now reserved exclusively for custom or non-built-in properties (such as Traits).// V7 Compact TestCase Example
{
"Id": "guid-here",
"FullyQualifiedName": "MyNamespace.MyTest",
"DisplayName": "MyTest",
"ExecutorUri": "executor://mstest",
"Source": "test.dll",
"CodeFilePath": null,
"LineNumber": -1,
"Properties": [
// Only custom/non-built-in properties (e.g. Traits)
]
}VSTest capabilities are split into two distinct NuGet packages depending on your platform requirements and the level of test execution support needed:
Use this package if you need a cross-platform runner (Windows, macOS, or Linux) for modern CI systems. It contains a subset of capabilities designed for portability.
vstest.console.exe, vstest.console.dllBlame and other cross-platform collectors.trx.Use this package if you are on Windows and require the full suite of Visual Studio test capabilities, including legacy support and built-in adapters.
Microsoft.TestPlatform.Portable plus:MSTest V1, Ordered Test, Generic Test, and Web test adapters.CodeCoverage, Fakes, TIA, video, and SysInfo.Arcade's template architecture uses a layered approach to support both standard and 1ES pipelines:
/templates or /templates-official): These are intermediate YAML files that act as entry points. They define the is1ESPipeline parameter (false for /templates, true for /templates-official) and redirect to the core logic./core-templates): This contains the actual base template logic used by both scenarios.core-templates that redirect back to specific logic files in either templates or templates-official when logic is dependent on the entry point.Key structural rules:
stages, jobs, and job levels are implemented as shims.steps and variables levels typically contain direct logic because they are too granular for shims.The Event Log DataCollector is a Windows-only tool that captures Windows Event Viewer logs during test execution. It saves logs into an Event Log.xml file, which is attached to the test result report (.trx).
It generates:
Event Log.xml for the entire test session.Event Log.xml files for each test case for granular analysis.This is particularly useful for remote testing scenarios where direct access to the machine's Event Viewer is unavailable.