DOCSEARCHbyAlgolia
Submit Docs

Swift Testing Documentation

repository·main·Indexed 24 days ago

https://github.com/swiftlang/swift-testing

A modern, expressive testing framework for Swift featuring macros for intuitive APIs, parameterized testing, and advanced organization via traits and tags. It supports Apple platforms, Linux, FreeBSD, Windows, and experimental support for Wasm and Android. The framework is designed to work side-by-side with XCTest and provides a stable ABI and JSON specification for tool integration, including a JSON event stream for custom test runners and IDE plugins.

Tokens
14.4K
Snippets
36
Records
71
Agent score
80%

What's inside Swift Testing

  1. Customize tests with Traits

    main

    Traits allow you to describe runtime conditions or execution constraints for tests or test suites. You can use traits to:

    • Enable or disable tests based on conditions (e.g., .enabled(if: ...)).
    • Specify execution time limits.
    • Define requirements for specific operating systems or devices.
    @Test(.enabled(if: AppFeatures.isCommentingEnabled))
func videoCommenting() async throws {
    let video = try #require(await videoLibrary.video(named: "A Beach"))
    #expect(video.comments.contains("So picturesque!"))
}

  2. Identify tooling-only APIs with @_spi(ForToolsIntegrationOnly)

    main

    Interfaces marked with @_spi(ForToolsIntegrationOnly) are intended for external tools (such as Swift Package Manager) that integrate with the testing library.

    Test authors should avoid using these interfaces unless they are specifically building tooling around Swift Testing. While breaking changes in this group are typically preceded by deprecation to allow tool authors time to migrate, stability is not guaranteed.

  3. Understand SPI groups in Swift Testing

    main

    Swift Testing uses System Programming Interfaces (SPI) to categorize non-public APIs. There are three distinct categories of SPI that you may encounter:

    1. Tooling Integration: Interfaces intended for tools (like Swift Package Manager) rather than test authors. These are marked with @_spi(ForToolsIntegrationOnly).
    2. Experimental Features: Interfaces available for test authors but currently under active development. These are marked with @_spi(Experimental). They may be modified or removed before becoming part of the stable public API.
    3. Internal Shared Interfaces: Private interfaces shared across targets that cannot use the package access level for technical reasons.

    If an interface is both experimental and intended for tools, it will be marked with both @_spi(ForToolsIntegrationOnly) and @_spi(Experimental).

  4. Implement a TestContentRecord accessor function

    main

    The accessor is a C function used to initialize memory with the relevant test content.

    Behavioral Requirements:

    • Success: Returns true and initializes the memory at outValue to an instance of the appropriate type. The caller is responsible for deinitializing this memory.
    • Failure: Returns false and leaves outValue uninitialized.
    • Type Safety: The type argument is a pointer to the expected type. If the type does not match the record's expected type, the accessor must return false and must not modify outValue.
    • Hinting: The hint argument is an optional input. If nil is passed, the accessor should behave as if it matched.
    • Reserved: The reserved argument must be assumed to be 0 and must not be accessed.

    Safety Warning: When calling accessors, use withUnsafeTemporaryAllocation(of:capacity:_:) or withUnsafePointer(to:_:) to ensure pointers are correctly sized and aligned.

  5. Understand the Swift Testing `ABI` namespace

    main
    The ABI namespace contains types and constants related to Swift Testing's ABI-stable and -semi-stable interfaces. These symbols are marked @_spi(ForToolsIntegrationOnly), meaning they are intended for tools authors (e.g., IDE plugins, CI reporters) rather than typical test authors. They are primarily used to integrate external tools with Swift Testing's runtime output.

  7. Understand Swift Testing's ABI and JSON specification

    main

    Swift Testing provides stable interfaces (ABI) for tools to interact with the testing framework. Key components include:

    • JSON Event Stream: Use the SPI (System Programming Interface) described in ABI/EventStreamHandling.md to work with the library's JSON event stream.
    • JSON Specification: The ABI/JSON.md file contains the official JSON specification used by tools to interact with Swift Testing directly or via the swift test command-line tool.
    • Test Content: The ABI/TestContent.md documents the section emitted by the Swift compiler into test products, which contains test definitions and metadata used by Swift Testing and potentially other third-party testing libraries.

  8. Understand how #expect() expands in Swift Testing

    main

    The #expect() macro undergoes a transformation during compilation to capture the expression's structure. In recent versions, it has moved from using Testing.__checkBinaryOperation to Testing.__checkCondition.

    This transformation uses a Testing.__ExpectationContext (represented as __ec) to wrap sub-expressions. This allows the testing framework to uniquely identify and reconstruct the syntax tree of the expression when a test fails, providing better diagnostic information. The hexadecimal integer literals used in the expansion represent unique identifiers for captured syntax nodes from the original Abstract Syntax Tree (AST).

  9. Naming conventions for Swift symbols

    main

    When contributing to the testing library, follow these naming rules for Swift symbols:

    • Public interface symbols: Follow the Swift API Design Guidelines.
    • Internal-use public symbols: If a symbol must be public for technical reasons but is not part of the intended public API, prefix it with two leading underscores (e.g., public func __check()).
    • Private symbols: Prefix private symbols with a single leading underscore (e.g., private var _errorCount: Int).
    • Storage for high-visibility symbols: Use a single underscore for private storage variables if the preferred name is used by a public property (e.g., _errorCount for errorCount).
    • Explicit types: Properties, variables, and constants with public access (or @usableFromInline) must have an explicitly specified type, even if the type can be inferred from an initialization expression.

  10. Work with `ABI.Version` and `ABI.VersionNumber`

    main

    Swift Testing uses the ABI.Version protocol to define specific ABI versions. Each version is tied to a Swift toolchain release (e.g., ABI.v6_3 for Swift 6.3).

    • ABI.Version: A protocol implemented by types representing a specific ABI version.
    • ABI.VersionNumber: A Comparable and Codable value representing the version. You can use the versionNumber property to compare versions.
    • ABI.ExperimentalVersion: Represents experimental and unsupported ABI variants.

    Note: Patch releases (e.g., 6.3.1) typically share the same ABI version as their minor release (6.3.0).

    let abi: (some ABI.Version).Type 
let isNewerThan6_3 = abi.versionNumber > ABI.v6_3.versionNumber

  11. How #expect() and #require() capture values

    main

    The #expect() and #require() macros are expression macros that attempt to expand the AST (Abstract Syntax Tree) of their condition argument. Instead of treating the condition as a simple boolean, they look for specific patterns like binary operators, member function calls, or is/as? casts.

    When an expectation fails, this expansion allows Swift Testing to provide detailed diagnostics by showing the individual values of the subexpressions involved in the failure, rather than just the final boolean result.