DOCSEARCHbyAlgolia
Submit Docs

Testspace Documentation

website·Indexed 33 days ago

https://help.testspace.com/

Documentation for Testspace, a platform for publishing and tracking automated test results and manual testing specs. Includes guides on the Testspace Client CLI, REST API, and integration with CI/CD systems like Azure Pipelines and Jenkins. Supports various test output formats such as JUnit, NUnit, Visual Studio TRX, and xUnit.net, as well as code coverage formats like Clover and Cobertura.

Tokens
45.8K
Snippets
236
Records
416
Agent score
24%

What's inside Testspace

  1. Overview of Testspace Client CLI

    The Testspace Client is a lightweight command-line tool for Windows, Linux, and macOS used to push test results to Testspace. It can be used interactively or within automated build/test pipelines. The tool supports environment variable assignments directly in the command line using NAME=VALUE syntax.
    # General syntax
testspace [--version] [--help] [<command>] [<args>]

# Example: Passing an environment variable during execution
testspace "mytests.xml" "my-space" NAME=VALUE

  2. Overview of Testspace Test Management

    Testspace is a test management platform designed for Automated, Manual, and Exploratory testing. It provides a dashboard for publishing automation results, a framework for executing manual tests authored in Markdown, and built-in support for capturing observations during exploratory testing. Key features include flaky test detection, integration with GitHub, GitLab, Bitbucket, and Jira for issue tracking, and built-in analytics and reporting.

  3. Overview of Testspace Issues Providers

    The Issues provider determines where Testspace creates and references issues for both automated test result annotations and manual test-case failures.

    Project Types and Configuration:

    • Connected Projects: Default to the service (GitHub, GitLab, or Bitbucket) the repository is connected to. GitHub can be overridden to a different org/repo, or changed to Jira/Generic via .testspace.yml using issues.provider.
    • Standalone Projects: Configured via the 'Issues provider' dropdown. Only Generic, GitHub, and Jira are available.

    Provider Capabilities:

    • GitHub, Jira, GitLab: Support issue creation, posting follow-up comments (one-way from Testspace to provider), status auto-sync, and labels.
    • Bitbucket: Supports issue creation, follow-up comments, and status auto-sync, but does not support labels.
    • Generic: Reference-only. Does not support creation, comments, sync, or labels.

  4. Overview of Testspace Includes

    An include is a reusable markdown fragment that can be embedded into a specification or another include using the {% include %} tag. Testspace supports two modes for managing includes: Standalone and Connected.

  5. Overview of Project Notes

    Notes are a Project-level feature used to share information and status with team members. A single Notes collection is shared across all Spaces within a Project. Notes are persisted and searchable. The Notes Summary view provides metadata for each note, including the owner, the number of outstanding to-do items, file attachment counts, and comment counts.

  6. Testspace publishing capabilities overview

    Testspace provides a comprehensive system for collecting and analyzing software quality metrics. Key capabilities include:

    • Support for branch-based workflows (forks, pull requests).
    • Compatibility with CI matrices and parallel jobs.
    • Support for industry-standard test output (JUnit, NUnit, TRX).
    • Publishing of code coverage and other artifacts.
    • Ability to publish derived Custom Metrics (e.g., timing, error logs).
    • Inclusion of external content such as build machine links and log annotations.
    • Automatic organization of results based on folder and source directory structure.

  7. Configure repository and build metadata for Testspace push

    When pushing content using the Testspace Client, you can provide metadata about the repository, build status, and external service links using the following options:

    • --repo <type[:path]>: Specifies the repository type (git, svn, or p4) to collect commit details and generate a code change report. If a path is provided, it specifies the local repository location; otherwise, the current directory is used.
      • For git, the path is equivalent to the --git-dir option or the GIT_DIR environment variable.
      • For p4, an active login ticket and the P4PORT and P4CLIENT environment variables must be configured.
    • --link <[token@]type>: Links an external service. Supported types are codecov or coveralls. An optional API access token can be prefixed (e.g., token@codecov).
    • --build-url <url>: The URL of the build that produced the content.
    • --build-status <status>: The status of the build. Allowed values: success, failure, or error.
    • --message <text>: A textual description to associate with the pushed content.
    # Example: Push with git repo, build status, and a message
testspace push --repo git:/path/to/repo --build-status success --message "Production build"

# Example: Link to Codecov with a token
testspace push --link mytoken@codecov

  8. Create discussions and comments for test results

    Testspace allows users to initiate discussions about specific result sets by creating a Note. This can be done in two ways:

    1. Via GUI: In the Results view, click the hamburger menu on a result row and select 'Comment'. Enter your text in the dialog to create a Note pre-populated with result summaries.
    2. Via Email: If subscribed to publication notifications, replying directly to a result publication email will automatically generate a Comment and a corresponding Note in Testspace.

  9. Understand Release vs Sandbox spaces in Connected projects

    In Connected projects, Testspace maps spaces to repository branches. Spaces are automatically created and deleted based on repository activity. There are two types of spaces:

    • Release spaces: Used for test execution. The repository's default branch (e.g., main) is always a release space.
    • Sandbox spaces: Used for test development. These spaces do not persist spec runs. All non-default branches are sandbox spaces unless explicitly configured as release spaces.

  10. Integrate Version Control Services with Testspace

    Testspace provides direct integration with major version control services, allowing it to function either as a standalone dashboard for proprietary automation systems or as an integrated tool. Supported services include GitHub, GitLab, and Bitbucket.

  11. Understand the Testspace Space Results dashboard

    A Space in Testspace is used to collect and track test results from automated tests (via the Testspace client) and manual testing sessions. The Current tab provides a snapshot of the most recent completed Result record, displaying a hierarchy of folders and suites.

    Key dashboard elements include:

    • Metric Badges: Enabled quality metrics.
    • Duration: Total test time and time since completion.
    • Build Link: Direct link to the associated CI job.
    • Failures: Filtered view of failing tests.
    • Show tree: Quick navigation of the test hierarchy.
    • Annotations: Supplemental data associated with the Result.

    Folders and Suites provide roll-up totals for passing/failing suites, passing/failing test cases, failure counts (new, tracked, resolved), and total duration.

  12. Understand Testspace Space Insights indicators

    Testspace analyzes testing data to generate high-level Insights viewable in the Indicators and Metrics sections. These indicators assess workflow performance based on four status levels: Undetermined (insufficient data), Good (meeting/exceeding expectations), Fair (not meeting highest performance), and Poor (requires review). The primary indicators are:

    • Results Strength: Measures stability of results and infrastructure.
    • Test Effectiveness: Measures if tests effectively capture side-effects.
    • Workflow Efficiency: Measures how quickly and efficiently failures are resolved.