Spectator OpenAPI Contract Testing for Laravel

repository·master·Indexed 43 days ago

https://github.com/hotmeteor/spectator

A lightweight contract testing tool for Laravel that ensures API implementations strictly adhere to OpenAPI specifications. It extends Laravel's TestResponse with assertions like assertValidRequest() and assertValidResponse(), provides CLI tools for linting and auditing specs (spectator:validate, spectator:coverage, spectator:routes), and includes a PHPUnit 11 extension for tracking contract coverage.

Tokens
8.1K
Snippets
31
Records
38
Agent score
54%

What's inside Spectator

  1. How Spectator works internally

    master

    Spectator operates by registering a middleware that intercepts every test request. The process follows these steps:

    1. Intercept: The middleware catches the request.
    2. Match: It matches the request against the loaded OpenAPI spec's PathItem.
    3. Validate: It validates both the request and the response against the specification.
    4. Capture: Any captured exceptions are stored on the RequestFactory singleton, allowing assertions to read them after the response has been returned.

    Key Dependencies

    • cebe/php-openapi: Used to parse OpenAPI 3.x specs into typed objects.
    • opis/json-schema: Used to validate request/response data against JSON Schema.
  2. How to write contract tests with Spectator

    master

    Contract tests verify that your API requests and responses conform to your OpenAPI spec. Unlike functional tests, they focus on the shape of the data rather than the business logic.

    1. Point to a spec

    Use Spectator::using('filename.yml') to specify which spec to validate against. This can be called in setUp() or within individual tests.

    2. Make assertions

    Spectator extends Laravel's TestResponse with several assertion methods:

    • assertValidRequest(): Request matches the spec.
    • assertInvalidRequest(): Request does not match the spec.
    • assertValidResponse(?int $status): Response matches the spec (optionally checking status code).
    • assertInvalidResponse(?int $status): Response does not match the spec.
    • assertValidationMessage(string $message): Checks if a validation error contains a specific string.
    • assertErrorsContain(string|array $errors): Checks for specific strings in validation errors.
    • assertPathExists(): Checks if the requested path is in the spec.
    • dumpSpecErrors(): Dumps errors without failing the test (useful for debugging).

    3. Resetting

    To stop using the current spec in a test, call Spectator::reset();.

    use Spectator//
    
    class UserApiTest extends TestCase
    {
        protected function setUp(): void
        {
            parent::setUp();
            Spectator::using('Api.v1.yml');
        }
    
        #[Test]
        public function test_create_user(): void
        {
            $this->postJson('/users', ['name' => 'Alice', 'email' => 'alice@example.com'])
                ->assertValidRequest()
                ->assertValidResponse(201);
        }
    }
  3. Install Spectator via Composer

    master

    Install Spectator as a development dependency in your Laravel project using Composer, then publish the configuration file to config/spectator.php.

    composer require hotmeteor/spectator --dev
    
    php artisan vendor:publish --provider="Spectator\SpectatorServiceProvider"
  4. Validate OpenAPI specs in CI

    master

    To catch malformed OpenAPI specifications before running your test suite, add the spectator:validate command as an early step in your CI pipeline. This command checks the validity of your spec files.

    Use the --spec flag to specify the file and the --format flag to define the output format (e.g., json).

    # GitHub Actions example
    - name: Validate OpenAPI spec
      run: php artisan spectator:validate --spec=Api.v1.yml --format=json
  5. Upgrade Spectator from v1 to v2

    master

    Upgrading from v1 to v2 involves several breaking changes regarding environment requirements, configuration, and assertion behavior.

    Minimum Requirements

    • PHP: 8.1+
    • Laravel: 10+

    Assertion Behavior Changes

    • $response->assertValidRequest(): Now fails if the OpenAPI spec is missing or invalid.
    • $response->assertInvalidRequest() and $response->assertInvalidResponse(): Now fail if the spec is missing or invalid (previously they passed if the spec was missing).
    • $response->assertInvalidRequest() and $response->assertInvalidResponse(): Now fail if called on a perfectly valid HTTP call (previously they did not fail).
    • HTTP Code Prioritization: When passing a code to $response->assertValidResponse($code) or $response->assertInvalidResponse($code), Spectator now prioritizes validating the actual HTTP code over the specification validation.

    OpenAPI Specification Changes

    • Charset Handling: Spectator now ignores the charset definition in the Content-Type header. You must remove charset parts (e.g., ; charset=utf-8) from your OpenAPI specification.
    • Strict Validation: Validation is stricter. A non-empty response body is now considered invalid if the specification does not define a response content.

    Configuration

    • The configuration format has changed; verify your config/spectator.php against the latest version.
    /users:
      get:
        summary: Get users
        tags: []
        responses:
          '200':
            description: OK
            content:
    -         application/json; charset=utf-8:
    +         application/json:
                schema:
                  type: object
                  properties:
                    id:
                      type: integer
  6. Debug Spectator validation errors

    master

    When a validation fails, Spectator provides an annotated schema view.

    Symbol Legend:

    • ++ — object allows additionalProperties
    • * — property is required
    • ? — property is nullable

    To inspect errors without failing the test execution, use dumpSpecErrors() on the response object.

    $this->postJson('/users', $payload)
        ->dumpSpecErrors()
        ->assertValidRequest();
  7. Upgrade Spectator from v2 to v3

    master

    When upgrading to v3, ensure your environment meets the new minimum requirements and update your configuration to include the new error_format key.

    Minimum Requirements

    • PHP: 8.3+
    • Laravel: 12+

    Configuration Update

    If you have a published config/spectator.php file, you must manually add the error_format key to prevent configuration errors:

    'error_format' => env('SPECTATOR_ERROR_FORMAT', 'text'),

    By default, the format is text. You can change this to json via the environment variable SPECTATOR_ERROR_FORMAT=json or by calling Spectator::useJsonErrors().

  8. Track contract coverage with SpectatorExtension

    master

    Spectator provides a PHPUnit 11 extension, Spectator\Coverage\SpectatorExtension, which tracks which operations in your OpenAPI spec are exercised during a test run. It prints a coverage summary at the end of the suite.

    Configuration

    Register the extension in your phpunit.xml file. You can configure the following parameters:

    • min_coverage: A numeric value (e.g., 80). If the actual coverage is below this value, the extension causes PHPUnit to exit with code 1, failing the CI job.
    • format: The output format for the summary. Options are json or text (default is text).
    <extensions>
        <bootstrap class="Spectator\Coverage\SpectatorExtension">
            <!-- Fail the suite if coverage drops below 80% -->
            <parameter name="min_coverage" value="80"/>
            <!-- Optional: json | text (default: text) -->
            <parameter name="format" value="text"/>
        </bootstrap>
    </extensions>
  9. Configure Path Prefix and Error Format

    master

    Path Prefix

    If your API is mounted under a prefix (e.g., /v1), tell Spectator so it can strip the prefix before matching spec paths. You can set this in .env or at runtime.

    • Env: SPECTATOR_PATH_PREFIX=v1
    • Runtime: Spectator::withPathPrefix('v1');

    Error Format

    Switch to JSON output for failed assertions (useful for CI/CD or LLM tools) instead of the default human-readable terminal output.

    • Env: SPECTATOR_ERROR_FORMAT=json
    • Runtime: Spectator::useJsonErrors(); (to enable) or Spectator::useTextErrors(); (to revert to text).
  10. Configure machine-readable JSON error output

    master

    You can configure Spectator to output validation errors in JSON format instead of ANSI-coloured text. This is useful for log aggregators or feeding errors directly into LLM workflows for automated root-cause analysis or spec repair.

    Set the SPECTATOR_ERROR_FORMAT environment variable to json.

    SPECTATOR_ERROR_FORMAT=json
  11. How Spectator Middleware intercepts requests and responses

    master

    The Spectator\Middleware class is responsible for intercepting incoming HTTP requests and outgoing responses to validate them against an OpenAPI specification.

    The Lifecycle of a Spectator Request:

    1. Spec Check: If no specification is provided via the RequestFactory, the middleware passes the request through without intervention.
    2. Path Resolution: The middleware attempts to match the current route and HTTP method against the OpenAPI paths. It supports exact matches and regex-based partial matches (e.g., handling route parameters like {id}).
    3. Request Validation: Before the application processes the request, the middleware validates the incoming Request against the PathItem in the spec. If validation fails, a RequestValidationException is captured.
    4. Response Validation: After the application processes the request and generates a response, the middleware validates the Response against the corresponding operation in the spec. If validation fails, a ResponseValidationException is captured.
    5. Coverage Tracking: The middleware automatically records hits to specific paths and methods in the CoverageTracker to help identify which parts of your OpenAPI spec are being exercised by your tests.

    Error Handling: If a validation error occurs and the exception is reportable by the application's ExceptionHandler, the middleware returns a JSON response containing the error details. If the exception implements a getErrors() method, those errors are included in the response under the specErrors key.

  12. Configure Spectator spec sources

    master

    Spectator uses the SPEC_SOURCE configuration to determine where to find your OpenAPI spec files. You can configure this in your .env file or config/spectator.php using one of three modes:

    Local

    Reads specs from the local filesystem.

    • SPEC_SOURCE=local
    • SPEC_PATH=/path/to/specs

    Remote

    Fetches specs over HTTP (e.g., raw GitHub URLs).

    • SPEC_SOURCE=remote
    • SPEC_PATH=https://raw.githubusercontent.com/org/repo/main/specs
    • SPEC_URL_PARAMS="?token=abc123" (optional)

    GitHub

    Fetches specs from a private GitHub repository using a Personal Access Token.

    • SPEC_SOURCE=github
    • SPEC_GITHUB_REPO=org/repo
    • SPEC_GITHUB_PATH=main/specs (branch + path to directory)
    • SPEC_GITHUB_TOKEN=ghp_yourtoken
    # Example Local Config
    SPEC_SOURCE=local
    SPEC_PATH=/path/to/specs