syft

Syft is a CLI tool and Go library used to generate Software Bill of Materials (SBOMs).

Key capabilities include:

• Scan Targets: Supports container images (OCI, Docker, Singularity), filesystems, and archives.
• Ecosystem Support: Detects dozens of packaging ecosystems including Alpine (apk), Debian (dpkg), RPM, Go, Python, Java, JavaScript, Ruby, Rust, PHP, and .NET.
• Output Formats: Supports CycloneDX, SPDX, and Syft JSON.
• Security Integration: Designed to work with Grype for vulnerability scanning.
• Attestations: Can create signed SBOM attestations using the in-toto specification.

Syft generates Common Platform Enumeration (CPE) identifiers to enable vulnerability matching (e.g., with Grype). It uses a two-tier approach:

1. Dictionary Lookups (Authoritative): Uses pre-validated CPEs from the official NIST CPE dictionary. This provides high accuracy for ~22K entries across 11 ecosystems (including npm, RubyGems, PyPI, Jenkins Plugins, crates.io, PHP, Go Modules, and WordPress).
2. Heuristic Generation (Fallback): For packages not in the dictionary, Syft uses ecosystem-specific logic (e.g., extracting vendor from Java groupId or parsing Go module paths) to generate candidate CPEs. This ensures broad coverage for all discovered package types.

Syft uses a multi-phase generation process to automatically build and maintain capabilities.yaml files for its catalogers. This ensures that cataloger metadata, configuration options, and detection methods are accurately documented and synchronized with the source code.

The process follows four main stages:

1. Discovery Phase: Parses source code (AST) to find parsers, detectors, configuration structs, and application-level configurations.
2. Merge Phase: Loads existing capabilities.yaml files, updates AUTO-GENERATED fields while preserving MANUAL fields, and detects orphaned entries.
3. Write Phase: Groups catalogers by ecosystem and writes updated YAML files to syft/pkg/cataloger/*/capabilities.yaml and appconfig.yaml.
4. Validation Phase: Verifies cataloger presence, metadata/package type coverage, and runs completeness tests.

Depending on what you are changing, the workflow differs:

Changing Parser Detection Patterns

Detection patterns are AUTO-GENERATED.

1. Change the code.
2. Run go generate ./internal/capabilities.
3. Review changes in capabilities.yaml via git diff.

Changing Metadata Types

Metadata types are AUTO-GENERATED via test observations.

1. Change the code.
2. Update tests if necessary.
3. Run tests to update observations: go test ./syft/pkg/cataloger/something.
4. Run go generate ./internal/capabilities.

Changing Capabilities

Capabilities are MANUAL and preserved during regeneration.

1. Edit the capabilities.yaml file directly.
2. Validate with SYFT_ENABLE_COMPLETENESS_TESTS=true go test ./internal/capabilities/....

To add support for a new ecosystem in the dictionary lookup process:

1. Add the appropriate URL pattern in dictionary/index-generator/generate.go.
2. Regenerate the index using make generate:cpe-index.

To improve heuristic (fallback) generation:

1. Modify the specific ecosystem logic file (e.g., java.go or python.go).
2. Add curated mappings to candidate_by_package_type.go to handle specific vendor/product name translations.

Because the AST parser cannot determine the specific metadata or package types a parser produces, Syft relies on test-observations.json files. These files are automatically generated by pkgtest.CatalogTester helpers during test execution.

To document a parser, ensure your tests use the pkgtest helpers. The resulting test-observations.json (located in syft/pkg/cataloger/*/testdata/) will record the mapping between parsers and their produced types.

Example JSON structure in test-observations.json:

{
  "package": "golang",
  "parsers": {
    "parseGoMod": {
      "metadata_types": ["pkg.GolangModuleEntry"],
      "package_types": ["go-module"]
    }
  },
  "catalogers": {
    "linux-kernel-cataloger": {
      "metadata_types": ["pkg.LinuxKernel"],
      "package_types": ["linux-kernel"]
    }
  }
}

If you have modified the data models (such as adding a new pkg.*Metadata type), you must generate a new JSON schema. Run the following command from the root of the repository:

make generate-json-schema

Behavior of the command:

• If no schema exists for the current version, it creates schema/json/schema-$VERSION.json.
• If a schema exists and matches the new model, no action is taken.
• If a schema exists but does not match the new model, the command will error, indicating you must increment the version in internal/constants.go.

CRITICAL: Never delete or modify an existing JSON schema once it has been published in a release. Always create a new schema file with an incremented version number.

If you have made changes to cataloger code, you must regenerate the ecosystem capability files to ensure the documentation stays in sync with the codebase. Use the following command:

make generate-capabilities

This process:

1. Runs unit tests under ./syft/pkg... to gather up-to-date cataloger behavior.
2. Regenerates code describing cataloger capabilities.
3. Runs completeness tests to ensure all capability YAML claims are consistent with test observations.

Note: If completeness tests fail after regeneration, it means you must manually update the capability YAML files to reflect the new behavior observed in tests.

When introducing a new pkg.*Metadata type assigned to the pkg.Package.Metadata struct field, follow these steps to ensure schema integrity:

1. Add an integration test: Create a new test case in cmd/syft/internal/test/integration/catalog_packages_cases_test.go that exercises the new package type with its corresponding metadata.
2. Update the version: Increment the JSONSchemaVersion in internal/constants.go using the MODEL.REVISION.ADDITION rules.
3. Regenerate the schema: Run make generate-json-schema to produce the updated schema file in schema/json/.