spdx-tools Python Library

repository·main·Indexed 42 days ago

https://github.com/spdx/tools-python

A Python library for parsing, validating, and creating SPDX (Software Package Data Exchange) documents. It supports Tag/Value, RDF, JSON, YAML, and XML formats, with internal data models for SPDX v2.2 and v2.3 and experimental support for SPDX 3.0. Includes the pyspdxtools CLI for document validation, format conversion, and relationship graph generation.

Tokens
7.9K
Snippets
26
Records
32
Agent score
54%

What's inside spdx-tools

  1. Work with the SPDX v2 Data Model

    main

    The spdx_tools.spdx.model package provides the internal data model for SPDX v2.2 and v2.3.

    Key Concepts

    • Document Class: The main entry point for an SPDX document.
    • Type Safety: Classes use @dataclass_with_properties. Type hints are enforced during initialization and property assignment. Setting a wrong type raises ConstructorTypeError or TypeError.
    • List Manipulation: To avoid circumventing type checking, do not use in-place methods like list.append(item). Instead, use concatenation: list = list + [item].
    • Deprecated Fields: documentDescribes and hasFiles are converted to relationships in the internal model and will not be written to output files.
  2. Experimental support for SPDX 3.0

    main

    SPDX 3.0 support is experimental, incomplete, and not recommended for production. It is contained in the spdx3 subpackage.

    Core Concepts

    • Elements: In v3, all entities (packages, files, snippets, relationships, etc.) are subclasses of the Element class.
    • Payloads: Elements intended for the same file are collected in a Payload, which is a dictionary mapping SpdxId to the Element.

    Key Tasks

    • Conversion: Convert v2.2/v2.3 documents to v3.0 using spdx_tools.spdx3.bump_from_spdx2.spdx_document.bump_spdx_document(). This returns a Payload containing an SpdxDocument and all constituent elements.
    • Serialization:
      • Use spdx_tools.spdx3.writer.json_ld.json_ld_writer to generate JSON-LD files.
      • Use spdx_tools.spdx3.writer.console.payload_writer for debug console output.
  3. Install spdx-tools

    main

    Install the library from PyPI using pip. It is recommended to use a virtual environment (venv).

    pip install spdx-tools==0.8.3

    To install with optional dependencies for graph generation (requires networkx and pygraphviz), use:

    pip install ".[graph_generation]"
    pip install spdx-tools==0.8.3
  4. Generate a JSON-LD context file manually

    main

    If you need to manually produce a JSON-LD context file and serialization example for SPDX 3.0, follow these steps:

    1. Parse the model: Run the spec parser to generate RDF output.
    2. Convert RDF to JSON-LD: Convert the resulting .ttl file to JSON-LD using a tool like ttl2jsonld.
    3. Generate context: Use owl_to_context.py to convert the OWL file into a context file.
    4. Deployment: Place the resulting context.json in src/spdx_tools/spdx3/writer/json_ld/ to use it with the library.
    # 1. Run the spec parser
    python spec-parser/main.py spdx-3-model/model parser_output
    
    # 2. (External step) Convert parser_output/rdf/spdx-model.ttl to JSON-LD
    
    # 3. Convert OWL to context
    # (Requires owl_to_context.py)
    
    # 4. Place context.json in src/spdx_tools/spdx3/writer/json_ld/
  5. Example: Edit an existing SPDX document

    main

    This example demonstrates how to load a document, modify its metadata, add a file and a relationship, validate the changes, and save the result.

    import logging
    from license_expression import get_spdx_licensing
    from spdx_tools.spdx.model import (Checksum, ChecksumAlgorithm, File,
                                       FileType, Relationship, RelationshipType)
    from spdx_tools.spdx.parser.parse_anything import parse_file
    from spdx_tools.spdx.validation.document_validator import validate_full_spdx_document
    from spdx_tools.spdx.writer.write_anything import write_file
    
    # read in an SPDX document from a file
    document = parse_file("spdx_document.json")
    
    # change the document's name
    document.creation_info.name = "new document name"
    
    # define a file and a DESCRIBES relationship between the file and the document
    checksum = Checksum(ChecksumAlgorithm.SHA1, "71c4025dd9897b364f3ebbb42c484ff43d00791c")
    
    file = File(name="./fileName.py", spdx_id="SPDXRef-File", checksums=[checksum],
                file_types=[FileType.TEXT],
                license_concluded=get_spdx_licensing().parse("MIT and GPL-2.0"),
                license_comment="licenseComment", copyright_text="copyrightText")
    
    relationship = Relationship("SPDXRef-DOCUMENT", RelationshipType.DESCRIBES, "SPDXRef-File")
    
    # add the file and the relationship to the document
    # (note that we do not use "document.files.append(file)" as that would circumvent the type checking)
    document.files = document.files + [file]
    document.relationships = document.relationships + [relationship]
    
    # validate the edited document and log the validation messages
    validation_messages = validate_full_spdx_document(document)
    for validation_message in validation_messages:
        logging.warning(validation_message.validation_message)
    
    # if there are no validation messages, the document is valid
    if not validation_messages:
        write_file(document, "new_spdx_document.rdf", validate=False)
  6. Parse, Validate, and Write SPDX documents in Python

    main

    Use the following functions from the spdx_tools library to manipulate SPDX documents programmatically.

    Parsing

    Use parse_file(file_name) from spdx_tools.spdx.parser.parse_anything. It returns a Document instance or raises SPDXParsingError if the file is invalid.

    Validating

    Use validate_full_spdx_document(document) from spdx_tools.spdx.validation.document_validator. It returns a list of ValidationMessage objects containing error descriptions and ValidationContext.

    Writing

    Use write_file(document, file_name) from spdx_tools.spdx.writer.write_anything. The format is determined by the filename extension. Validation is performed by default before writing. To skip validation, use write_file(document, file_name, validate=False).

    from spdx_tools.spdx.parser.parse_anything import parse_file
    from spdx_tools.spdx.validation.document_validator import validate_full_spdx_document
    from spdx_tools.spdx.writer.write_anything import write_file
    
    document = parse_file("document.json")
    errors = validate_full_spdx_document(document)
    if not errors:
        write_file(document, "document.rdf")
  7. Use the pyspdxtools CLI for parsing, validating, and converting

    main

    The pyspdxtools command-line interface allows you to interact with SPDX files. The input and output formats are automatically inferred from the file extensions (e.g., .json, .tag, .rdf, .yaml, .xml).

    Parsing and Validating

    To parse and validate a file: pyspdxtools -i <filename>

    Converting Formats

    To convert a file from one format to another: pyspdxtools -i <input_file> -o <output_file>

    To skip the validation process during conversion (use with caution): pyspdxtools -i <input_file> -o <output_file> --novalidation

    Generating Graphs (Optional)

    If networkx and pygraphviz are installed, you can generate a visual graph of the document structure: pyspdxtools -i <input_file> --graph -o <output_file> (The <output_file> should have a valid extension for pygraphviz, such as .png).

    pyspdxtools -i tests/spdx/data/SPDXJSONExample-v2.3.spdx.json -o output.tag
    pyspdxtools -i tests/spdx/data/SPDXJSONExample-v2.3.spdx.json --graph -o SPDXJSONExample-v2.3.spdx.png
  8. Represent an Actor in an SPDX document

    main

    The Actor class is used to represent entities involved in an SPDX document, such as people, organizations, or tools. It includes the entity's type, name, and an optional email address.

    ActorType

    An actor must be assigned one of the following ActorType values:

    • PERSON
    • ORGANIZATION
    • TOOL

    Serialization

    Calling to_serialized_string() or using str() on an Actor instance returns a formatted string representation used across all SPDX serialization formats. The format is: {ActorType.name.title()}: {name} ({email}) (where the email part is omitted if not provided).

    from spdx_tools.spdx.model.actor import Actor, ActorType
    
    # Create a Person actor
    person = Actor(actor_type=ActorType.PERSON, name="Jane Doe", email="jane@example.com")
    print(person.to_serialized_string())
    # Output: Person: Jane Doe (jane@example.com)
    
    # Create an Organization actor without an email
    org = Actor(actor_type=ActorType.ORGANIZATION, name="Acme Corp")
    print(org)
    # Output: Organization: Acme Corp
  9. Validate SPDX IDs with validate_spdx_id()

    main

    Use validate_spdx_id() to verify that an SPDX ID is syntactically correct and, optionally, exists within a specific Document.

    An SPDX ID can be a simple internal ID (e.g., SPDXRef-example) or a reference to an external document using a colon separator (e.g., DocumentRef-external:SPDXRef-internal).

    Parameters:

    • spdx_id (str): The ID to validate.
    • document (Document): The SPDX document to check against.
    • check_document (bool): If True, verifies if the ID exists anywhere in the document. Defaults to False.
    • check_files (bool): If True, verifies if the ID exists specifically within the document's files. Defaults to False.

    Returns: A List[str] containing validation error messages. An empty list indicates the ID is valid.

    from spdx_tools.spdx.validation.spdx_id_validators import validate_spdx_id
    
    # Example: Validating an internal ID exists in the document
    errors = validate_spdx_id("SPDXRef-File-1", document, check_document=True)
    if not errors:
        print("ID is valid and present")
    else:
        print(f"Validation errors: {errors}")
  10. Convert a datetime object to an SPDX-compliant ISO string

    main

    Use datetime_to_iso_string to convert a Python datetime object into a string formatted for SPDX documents.

    Behavior:

    • Timezone Handling: If the datetime object is timezone-aware, it is converted to UTC and then treated as a naive object for the final string.
    • Microsecond Stripping: SPDX does not support microseconds; this function automatically sets microseconds to 0 if they are present.
    • Format: The resulting string is an ISO-8601 representation suffixed with Z (e.g., YYYY-MM-DDTHH:MM:SSZ).
    from spdx_tools.spdx.datetime_conversions import datetime_to_iso_string
    from datetime import datetime, timezone
    
    # Example with microseconds and timezone
    date_obj = datetime(2023, 10, 27, 10, 30, 0, 123456, tzinfo=timezone.utc)
    
    spdx_string = datetime_to_iso_string(date_obj)
    # Output: '2023-10-27T10:30:00Z'