InputStream Adaptive

repository·Piers·Indexed 19 days ago

https://github.com/xbmc/inputstream.adaptive

A project part of Kodi featuring a C++11-based incremental WebM parser and DRM support via CdmAdapter. The WebM parser supports recursive elements, arbitrary seeking, and a producer-consumer model using Reader and Callback interfaces. The CdmAdapter manages Content Decryption Module (CDM) sessions, including session lifecycle, decryption, and decoding.

Tokens
6.2K
Snippets
15
Records
23
Agent score
68%

What's inside xbmc-inputstream.adaptive

  1. Understand InputStream Adaptive's licensing rules

    Piers

    InputStream Adaptive is part of Kodi and is primarily provided under the GNU General Public License v2.0 or later (GPL-2.0-or-later).

    While the project as a whole is licensed under the main LICENSE file, individual source files may carry different licenses (including dual licenses like a GPL variant combined with a permissive license like BSD or MIT) as long as they remain compatible with GPL-2.0. All SPDX identifiers used must have a corresponding license file in the LICENSES subdirectory to support tool verification (e.g., ScanCode toolkit).

  2. How the WebM Parser works

    Piers

    The WebmParser follows a producer-consumer model involving three main components:

    1. WebmParser: The core engine that performs the parsing work.
    2. Reader: The data source. The parser requests data from a Reader instance.
    3. Callback: The event handler. As the parser identifies WebM elements, it notifies the Callback implementation.

    This architecture supports incremental parsing, meaning parsing can be stopped and resumed, and it supports arbitrary seeking, allowing you to start parsing from the middle of a file rather than just the beginning.

  3. Implement a custom Reader

    Piers

    The Reader interface acts as the data source for the parser. You can subclass Reader to implement your own data source.

    Provided Implementations

    If you do not want to implement a custom one, use these built-in blocking readers:

    • FileReader: Reads from a FILE*.
    • IstreamReader: Reads from a std::istream.
    • BufferReader: Reads from a std::vector<std::uint8_t>.

    Handling Partial Reads and EOF

    • Partial Reads: If Reader::Skip() or Reader::Read() return Status::kOkPartial, the parser will attempt to call them again.
    • Blocking/Waiting: If no data is available, return Status::kWouldBlock. The parser will stop and can be resumed later when more data is available.
    • End of File: When no more data is available, return Status::kEndOfFile.
      • If the file ended at a valid location, the parser returns Status::kOkCompleted.
      • If the file ended prematurely, the parser returns Status::kEndOfFile.
    • Post-EOF calls: Implementations should be able to handle being called multiple times after Status::kEndOfFile has been returned (e.g., by nested parsers terminating) and should consistently return Status::kEndOfFile in those cases.
  4. Build the WebM Parser

    Piers

    The WebM parser is not enabled by default when building libwebm. You must explicitly enable it using CMake or by compiling manually.

    Using CMake

    Enable the ENABLE_WEBM_PARSER feature in your interactive CMake builder, or pass the following flag via the command line:

    -DENABLE_WEBM_PARSER:BOOL=ON

    Manual Compilation (Static Library)

    To compile the code into a static library without CMake, use the following commands:

    c++ -Iinclude -I. -std=c++11 -c src/*.cc
    ar rcs libwebm.a *.o
    # Manual Compilation (Static Library)
    ```sh
    c++ -Iinclude -I. -std=c++11 -c src/*.cc
    ar rcs libwebm.a *.o
  5. Annotate source files with SPDX license identifiers

    Piers

    To ensure machine-parsable and precise license compliance, InputStream Adaptive requires the use of SPDX license identifiers in the top comment of every source file.

    Placement

    Identifiers must be added at the top of the file.

    Syntax

    Use an <SPDX License Expression>. This can be:

    • A single short-form identifier (e.g., GPL-2.0-or-later).
    • A combination using WITH for license exceptions.
    • A combination using AND or OR for multiple licenses.

    Style for C/C++

    For C/C++ header or source files, use the following comment format:

    /*
     *  Copyright (C) <year> <copyright holders>
     *  This file is part of <software> - <URL>
     *
     *  SPDX-License-Identifier: <SPDX License Expression>
     *  See <license file/license index file> for more information.
     */
  6. Implement a custom Callback to receive parsing events

    Piers

    To react to WebM parsing events (like when an element begins), you must create a class that inherits from video_webm_parser::Callback and overrides its virtual methods.

    In the OnElementBegin override, you receive video_webm_parser::ElementMetadata containing the element's id, position, header_size, and size. You must also set the video_webm_parser::Action* action pointer to determine the next step (e.g., video_webm_parser::Action::kRead) and return a video_webm_parser::Status.

    #include <iomanip>
    #include <iostream>
    #include <webm/callback.h>
    #include <webm/file_reader.h>
    #include <webm/status.h>
    #include <webm/webm_parser.h>
    
    class MyCallback : public video_webm_parser::Callback {
     public:
      video_webm_parser::Status OnElementBegin(const video_webm_parser::ElementMetadata& metadata,
                                               video_webm_parser::Action* action) override {
        // Access metadata fields:
        // metadata.id
        // metadata.position
        // metadata.header_size
        // metadata.size
    
        // Set the action to continue reading
        *action = video_webm_parser::Action::kRead;
        return video_webm_parser::Status(video_webm_parser::Status::kOkCompleted);
      }
    };
    
    int main() {
      MyCallback callback;
      video_webm_parser::FileReader reader(std::freopen(nullptr, "rb", stdin));
      video_webm_parser::WebmParser parser;
      video_webm_parser::Status status = parser.Feed(&callback, &reader);
      
      if (status.completed_ok()) {
        std::cout << "Parsing successfully completed\n";
      } else {
        std::cout << "Parsing failed with status code: " << status.code << '\n';
      }
    }
  7. Configure fuzzing for the WebM Parser

    Piers

    If you are using fuzzing/webm_fuzzer.cc with AFL or libFuzzer, you can use the following macros to control behavior:

    • WEBM_FUZZER_BYTE_ELEMENT_SIZE_LIMIT: Define this as an integer to limit the maximum size of ASCII/UTF-8/binary elements. This prevents the fuzzer from triggering massive allocations that cause false positives or memory exhaustion. If an element exceeds this limit, the parser returns Status::kNotEnoughMemory.
    • WEBM_FUZZER_SEEK_FIRST: Define this to force WebmParser::DidSeek() to be called before parsing begins, allowing you to test seeking code paths.
  8. Example: C/C++ copyright and SPDX notice

    Piers

    A typical copyright notice for a file licensed under GPL-2.0-or-later in this project should look like this:

    /*
     *  Copyright (C) 2005-2022 Team Kodi
     *  This file is part of Kodi - https://kodi.tv
     *
     *  SPDX-License-Identifier: GPL-2.0-or-later
     *  See LICENSES/README.md for more information.
     */
  9. Implement a Callback to handle parsed elements

    Piers

    The Callback class is notified as the parser builds objects representing WebM elements. You should subclass Callback and override the methods corresponding to the elements you wish to process.

    Core Callback Methods

    • OnElementBegin(): Called for every element encountered. Useful for mapping the file structure or skipping elements.
    • OnUnknownElement(): Called when an element is unrecognized or improperly nested. The default behavior is to skip the element.
    • OnVoid(): Called to handle Void elements.

    Element Nesting and Lifecycle

    Most methods follow a *Begin() and *End() pattern. The parser guarantees that these are called in the proper nesting order defined by the WebM DOM. For example, OnTrackEntry() will only be called between OnSegmentBegin() and OnSegmentEnd().

    Controlling Parser Flow

    • Continue Parsing: Return Status::kOkCompleted from a callback method.
    • Stop Parsing: Return any other status code (except Status::kEndOfFile). The parser will stop and return that status to the caller.
    • Resuming: If you return a non-error status (like Status::kWouldBlock or a custom status > 0), the parser may be resumed, and it will call the same callback method again.
    /* Example of the nesting order for specific elements */
    // Callback::OnEbml()
    // Callback::OnSegmentBegin()
    //    Callback::OnSeek()
    //    Callback::OnInfo()
    //    Callback::OnClusterBegin()
    //       Callback::OnSimpleBlockBegin()
    //          Callback::OnFrame()
    //       Callback::OnSimpleBlockEnd()
    //       ...
    //    Callback::OnClusterEnd()
    //    Callback::OnTrackEntry()
    //    ...
    // Callback::OnSegmentEnd()
  10. Use WebmParser to parse a file

    Piers

    To parse a WebM file, construct a WebmParser instance and call WebmParser::Feed(), passing in your Reader and Callback implementations.

    Standard Parsing

    WebmParser parser;
    // parser.Feed(callback, reader) returns Status::kOkCompleted on success
    Status status = parser.Feed(myCallback, myReader);

    Seeking to the middle of a file

    If you want to start parsing from an arbitrary point, you must call WebmParser::DidSeek() before calling Feed().

    Important: You must seek to the beginning of a WebM element. Seeking to a location that is not the start of an element (like the middle of a frame) will cause parsing to fail.

    Calling DidSeek() resets the parser state and clears internal errors, allowing the same WebmParser instance to be reused for a different file or a different seek location.

    WebmParser parser;
    
    // To start from the middle of a file:
    parser.DidSeek(); 
    
    // Then feed the data starting from that element:
    Status status = parser.Feed(myCallback, myReader);
  11. Format license files in the LICENSES directory

    Piers

    Every SPDX identifier used in the source code must have a corresponding file in the LICENSES subdirectory. This allows tools to verify compliance and provides the full license text. A typical license file follows this structure:

    Valid-License-Identifier: GPL-2.0-or-later
    SPDX-URL: https://spdx.org/licenses/GPL-2.0-or-later
    Usage-Guide:
      To use the GNU General Public License v2.0 or later put the following SPDX
      tag/value pair into a comment according to the placement guidelines in
      the licensing rules documentation:
    SPDX-License-Identifier: GPL-2.0-or-later
    License-Text:
      Full license text