DOCSEARCHbyAlgolia
Submit Docs

Yggdrasil Decision Forests (YDF)

repository·main·Indexed 20 days ago

https://github.com/google/yggdrasil-decision-forests

A library for training, evaluating, interpreting, and serving decision forest models, including Random Forest, Gradient Boosted Decision Trees, CART, and Isolation Forest. It provides a high-level Python API for model analysis and benchmarking, a low-level C++ API for training configuration, and experimental ports for native inference in Go and JavaScript (NodeJS and Browser).

Tokens
46.4K
Snippets
136
Records
205
Agent score
71%

What's inside Yggdrasil Decision Forests

  1. Generate predictions with YDF in JS

    main

    The ydf-inference package allows you to run inference on machine learning models trained with YDF (Python) in both NodeJS and web browsers.

    To use it, you must first train and export a model from Python, typically by saving it and zipping the directory. When zipping the model directory, it is important to use the -j flag to avoid including the directory structure, which ensures the model can be loaded correctly by the JS runtime.

    Workflow:

    1. Train a model in Python using ydf.
    2. Save the model using model.save("path").
    3. Zip the model directory (e.g., zip -rj model.zip path/to/model).
    4. Load the model in JS using ydf.loadModelFromZipBlob() (NodeJS) or ydf.loadModelFromUrl() (Browser).
    5. Call model.predict(examples) with a batch of data.
    6. Call model.unload() to release resources.
    # Python training snippet
import ydf
import pandas as pd

# Train a Gradient Boosted Trees model
learner = ydf.GradientBoostedTreesLearner(label="income", pure_serving_model=True)
model = learner.train(train_ds)

# Save the model
model.save("/tmp/my_model")

# Zip the model (CRITICAL: use -j to not include directory structure)
!zip -rj /tmp/my_model.zip /tmp/my_model

  2. Access TensorFlow RecordIO and Example file formats without TensorFlow dependency

    main
    The tensorflow_no_dep directory provides access to the TensorFlow RecordIO and TensorFlow Example file formats without requiring a full TensorFlow installation. This is useful if you need to work with these specific data formats but want to avoid the heavy dependency of the TensorFlow library. This directory contains copies of the files available in the Public TensorFlow Pip package.

  3. Use Yggdrasil Decision Forests in JavaScript

    main

    The JavaScript API allows you to run Yggdrasil Decision Forests (YDF) models or TensorFlow Decision Forests models in web environments. Depending on your requirements, you can choose between two specialized npm packages:

    1. Inference only: Use ydf-inference if you only need to load a pre-trained model and generate predictions.
    2. Training and inference: Use ydf-training if you need both the ability to train models and the ability to run inference.

  4. Run inference on Yggdrasil or TensorFlow Decision Forests models in Go

    main

    This Go port allows you to perform native inference on models trained with Yggdrasil or TensorFlow Decision Forests.

    Note: This API is EXPERIMENTAL and subject to change.

    Supported Models

    • Currently, only binary classification gradient boosted trees models are supported.

    Performance Considerations

    • The Go implementation is approximately 2x slower than the C++ implementation due to a straightforward implementation. If this becomes a bottleneck, contact the development team.

    Core Workflow

    1. Load the model: Use model_io.LoadModel to load the model into memory.
    2. Create an engine: Use serving.NewEngine (or serving.NewEngineWithCompatibility for models trained via the TensorFlow Python API) to create a serving engine. Once the engine is created, the original model object can be discarded.
    3. Allocate memory: Use engine.AllocateExamples and engine.AllocatePredictions to prepare buffers for input and output. Reusing these buffers is recommended for performance-sensitive code.
    4. Set features: Use examples.FillMissing() to reset values, then use SetNumerical or SetCategorical to populate specific feature indices.
    5. Predict: Call engine.Predict to run inference on the allocated examples.
    import (
    model_io "github.com/google/yggdrasil-decision-forests/yggdrasil_decision_forests/port/go/model/io/canonical"
    "github.com/google/yggdrasil-decision-forests/yggdrasil_decision_forests/port/go/serving"
)

// Load and prepare engine
model, err := model_io.LoadModel(modelPath)
engine, err := serving.NewEngine(model)

// Use engine for prediction
examples := engine.AllocateExamples(10)
predictions := engine.AllocatePredictions(10)
examples.FillMissing()
examples.SetNumerical(0, featureAge, 30)
engine.Predict(examples, 2, predictions)

  5. What is a Data Specification (dataspec)?

    main

    A data specification (or dataspec) defines the attributes in a dataset, including their names, semantics, and metadata.

    Key characteristics:

    • It is stored as a Protocol Buffer (Protobuf).
    • YDF configuration files use Protobuf V2 in text format.
    • You can inspect a dataspec using the show_dataspec CLI command.
    • If automatic inference is incorrect, you can override it using a dataspec guide (e.g., using regex to force specific columns to be CATEGORICAL).

  6. Understand X@Y Metrics

    main

    X@Y metrics (e.g., Precision at a specific Recall) are computed conservatively without interpolation. Depending on the metric, 'conservative' means either a lower or upper bound:

    • Precision @ Recall: Precision at the highest threshold where recall $\ge$ limit.
    • Precision @ Volume: Precision at the highest threshold where volume $\ge$ limit.
    • Recall @ Precision: Highest recall where precision $\ge$ limit.
    • Recall @ False Positive Rate: Highest recall where FPR $\le$ limit.
    • False positive rate @ Recall: Smallest (best) FPR where recall $\ge$ limit.

    Confidence intervals for X@Y metrics are computed using non-parametric percentile bootstrapping.

  7. Handle missing values in YDF

    main

    YDF handles missing values via global imputation during training:

    • NUMERICAL / DISCRETIZED_NUMERICAL: Replaced with the feature mean.
    • CATEGORICAL / BOOLEAN: Replaced with the most frequent value.
    • CATEGORICAL_SET: Always routed to the negative branch of a split.

    Configuration: If the hyperparameter allow_na_conditions is enabled, the algorithm can create explicit splits for "feature is NA".

    Format-specific missing values:

    • Numpy: Use np.Nan for NUMERICAL. Use empty strings for CATEGORICAL. BOOLEAN and CATEGORICAL_SET do not support missing values.
    • CSV: Represented by the string na or empty strings.
    • Avro: Uses the underlying Avro type definitions.

  8. Classification Metrics: ROC and AUC

    main

    To evaluate binary classification performance:

    • ROC (Receiver Operating Characteristic): A curve showing the relationship between Recall (True Positive Rate) and the False Positive Rate. It is computed without the convex hull rule.
    • AUC (Area Under the Curve of the ROC): The integral of the ROC curve, computed using the trapezoidal rule without the convex hull rule.

    Confidence intervals for AUC are provided as AUC CI [H] (Hanley et al method) or AUC CI [B] (non-parametric percentile bootstrapping).

  9. Use Oblique Models to remove stair-step artifacts

    main
    Standard decision trees use axis-aligned splits, which cause "stair-step" patterns on diagonal lines. Oblique Models (such as Oblique Random Forests or Oblique Gradient Boosted Trees) solve this by allowing a single split to test multiple attributes simultaneously, typically using a linear equation. This enables the model to represent straight diagonal lines and smoother curves more effectively than axis-aligned models.

  10. Compare Decision Forests with kNN, SVM, and Neural Networks

    main

    When choosing between decision forests and other model types for pattern reconstruction:

    • k-Nearest-Neighbors (kNN): Avoids stair-steps but often produces an "organic" texture that is actually an artifact of overfitting. It may fail to capture global structures (like outer boundaries).
    • Support Vector Machines (SVM): Produces very smooth surfaces. While excellent at expressing certain geometric shapes (like ellipses via linear inequalities), they may miss fine details.
    • Neural Networks (MLP): Learn patterns globally rather than independently across the feature space. This allows them to generalize complex repeating patterns (like a circle) well, but they can be difficult to train, and their performance is highly sensitive to hyperparameters like the number and size of hidden layers. They may also exhibit "overshooting" or "undershooting" at boundaries.

  12. How the Distribute computation model works

    main

    Distribute uses a manager-worker architecture to implement distributed algorithms.

    Initialization

    • A manager and N worker processes are executed across multiple machines.
    • Each worker is assigned a unique integer ID in the range [0, N).
    • The manager initializes the pipeline with a welcome blob (e.g., a proto). This blob is immutable and is provided to all workers upon startup and whenever a worker is restarted after preemption.

    Computation via Queries

    • Computation is driven by queries. A query consists of query data (typically a proto).
    • A query is executed by a worker, which returns answer data and an absl::Status.
    • Query Types:
      • Global Queries: Can be executed by any available worker.
      • Targeted Queries: Sent to a specific worker via its worker ID.
    • Execution Modes:
      • Synchronous (Blocking): The caller waits for the result.
      • Asynchronous (Non-blocking): The caller continues execution and retrieves results later.
    • Cascading: Workers can emit new queries while executing an existing one.

    Failure Handling

    • Worker Preemption: If a worker is preempted, it is re-initialized with the same welcome blob. The welcome blob can be used to point workers to checkpoint locations (e.g., a CNS path).
    • Manager Failure: If the manager is restarted, all workers are also restarted.
    • Query Resilience:
      • If a worker fails during a global query, the next available worker automatically picks up the task.
      • If a worker fails during a targeted query, the emitter waits for that specific worker to return online and automatically re-sends the query.
    • Emitter Failure: If the entity that emitted a query (manager or worker) is interrupted, any in-flight query answers are discarded.