repository·master·Indexed 12 days ago
https://github.com/dmlc/xgboostA highly efficient, distributed gradient boosting library designed for scalable machine learning tasks. It supports Python, R, C/C++, Java, and Scala, with integrations for Spark, Kubernetes, and Dask. Features include a C API for custom data iterators, support for external memory loading, and compatibility with hyperparameter optimization tools like Optuna and FLAML.
XGBoost4J is the JVM-based package for XGBoost, designed to integrate XGBoost's optimizations and performance into the JVM ecosystem. It allows developers to:
XGBoost (eXtreme Gradient Boosting) is an optimized distributed gradient boosting library designed for efficiency, flexibility, and portability. It implements machine learning algorithms under the Gradient Boosting framework, specifically providing parallel tree boosting (GBDT, GBM) to solve data science problems quickly and accurately.
Key features include:
XGBoost4J-Spark integrates XGBoost with Apache Spark by fitting XGBoost into the Spark MLLIB framework. This allows developers to combine XGBoost's high-performance gradient boosting algorithms with Spark's distributed data processing capabilities for:
The following topics are covered in the official XGBoost tutorials. Use these as starting points for specific implementation tasks:
Learning-to-rank (LTR) in XGBoost aims to train models that arrange query results into an ordered list. XGBoost implements LTR using the LambdaMART algorithm, which is an adaptation of the LambdaRank framework to gradient boosting trees.
Key characteristics include:
rank:ndcg (default) and rank:map.qid). Samples within a query group must be sorted by their qid in non-decreasing order.XGBoost provides a Dask interface to run training and prediction across a distributed cluster. A Dask cluster consists of a scheduler, workers, and a client (the user entry point).
When using the XGBoost Dask interface, you must pass the client object as an argument to most functions. If client is set to None, XGBoost will attempt to use the default client returned by Dask.
Key Requirements:
X and y) must be Dask DataFrames or Dask Arrays.if __name__ == "__main__": to avoid errors in distributed environments.from xgboost import dask as dxgb
import dask.array as da
import dask.distributed
if __name__ == "__main__":
cluster = dask.distributed.LocalCluster()
client = dask.distributed.Client(cluster)
# X and y must be Dask dataframes or arrays
num_obs = 1e5
num_features = 20
X = da.random.random(size=(num_obs, num_features), chunks=(1000, num_features))
y = da.random.random(size=(num_obs, 1), chunks=(1000, 1))
dtrain = dxgb.DaskDMatrix(client, X, y)
output = dxgb.train(
client,
{"verbosity": 2, "tree_method": "hist", "objective": "reg:squarederror"},
dtrain,
num_boost_round=4,
evals=[(dtrain, "train")],
)In the XGBoost R package, parameters for the core library (such as max_depth, regularization terms, or device) are passed to the xgb.train function as an R list object.
To provide a more idiomatic experience with IDE autocompletion and in-package documentation, it is recommended to use the xgb.params constructor function to build this list before passing it to xgb.train.
# Conceptual usage pattern
params <- xgb.params(
max_depth = 6,
eta = 0.1,
# ... other parameters
)
xgb.train(
params = params,
# ... other arguments
)The XGBoost runtime is a Go plugin within the Kubeflow Trainer controller. It manages the lifecycle of a distributed training job through two primary mechanisms:
It implements the EnforceMLPolicyPlugin interface to inject DMLC_* environment variables and expose the required container port (29500) for the Rabit tracker.
Workers discover the RabitTracker on the rank-0 pod using a Kubernetes headless service. The DMLC_TRACKER_URI is automatically constructed using the pattern:
<trainjob-name>-node-0-0.<trainjob-name>
Example Pod Mapping for a job named myjob with 4 nodes:
myjob-node-0-0: DMLC_TASK_ID=0 (Acts as both Tracker and Worker)myjob-node-0-1: DMLC_TASK_ID=1 (Worker)myjob-node-0-2: DMLC_TASK_ID=2 (Worker)myjob-node-0-3: DMLC_TASK_ID=3 (Worker)XGBoost supports distributed learning-to-rank via Dask, Spark, and PySpark.
Key Considerations for Distributed Training:
IDCG that occur when a group is split into smaller subgroups.The base_margin is a metadata field for DMatrix (or a parameter in the sklearn fit method) that specifies the global bias for the boosted model.
If base_margin is supplied, it overrides the base_score training parameter. This is useful when you want to train an XGBoost model to continue boosting from the predictions of another model.
A git push --force is required when you perform operations that alter the commit history, such as rebase or squash.
Safety Rule: It is safe to force push to your own fork as long as the commits you are altering are exclusively your own. Avoid force pushing to shared branches where others may have based their work on your previous commit history.