rustlearn

The library provides several machine learning models that support fitting and prediction on both dense and sparse data:

• Logistic Regression: Using stochastic gradient descent (sgdclassifier).
• Support Vector Machines (SVM): Using the libsvm library.
• Decision Trees: Using the CART algorithm.
• Random Forests: Using CART decision trees.
• Factorization Machines.

To use rustlearn, start by importing the prelude to access all linear algebra primitives and common traits. You can then import specific models and utilities from their respective submodules.

Always use the prelude for core functionality and submodule imports for specific model configurations (like Hyperparameters).

use rustlearn::prelude::*;

use rustlearn::linear_models::sgdclassifier::Hyperparameters;
// more imports

To verify your installation or contribute to the project, use the following commands:

• Run basic tests: cargo test
• Run all tests (including slow/generated tests): cargo test --features "all_tests" --release
• Run benchmarks (requires nightly branch): cargo bench --features bench

The rustlearn crate provides several machine learning primitives and models:

Matrix Primitives

• Dense matrices
• Sparse matrices

Available Models

• Logistic Regression: Using stochastic gradient descent.
• Support Vector Machines (SVM): Using the libsvm library.
• Decision Trees: Using the CART algorithm.
• Random Forests: Using CART decision trees.
• Factorization Machines

Note: All models support fitting and prediction on both dense and sparse data and support parallelization for fitting and prediction.

The SGDClassifier implements logistic regression using Stochastic Gradient Descent with an Adagrad adaptive per-parameter learning rate.

• Adagrad: The learning rate for each parameter decreases based on the square root of the sum of its historical squared gradients. This allows for faster learning on rare features and more stable updates for common features.
• L2 Regularization: Penalizes the magnitude of coefficients to prevent overfitting.
• L1 Regularization: Promotes sparsity by truncating coefficients at zero whenever an update would cause a sign change.
• Data Support: The implementation is optimized for both dense Array and SparseRowArray inputs.

To use rustlearn, start by importing the prelude to access all linear algebra primitives and common traits. Individual models and utilities should be imported from their specific submodules.

Import Pattern

use rustlearn::prelude::*;

// Import specific models or hyperparameters
use rustlearn::linear_models::sgdclassifier::Hyperparameters;

You can convert between dense Array types and sparse types using the From trait or the .todense() method:

• Dense to Sparse: Use SparseRowArray::from(&dense_array) or SparseColumnArray::from(&dense_array).
• Sparse to Dense: Use sparse_array.todense() to return a dense Array.

use rustlearn::prelude::*;

// Dense to Sparse
let dense = Array::from(&vec![vec![1.0, 0.0], vec![0.0, 2.0]]);
let sparse = SparseRowArray::from(&dense);

// Sparse to Dense
let back_to_dense = sparse.todense();

To use a Factorization Machine, create a Hyperparameters instance, call .build() (or .one_vs_rest() for multiclass) to get a model, and then use .fit() to train it on your data.

For faster training on large datasets, use .fit_parallel(X, y, num_threads) which implements multithreaded fitting via asynchronous stochastic gradient descent (Hogwild).

use rustlearn::prelude::*;
use rustlearn::factorization::factorization_machines::Hyperparameters;
use rustlearn::datasets::iris;

let (X, y) = iris::load_data();

// Build a two-class model
let mut model = Hyperparameters::new(X.cols(), 10).build();

// Train the model
model.fit(&X, &y).unwrap();

// Predict
let prediction = model.predict(&X).unwrap();

To efficiently process only the data present in a sparse matrix, use the row or column iterators.

For SparseRowArray, use .iter_rows() to get an iterator over rows. Each row can then be iterated using .iter_nonzero() to yield (index, value) pairs.

For SparseColumnArray, use .iter_columns() to get an iterator over columns, followed by .iter_nonzero() on each column view.

use rustlearn::prelude::*;

// Assuming 'array' is a SparseRowArray
for (row_idx, row) in array.iter_rows().enumerate() {
    for (column_idx, value) in row.iter_nonzero() {
        println!("Entry at ({}, {}) is {}", row_idx, column_idx, value);
    }
}

Training is performed using the fit method. The model supports both dense Array and SparseRowArray data types.

Important: Epochs Calling fit once processes the provided dataset once. To perform multiple epochs of training (which is common for convergence), you must call fit repeatedly in a loop.

fit returns a Result<(), &'static str> and will error if data dimensionality or labels are mismatched.

// Single epoch
model.fit(&X, &y).unwrap();

// Multiple epochs
let num_epochs = 20;
for _ in 0..num_epochs {
    model.fit(&X, &y).unwrap();
}