decisiontree Ruby Library

Decision Tree is a Ruby library that implements the ID3 (information gain) algorithm for decision tree learning. It supports both discrete and continuous datasets.

• Discrete models: Assume unique labels and can be graphed and converted into PNGs for visual analysis.
• Continuous models: Iteratively choose the best threshold between all possible assignments for a variable, resulting in a binary tree partitioned by thresholds (e.g., temperature > 20C).

The library provides two main training approaches:

• Ruleset: A class that trains an ID3Tree using 2/3 of the training data, converts it into a set of rules, and then prunes those rules using the remaining 1/3 of the data (following the C4.5 approach).
• Bagging: A bagging-based trainer that trains 10 Ruleset trainers and selects the best output based on voting during prediction.

For datasets containing both discrete and continuous variables, pass a hash to the ID3Tree.new constructor using the color: or hunger: syntax (mapping attribute names to :discrete or :continuous) to specify the type for each attribute.

require 'decisiontree'

labels = ["hunger", "color"]
training = [
        [8, "red", "angry"],
        [6, "red", "angry"],
        [7, "red", "angry"],
        [7, "blue", "not angry"],
        [2, "red", "not angry"],
        [3, "blue", "not angry"],
        [2, "blue", "not angry"],
        [1, "red", "not angry"]
]

dec_tree = DecisionTree::ID3Tree.new(labels, training, "not angry", color: :discrete, hunger: :continuous)
dec_tree.train

test = [7, "red", "angry"]
decision = dec_tree.predict(test)
puts "Predicted: #{decision} ... True decision: #{test.last}"

# => Predicted: angry ... True decision: angry

To train a tree using continuous data, instantiate DecisionTree::ID3Tree with an array of attribute names, the training data (an array of arrays where the last element is the label), a default value, and the :continuous type. Call .train to build the model, then use .predict(test_array) to get a decision.

require 'decisiontree'

attributes = ['Temperature']
training = [
  [36.6, 'healthy'],
  [37, 'sick'],
  [38, 'sick'],
  [36.7, 'healthy'],
  [40, 'sick'],
  [50, 'really sick'],
]

# Instantiate the tree, and train it based on the data (set default to 'sick')
dec_tree = DecisionTree::ID3Tree.new(attributes, training, 'sick', :continuous)
dec_tree.train

test = [37, 'sick']
decision = dec_tree.predict(test)
puts "Predicted: #{decision} ... True decision: #{test.last}"

# => Predicted: sick ... True decision: sick

When initializing ID3Tree, you can specify whether attributes are :discrete or :continuous.

• Global Type: Pass a single symbol :discrete or :continuous to apply that type to all attributes.
• Per-Attribute Type: Pass a Hash where keys are attribute names (as symbols) and values are the type (:discrete or :continuous). This allows mixing continuous variables (like temperature) with discrete variables (like color) in the same tree.

Example of per-attribute configuration:

type_config = { temperature: :continuous, color: :discrete }
tree = DecisionTree::ID3Tree.new(attributes, data, 'default_val', type_config)

The Bagging class implements an ensemble method by training multiple Ruleset instances on different subsets of the data. This helps improve model stability and accuracy.

Workflow:

1. Initialize Bagging with attributes, data, default, and type.
2. Call .train (this will process 10 different classifiers).
3. Call .predict(test) to get the consensus prediction and the weighted accuracy.

The Ruleset class provides a way to extract and prune the decision tree into a set of human-readable rules. It automatically splits data into training and pruning sets to prevent overfitting.

Workflow:

1. Initialize Ruleset with attributes, data, default, and type.
2. Call .train to build the tree and prune the rules.
3. Use .predict(test) to get a prediction and its associated accuracy.
4. Use .to_s to view the rules in a readable format.

Once a tree is trained using .train, use the .predict(test) method to classify new data points.

test should be an array representing the features of the instance you want to classify, following the same index order as the training attributes.

The ID3Tree#graph method can generate a visual representation of the trained tree using the graphr gem.

Requirements:

• The graphr gem must be installed (gem install graphr).

Usage: graph(filename, file_type)

• filename: The base name of the output file.
• file_type: The format of the output (defaults to 'png').

The ID3Tree class implements the ID3 algorithm for training decision trees on both discrete and continuous datasets.

To use it, initialize the class with a list of attributes, your training data, a default classification value (used when a leaf node is reached without further splits), and the data type configuration.

Data Format Requirements:

• The data should be an array of arrays (or objects responding to slice and last).
• The last element of each data row is the classification (label).
• The attributes should be a list of names/indices corresponding to the data columns.
• The type parameter can be a single symbol (:discrete or :continuous) applied to all attributes, or a Hash mapping specific attributes to their types.