coremltools

coremltools is a Python package used to convert machine learning models from third-party libraries into the Core ML format.

Supported Frameworks:

• TensorFlow 1.x
• TensorFlow 2.x
• PyTorch
• scikit-learn
• XGBoost
• LibSVM

Key Functionalities:

• Convert trained models to the Core ML format.
• Read, write, and optimize Core ML models.
• Verify conversion/creation (on macOS) by making predictions using Core ML.

The coremltools Python package is used to convert third-party machine learning models into the Core ML model package format. This allows you to integrate models into Apple apps using the Core ML framework, which leverages the CPU, GPU, and Neural Engine (NE) for optimized on-device inference.

Key capabilities include:

• Converting trained models from frameworks like TensorFlow and PyTorch.
• Reading, writing, and optimizing Core ML models to reduce storage, power consumption, and latency.
• Verifying model conversion by performing predictions on macOS.

Model Intermediate Language (MIL) is an intermediate representation used by coremltools during the conversion process. When converting models from frameworks like TensorFlow or PyTorch, coremltools first builds a MIL program and then translates that representation into the Core ML protobuf representation.

When to use MIL directly:

• Handling unsupported operators: If you encounter an unsupported operator error during conversion, you can write a composite operator using the MIL builder.
• Defining models from scratch: You can use MIL to define a model manually instead of starting from a source framework (e.g., for testing single operations).

In Core ML Tools, the conversion process involves several stages. After the Frontend stage (e.g., PyTorch/TensorFlow to Model Intermediate Language), MIL-based Graph Optimizations are applied.

These optimizations are implemented as passes that operate on a Program (a Pythonic representation of the model). A Program consists of a main function implemented as a Block, which contains a list of Operators. Passes are applied to the Program to simplify and canonicalize it.

During conversion, passes are managed via a pass_pipeline (passed to ct.convert). You can use default pipelines, predefined specialized pipelines, or define your own custom optimization logic.

The Core ML model format is defined using Protocol Buffers (protobuf). The root of the model structure is the Model message, which is defined in Model.proto. The specification is composed of various protobuf message definitions that describe:

• Data Structures and Types: Core data representations used within the model.
• Feature Engineering: Definitions for feature types and engineering model types.
• Predictive Model Types: Specific implementations including:
  • Neural Networks (NeuralNetwork.rst)
  • Classifiers (Classifiers.rst)
  • Regressors (Regressors.rst)
  • SVM (SVM.rst)
  • Tree Ensembles (TreeEnsemble.rst)
  • MIL (Model Intermediate Language) (MIL.rst)
  • Other Models (OtherModels.rst)
• Identity: Definitions related to model identity.

Palettization (or weight clustering) compresses models by clustering float weights into a Lookup Table (LUT) of centroids. Instead of storing full weight values, the model stores indices that point to entries in the LUT.

Supported bit depths for N (the number of bits used for palettization) are: {1, 2, 3, 4, 6, 8}.

The ItemSimilarityRecommender is a model format that predicts similarity scores for items based on a list of input items and their scores, using a pre-computed table of item similarities.

By default, the model predicts items that are most similar to the input set but are not part of the input set itself. The predicted score for an item $k$ is calculated as:

$$\sum_{i \in \text{observed items}} \text{sim}(k,i) \times (\text{score}_i - \text{shift}_k)$$

Where:

• sim(k,i) is the similarity between item $k$ and observed item $i$.
• shift_k is the itemScoreAdjustment used to counteract global biases for popular items (often set to zero).

Note that sim(k,i) is often zero because only the most similar scores for each item are stored.

A Core ML model is composed of a specificationVersion, a ModelDescription, and a specific model Type. The specificationVersion determines compatibility with different OS versions (e.g., Version 1 for iOS 11, Version 6 for iOS 15).

message Model {
    int32 specificationVersion = 1;
    ModelDescription description = 2;
    bool isUpdatable = 10;
    
    oneof Type {
        // Various model types (Pipeline, Regressors, Classifiers, etc.)
        PipelineClassifier pipelineClassifier = 200;
        NeuralNetwork neuralNetwork = 500;
        MILSpec.Program mlProgram = 502;
        // ...
    }
}

Core ML models execute in either float 16 or float 32 precision. The way this is handled depends on the model type:

• Neural Networks: Intermediate tensors are untyped. The runtime dynamically assigns precision based on the hardware: GPU and Neural Engine (NE) typically use float 16, while the CPU uses float 32. Precision is controlled by configuring allowed compute units (e.g., All, CPU&GPU, or CPUOnly).
• ML Programs: Intermediate tensors are strongly typed within the model. The runtime respects these types as the minimum precision and will not reduce them. A float 32 typed ML program is guaranteed to run in float 32 precision across all hardware/software, including GPU and CPU.