repository·main·Indexed 25 days ago
https://github.com/huggingface/optimumAn extension of Transformers, Diffusers, TIMM, and Sentence-Transformers that provides optimization tools to maximize efficiency when training and running models on specific hardware accelerators. It supports various integrations including ONNX Runtime, OpenVINO, and Habana Gaudi, and provides a CLI for model optimization and export.
🤗 Optimum provides an exporters module that allows you to export models from PyTorch into various optimized formats. Currently, the supported exporting formats are:
optimum-onnx for ONNX Runtime compatibility.optimum-intel for Intel hardware acceleration.optimum-neuron for AWS Inferentia/Trainium hardware acceleration.🤗 Optimum provides specialized packages for performing quantization depending on your target hardware and model type:
optimum.onnxruntime package to quantize and run ONNX models.optimum.intel package to quantize 🤗 Transformers models while optimizing for accuracy and latency on Intel hardware.optimum.fx package to access wrappers around PyTorch quantization functions. This is a lower-level API providing high flexibility for graph-mode quantization of 🤗 Transformers models.optimum.gptq package to quantize and run Large Language Models (LLMs) using the GPTQ method.torch.fx integration to enable graph transformations for quantization-aware training (QAT) and post-training quantization (PTQ) via one-liner operations.optimum.utils.input_generators module provides classes to easily and re-usably generate dummy inputs. These are useful for tasks such as model tracing, exporting models to specific backends, or testing model outputs.To render subpackage documentation on the Hugging Face website, add the following steps to build_pr_documentation.yml and build_main_documentation.yml in the main optimum repository.
Note: Ensure BUILD_DIR follows the pattern {subpackage_name}-doc-build and the subpackage name is correctly listed in the combine_docs.py command.
# Add this
- uses: actions/checkout@v2
with:
repository: 'huggingface/optimum-habana'
path: optimum-habana
# Add this
- name: Make Habana documentation
run: |
cd optimum-habana
make doc BUILD_DIR=habana-doc-build VERSION=pr_$PR_NUMBER # Make sure BUILD_DIR={subpackage_name}-doc-build
sudo mv habana-doc-build ../optimum
cd ..
# Tweak this to include your subpackage
- name: Combine subpackage documentation
run: |
cd optimum
sudo python docs/combine_docs.py --subpackages habana --version pr_$PR_NUMBER # Make sure the subpackage is listed here!
sudo mv optimum-doc-build ../
cd ..Learn how to fine-tune sequence-to-sequence models, such as T5, using ONNX Runtime (e.g., on the BBC news corpus).
https://github.com/huggingface/notebooks/blob/main/examples/summarization_ort.ipynbTo use specific hardware accelerators, install the corresponding Optimum extra. It is recommended to use the --upgrade --upgrade-strategy eager flags to ensure all dependencies are updated to their latest compatible versions.
| Accelerator | Installation Command |
|---|---|
| ONNX | pip install --upgrade --upgrade-strategy eager optimum[onnx] |
| ONNX Runtime | pip install --upgrade --upgrade-strategy eager optimum[onnxruntime] |
| ONNX Runtime GPU | pip install --upgrade --upgrade-strategy eager optimum[onnxruntime-gpu] |
| OpenVINO | pip install --upgrade --upgrade-strategy eager optimum[openvino] |
| NVIDIA TensorRT-LLM | docker run -it --gpus all --ipc host huggingface/optimum-nvidia |
| AMD Instinct / Ryzen AI | pip install --upgrade --upgrade-strategy eager optimum[amd] |
| AWS Trainium & Inferentia | pip install --upgrade --upgrade-strategy eager optimum[neuronx] |
| Intel Gaudi (HPU) | pip install --upgrade --upgrade-strategy eager optimum[habana] |
| FuriosaAI | pip install --upgrade --upgrade-strategy eager optimum[furiosa] |
pip install --upgrade --upgrade-strategy eager optimum[onnx]Use OpenVINO to optimize, quantize, and deploy deep learning models on Intel hardware. Detailed integration information is available in the optimum-intel repository.
Installation:
pip install --upgrade --upgrade-strategy eager optimum[openvino]Apply both static and dynamic quantization to models using ONNX Runtime for any GLUE task.
https://github.com/huggingface/notebooks/blob/main/examples/text_classification_quantization_ort.ipynbLearn how to fine-tune models like DistilBERT on GLUE tasks using ONNX Runtime.
https://github.com/huggingface/notebooks/blob/main/examples/text_classification_ort.ipynbUse the Example: prefix followed by a Python code block using doctest syntax (using >>> for commands and ... for continued lines). This allows the documentation examples to be automatically tested for consistency.
Examples should be minimal, clear, and include the expected output.
```python
Example:
```python
>>> from transformers import Wav2Vec2Processor, Wav2Vec2ForCTC
>>> from datasets import load_dataset
>>> import torch
>>> dataset = load_dataset("hf-internal-testing/librispeech_asr_demo", "clean", split="validation")
>>> dataset = dataset.sort("id")
>>> sampling_rate = dataset.features["audio"].sampling_rate
>>> processor = Wav2Vec2Processor.from_pretrained("facebook/wav2vec2-base-960h")
>>> model = Wav2Vec2ForCTC.from_pretrained("facebook/wav2vec2-base-960h")
>>> # audio file is decoded on the fly
>>> inputs = processor(dataset[0]["audio"]["array"], sampling_rate=sampling_rate, return_tensors="pt")
>>> with torch.no_grad():
... logits = model(**inputs).logits
>>> predicted_ids = torch.argmax(logits, dim=-1)
>>> # transcribe speech
>>> transcription = processor.batch_decode(predicted_ids)
>>> transcription[0]
'MISTER QUILTER IS THE APOSTLE OF THE MIDDLE CLASSES AND WE ARE GLAD TO WELCOME HIS GOSPEL'
```
```