Phi Cookbook

This end-to-end (E2E) workflow demonstrates how to fine-tune, deploy, and integrate custom Phi-3 models using Azure Machine Learning and Prompt flow. The process is divided into three main scenarios:

1. Scenario 1: Set up Azure resources and Prepare for fine-tuning: Includes creating an Azure Machine Learning Workspace, requesting GPU quotas, configuring role assignments, and preparing datasets.
2. Scenario 2: Fine-tune the Phi-3 model and Deploy in Azure Machine Learning Studio: Involves setting up the Azure CLI, executing the fine-tuning process, and deploying the resulting model.
3. Scenario 3: Integrate with Prompt flow and Chat with your custom model: Focuses on connecting the deployed model to Prompt flow for interactive chat capabilities.

Olive (ONNX live) is a model optimization toolkit with a CLI designed to deliver models for the ONNX runtime with improved quality and performance. It optimizes models (typically PyTorch or Hugging Face) for specific deployment targets like NPUs, GPUs, or CPUs (e.g., Qualcomm, AMD, Nvidia, or Intel).

Key Features:

• Automated Workflows: Uses a sequence of passes (compression, quantization, graph optimization) and an auto-tuning search strategy to find the best model based on accuracy and latency constraints.
• Built-in Components: Over 40 optimization components for quantization, compression, and fine-tuning.
• CLI Commands: Supports olive quantize, olive auto-opt, and olive finetune.
• Deployment Support: Supports Multi LoRA serving, model packaging, and integration with Hugging Face and Azure AI.

Microsoft Olive is a hardware-aware model optimization tool designed to simplify the optimization of machine learning models for specific hardware architectures (cloud or edge). It supports both fine-tuning and inference for generative AI models like Phi-3.

Key capabilities include:

• Automated Optimization: Integrates and automates techniques tailored for specific hardware targets.
• End-to-End Solution: Considers constraints such as accuracy and latency while optimizing models.
• Extensibility: Allows users to plug in custom optimization innovations.
• Multi-Runtime Support: Works with AzureML, local GPU, CPU, and DirectML.

Olive (ONNX live) is a model optimization toolkit with a CLI designed to prepare models for the ONNX Runtime. It optimizes models for various deployment targets (NPU, GPU, CPU) provided by hardware vendors like Qualcomm, AMD, Nvidia, or Intel.

Key Features:

• Workflows: Executes an ordered sequence of optimization passes (e.g., compression, quantization, graph optimization).
• Auto-tuning: Uses search algorithms to automatically find the best parameters for accuracy and latency.
• Components: Over 40 built-in components for quantization, compression, and fine-tuning.
• Integrations: Supports Hugging Face, Azure AI, and Multi-LoRA serving.
• Efficiency: Includes a built-in caching mechanism to reduce costs.

Olive is a model optimization toolkit with a CLI designed to deliver models for the ONNX runtime with improved quality and performance. It optimizes models (typically PyTorch or Hugging Face) for specific deployment targets like NPUs, GPUs, or CPUs (e.g., Qualcomm, AMD, Nvidia, or Intel).

Key features include:

• Automated Workflows: Uses a sequence of passes (compression, quantization, graph optimization) and a search strategy to auto-tune parameters for accuracy and latency.
• Built-in Components: Over 40 optimization components for quantization, compression, and fine-tuning.
• CLI Commands: Provides olive quantize, olive auto-opt, and olive finetune.
• Integration: Supports Hugging Face, Azure AI, and Multi LoRA serving.
• Efficiency: Includes a caching mechanism to reduce costs.