DOCSEARCHbyAlgolia
Submit Docs

AI Dev Gallery Documentation

repository·main·Indexed 23 days ago

https://github.com/microsoft/ai-dev-gallery

A preview tool for Windows developers to integrate AI capabilities into applications. It features interactive samples, local model management from Hugging Face and GitHub, and the ability to export samples as standalone Visual Studio projects. The documentation covers installation, device requirements, adding new samples using the [GallerySample] attribute, implementing model loading for language, embedding, speech, and image models, and configuring Limited Access Features (LAF) environment variables.

Tokens
10.4K
Snippets
16
Records
29
Agent score
77%

What's inside AI Dev Gallery

  1. WinML Initialization Flow and Project Conventions

    main

    When implementing LoadModelAsync, follow the project's approved initialization pattern to ensure hardware acceleration and compatibility.

    Initialization Steps:

    1. Register Certified EPs: Use Microsoft.Windows.AI.MachineLearning.ExecutionProviderCatalog.GetDefault().EnsureAndRegisterCertifiedAsync() to install/register hardware acceleration packages like DirectML. Fall back to CPU if this fails.
    2. Configure SessionOptions: Create a SessionOptions object and call so.RegisterOrtExtensions() to enable necessary operators.
    3. Apply Execution Provider (EP) Policy: Use sampleParams.WinMlSampleOptions to decide how to select the EP:
      • Policy: Use so.SetEpSelectionPolicy(options.Policy.Value) to let the system choose the best available EP.
      • Explicit: Use so.AppendExecutionProviderFromEpName(options.EpName, options.DeviceType) to request a specific provider.
    4. Handle Model Compilation: If options.CompileModel is true, use so.GetCompiledModel(modelPath, options.EpName) to replace the model path with a compiled artifact for faster subsequent runs.
    5. Create Session: Instantiate new InferenceSession(modelPath, so).
    6. Notify UI: Call sampleParams.NotifyCompletion() to signal the UI that the model is ready.
    // Implementation snippet for InitializeSessionAsync
var catalog = Microsoft.Windows.AI.MachineLearning.ExecutionProviderCatalog.GetDefault();
await catalog.EnsureAndRegisterCertifiedAsync();

SessionOptions so = new();
so.RegisterOrtExtensions();

if (options.Policy != null) {
    so.SetEpSelectionPolicy(options.Policy.Value);
} else if (options.EpName != null) {
    so.AppendExecutionProviderFromEpName(options.EpName, options.DeviceType);
    if (options.CompileModel) {
        modelPath = so.GetCompiledModel(modelPath, options.EpName) ?? modelPath;
    }
}

_session = new InferenceSession(modelPath, so);

  2. How to use AI models and samples

    main

    The AI Dev Gallery allows you to explore over 25 interactive samples powered by local AI models.

    • Model Selection: When executing a sample, you can select which model you want to use from the available options (which include popular open source models and APIs from Microsoft Foundry on Windows).
    • Offline Usage: The app works offline once models are downloaded. You only need an internet connection to download new models from Hugging Face or GitHub.
    • Exporting Samples: While the app is required to run samples initially, you can export any sample as a standalone Visual Studio project once the required model is downloaded. This allows you to run the sample independently of the AI Dev Gallery.

  3. Version format and management in version.json

    main

    The application version is managed in the version.json file located in the repository root.

    • Format: X.Y.Z (where X is major, Y is minor, and Z is patch).
    • Manual Updates: If you need to publish a version that is not a simple patch increment (e.g., a new major or minor version), you must manually update the version number in version.json before running the release commands.

  4. What are Limited Access Features (LAF)?

    main

    Limited Access Features are Windows platform features that require explicit permission from Microsoft. To use these features, an application must provide a feature ID and a corresponding token using the Windows.ApplicationModel.LimitedAccessFeatures API.

    Key Method:

    • LimitedAccessFeatures.TryUnlockFeature(string featureId, string token, string usage)

    Requirements:

    • OS: Windows 10, version 1809 or newer.

  5. How AI Dev Gallery telemetry works

    main

    The AI Dev Gallery uses EventSource with Windows ETW TraceLogging to collect telemetry.

    Collection Flow:

    1. Product code calls static logging methods (e.g., Log(...), LogError(...), LogCritical(...)) defined in Telemetry/Events/*.cs. The parameters passed to these methods become the public properties of the event.
    2. The ITelemetry implementation (Telemetry.cs) processes the event, performing sensitive string replacement (e.g., masking user directories and names).
    3. Log Level Downgrading: If IsDiagnosticTelemetryOn is false, non-error LogLevel.Info and LogLevel.Measure events are downgraded to LogLevel.Local (local-only). Critical and error-level events are always preserved.
    4. Events are written via EventSource.Write(...) using the provider name Microsoft.Windows.AIDevGallery and specific keywords (TelemetryKeyword, MeasuresKeyword, or CriticalDataKeyword).

  6. View AI Dev Gallery telemetry locally using PerfView

    main

    To verify or debug telemetry events using PerfView (recommended):

    1. Run PerfView as Administrator.
    2. Select Collect $\rightarrow$ Collect.
    3. Uncheck the following: Thread Time, CPU, .NET/CLR, and Kernel (set Kernel Events to None under Advanced).
    4. In the Additional Providers field, manually type the following provider string: Microsoft.Windows.AIDevGallery:0xFFFFFFFFFFFFFFFFFFFF:Verbose
    5. Start the collection, exercise the application, and then click Stop.
    6. Open the resulting .etl.zip file, navigate to Events, and filter by the provider Microsoft.Windows.AIDevGallery to inspect the events and their properties.

    Tip: If the provider does not appear in the dropdown menu, you must type it manually.

    Microsoft.Windows.AIDevGallery:0xFFFFFFFFFFFFFFFF:Verbose

  7. How to add a new sample to the AI Dev Gallery

    main

    To add a new sample, place your files within the Samples folder. A sample is defined by a .xaml and a .xaml.cs file. The class in your .xaml.cs must be annotated with the [GallerySample] attribute to be discovered and loaded dynamically by the app.

    Folder Structure Rules:

    • Samples inside a model or API folder are grouped together in the app.
    • Samples not inside a model or API folder render as individual samples (useful for high-level guides or multi-model samples).
    • If your sample uses a new model, include a definition in Samples\Definitions\Models\ using .model.json or .modelgroup.json.
    • If your sample uses a specific API, add it to an apis.json file under Samples\Definitions\.

  8. How to contribute to AI Dev Gallery

    main

    You can participate in the AI Dev Gallery project through several channels:

    • Submit Pull Requests: Use this to fix bugs, improve documentation, add new examples, or optimize existing code.
    • Open Issues: Use this to report problems or suggest improvements.
    • Join Discussions: Engage in technical conversations or propose new ideas in the Discussions section.

    When submitting contributions, ensure you:

    1. Follow the project's Contribution Guidelines.
    2. Use the provided Pull Request template.
    3. Verify that your changes pass all CI checks.
    4. Provide clear and concise code and documentation.
    5. Include usage instructions for any new examples.

  9. How to file issues and get help

    main
    This project uses GitHub Issues to track bugs and feature requests. Before filing a new issue, search the existing issues to avoid creating duplicates. For new bugs or feature requests, create a new Issue in the repository.

  10. Install and run AI Dev Gallery

    main

    You can install the AI Dev Gallery via the Microsoft Store or by building it from the source code.

    To build from source:

    1. Set up the environment: Install Visual Studio 2022 or later with the Windows application development workload. Ensure you have Windows 10 or newer.
    2. Clone the repository: Use git to clone the repo.
    3. Run the project: Open AIDevGallery.sln in Visual Studio. Set AIDevGallery as the startup project and press <kbd>F5</kbd> to run.

    Important for ARM64 (Copilot+ PCs): You must build and run the solution as ARM64 (not x64) to ensure compatibility with models like Phi Silica.

    git clone https://github.com/microsoft/AI-Dev-Gallery.git

  11. Configure LAF via MSBuild properties for CI/packaged builds

    main

    For CI/CD pipelines or packaged builds, it is preferred to inject LAF values at build-time using MSBuild properties. This maps the values into AssemblyMetadata, which takes precedence over runtime environment variables.

    The project maps the following MSBuild properties to AssemblyMetadata keys:

    • LafToken $\rightarrow$ LAF_TOKEN
    • LafPublisherId $\rightarrow$ LAF_PUBLISHER_ID

    Usage in Azure Pipelines: Pass pipeline variables directly to the dotnet build command using the /p: flag.

  12. Create a WinML Sample in AI Dev Gallery

    main

    To add a new WinML sample to the AI Dev Gallery, you must create a WinUI page consisting of a Sample.xaml and Sample.xaml.cs file. The page class must inherit from BaseSamplePage and be decorated with the [GallerySample] attribute to provide metadata for the app's dynamic loading system.

    Implementation Steps:

    1. Create Files: Place your MySample.xaml and MySample.xaml.cs in a new folder under AIDevGallery/Samples/.
    2. Decorate Class: Add the [GallerySample] attribute to your class.
    3. Initialize Model: Override LoadModelAsync(SampleNavigationParameters sampleParams) to set up your InferenceSession. You must call sampleParams.NotifyCompletion() when initialization is finished to hide the UI loading spinner.
    4. Manage Lifecycle: Register for the Unloaded event to dispose of your InferenceSession and other unmanaged resources to prevent memory leaks.

    Refer to the Minimal Sample skeleton for a complete boilerplate implementation.

    // Example structure for a new sample
[GallerySample(
    Name = "My Minimal WinML Sample",
    Model1Types = [ModelType.SqueezeNet],
    Scenario = ScenarioType.ImageClassifyImage,
    NugetPackageReferences = ["Microsoft.ML.OnnxRuntime.Extensions"],
    Id = "00000000-0000-0000-0000-000000000000",
    Icon = "\uE8B9"
)]
internal sealed partial class MyMinimalSample : BaseSamplePage
{
    protected override async Task LoadModelAsync(SampleNavigationParameters sampleParams)
    {
        // Initialize session and call:
        sampleParams.NotifyCompletion();
    }
}