repository·csharp3.3·Indexed 42 days ago
https://github.com/guojin-yan/openvino-csharp-apiA .NET wrapper for Intel OpenVINO that enables high-performance deep learning inference for models like YOLO, ResNet, BERT, and LLMs across Windows, Linux, and macOS. It features zero-copy memory operations via Span<T>, async/await support, and multi-device targeting (CPU, iGPU, GPU, NPU). Includes the OpenVinoSharp.GenAI wrapper for Generative AI tasks such as LLM text generation, Whisper speech recognition, and Visual Language Models (VLM).
OpenVINO C# API provides managed .NET wrappers for two primary C-based interfaces:
Depending on your use case, you will interact with different runtime packages.
The samples/GenAI directory contains C# projects that replicate OpenVINO GenAI C API scenarios. These samples provide practical implementations for various generative AI tasks using managed wrappers.
Supported Scenarios:
Note: Image generation, RAG, speech generation, and video generation are currently not included as C# samples because their managed pipeline wrappers are incomplete.
The OpenVINO C# API is a .NET wrapper for Intel OpenVINO, enabling high-performance deep learning inference on Windows, Linux, and macOS.
Key Capabilities:
Span<T> and Memory<T> for zero-copy memory operations (requires .NET Core 2.1+ or .NET 4.7.2+).async/await support for high-concurrency scenarios (requires .NET Core 3.0+).OpenVinoSharp.GenAI wrapper for LLM, Whisper, and VLM models.The OpenVINO C# API provides two main tracks of sample applications:
GenAI features are located in the OpenVinoSharp.GenAI namespace and include:
LLMPipeline: Text generation (Greedy, Beam Search, Sampling, Streaming).WhisperPipeline: Automatic Speech Recognition (ASR) with timestamp support.VLMPipeline: Visual Language Models for image understanding and interactive Q&A.Note: The GenAI runtime is an optional dependency. If you only use traditional APIs like Core, Model, Tensor, CompiledModel, or InferRequest, the openvino_genai_c library will not be loaded.
The OpenVINO C# API documentation is organized into several key areas to help you install, use, and troubleshoot the library:
All open-source code in this project follows the Apache License 2.0.
Note on Third-Party Libraries: This project integrates several third-party libraries. If any third-party library's license conflicts with Apache 2.0, the original license of that third-party library takes precedence. Users are responsible for reading and complying with the licenses of all third-party components used.
The native runtime package handles binary resolution differently depending on your target framework:
runtimes/{RID}/native/ layout..props file (build/net46/{NUGET_ID}.props) to explicitly copy the native libraries to your output directory.build/netstandard2.0 and empty lib placeholders to ensure compatibility and prevent legacy .NET Framework fallback warnings during restoration.TextGeneration/Chat sample supports the --turn argument, allowing you to pass multiple scripted turns for testing multi-turn conversations.WhisperSpeechRecognition accepts standard language codes (e.g., --language en). The sample converts these to the required native format (e.g., <|en|>) before calling the native C API.VisualLanguageChat uses ChatHistory combined with GenerateWithHistory() to implement interactive flows. This approach is compatible with the VLM history export features in OpenVINO GenAI 2026.3.--allow-empty true only for ABI smoke testing.The runtime binaries are handled differently depending on your target framework:
build/net/{NUGET_ID}.props) that automatically copies the required runtime files to your output directory during the build process.The OpenVINO C# API provides managed wrappers for the OpenVINO GenAI C API. The coverage includes several key pipelines and configuration objects. When using these wrappers, note the following ownership and data handling rules:
LLMPipeline, GenerationConfig, DecodedResults.WhisperPipeline, WhisperGenerationConfig, WhisperDecodedResults, WhisperDecodedResultChunk.VLMPipeline, VLMDecodedResults.ChatHistory, JsonContainer, PerformanceMetrics.LLMPipeline or DecodedResults) own their native pointers and must be managed via the IDisposable pattern. Metrics returned by result objects are owned by the managed wrappers and released using the corresponding native free function.JsonContainer uses a two-call UTF-8 buffer pattern.generate call. Optional native strings that are NOT_FOUND map to null in C#.On Apple Silicon, you must ensure that the .NET SDK architecture matches the native runtime architecture.
Critical Rules:
Architecture mismatches will cause native library loading failures.
To verify your current architecture, use:
dotnet --info
uname -mdotnet --info
uname -mThe VLMPipeline is the primary abstraction for multi-modal inference in OpenVinoSharp.GenAI. It orchestrates the interaction between visual inputs and text prompts.
VLMPipeline: The main engine that manages the model and device context. It requires a model directory containing OpenVINO IR, tokenizer, detokenizer, and configuration files.ImageTensorLoader: A utility to convert raw image data into the Tensor format required by the pipeline. Note that the current implementation specifically supports RGB BMP and binary PPM/PNM formats. For JPG or PNG, you must convert them to RGB BMP first.GenerationConfig: Controls the text generation process (e.g., maximum tokens, streaming, and stopping conditions).VLMDecodedResults: The output object containing the generated text and performance metrics (like token generation speed).The imageTensor must remain in scope and must not be disposed of while pipeline.Generate is executing, as the pipeline borrows the tensor during the inference call.