DOCSEARCHbyAlgolia
Submit Docs

Kernel Memory

repository·main·Indexed 24 days ago

https://github.com/microsoft/kernel-memory

A multi-modal AI service for efficient dataset indexing and Retrieval Augmented Generation (RAG). It provides a framework for querying large datasets using natural language with built-in support for citations and data lineage. The service supports both a hosted web service via MemoryWebClient and a local serverless execution mode via MemoryServerless. It includes tools for custom ingestion pipelines using handlers and an evaluation system to measure RAG performance using metrics such as Faithfulness, Answer Relevancy, and Context Recall.

Tokens
23.8K
Snippets
64
Records
123
Agent score
80%

What's inside Kernel Memory

  1. Overview of Kernel Memory usage examples

    main

    The examples directory provides a comprehensive collection of scenarios for using Kernel Memory, ranging from basic web service interactions to advanced customizations. Key usage patterns include:

    • Deployment Modes: Using the Kernel Memory web service (client-server), running in serverless mode (embedded), or using .NET Aspire for orchestration.
    • Languages: Support for C# (.NET) and Python.
    • Integration: Using Kernel Memory as a plugin for Semantic Kernel, integrating with ASP.NET MVC, or using it via OpenAPI/curl.
    • Customization: Modifying the ingestion pipeline (custom handlers, decoders, web scrapers), customizing RAG/summarization prompts, adjusting text chunking/partitioning, and providing custom embedding/vector generators.
    • LLM Connectivity: Connecting to Azure services, Anthropic, Semantic Kernel connectors, or running local models via Ollama, ONNX, llama.cpp (LlamaSharp), or LM Studio.

  2. What is Kernel Memory

    main

    Kernel Memory (KM) is a multi-modal AI service designed for efficient dataset indexing and Retrieval Augmented Generation (RAG). It supports custom continuous data hybrid pipelines, synthetic memory, prompt engineering, and custom semantic memory processing.

    Key features include:

    • Natural Language Querying: Obtain answers from indexed data with citations and links to original sources.
    • Multi-modal Support: Handles various data types and formats.
    • Deployment Options: Available as a Web Service, Docker container, Plugin (for ChatGPT/Copilot/Semantic Kernel), or a .NET library for embedded applications.

  4. Use OpenAI as an LLM and Embedding connector in Kernel Memory

    main
    The Microsoft.KernelMemory.AI.OpenAI NuGet package provides the connector required to use OpenAI services for Large Language Model (LLM) text generation and embedding generation within Kernel Memory. This allows you to leverage OpenAI models for RAG (Retrieval-Augmented Generation) workflows.

  12. Use MemoryServerlessClient for serverless data ingestion and RAG

    main

    The MemoryServerlessClient allows you to perform data ingestion and memory retrieval (RAG) without deploying a dedicated Kernel Memory Service. Instead, it leverages cloud services directly to handle the heavy lifting.

    This architecture typically uses the following Azure services:

    • Azure Blobs: For file storage.
    • Azure AI Document Intelligence: For text extraction from images/documents.
    • Azure OpenAI: For generating embeddings and LLM-based answers.
    • Azure AI Search: For storing embeddings and text chunks.
    • Azure AI Content Safety: For validating LLM outputs.

    Configuration is managed via standard .NET configuration patterns (e.g., appsettings.json).