DOCSEARCHbyAlgolia
Submit Docs

Knowhere Documentation

repository·main·Indexed 24 days ago

https://github.com/ontos-ai/knowhere

Knowhere is a memory layer that transforms unstructured data (PDFs, Office files, images, tables, Markdown, and text) into a structured, navigable memory graph for AI agents. It supports Agentic RAG workflows by using a proprietary Tree-like algorithm to reconstruct document hierarchy, enabling agents to navigate section trees and graph links for traceable, well-contextualized retrieval. The system includes a knowhere-api (v0.1.0) and worker service, supporting various LLM providers including DeepSeek, OpenAI, ZhipuAI, and Alibaba DashScope.

Tokens
81.9K
Snippets
85
Records
452
Agent score
83%

What's inside Knowhere

  1. Overview of Transformer applications

    main

    Transformers are used across several major modalities:

    • Text and Language: Large Language Models (LLMs) like ChatGPT for processing and generating human-like language.
    • Audio: Speech-to-text (e.g., OpenAI Whisper) and music generation.
    • Vision (Analysis): Using Vision Transformers (ViT) to analyze images and videos (e.g., object detection).
    • Vision (Generation): Generating images (e.g., Stable Diffusion, Dall-E, Midjourney) and videos (e.g., Sora, Pika).

  2. Understand the contents of Probability Theory Lecture Notes

    main

    The 'Probability Theory Lecture Notes' by Phanuel Mariano is a structured educational document covering fundamental concepts of probability and combinatorics. The notes are organized into chapters that progress from basic counting principles to advanced multivariate distributions and limit laws.

    Table of Contents Overview:

    • Chapter 1: Combinatorics (Counting Principle, Permutations, Combinations, Multinomial Coefficients)
    • Chapter 2: Axioms of Probability (Sample Space, Axioms, Equally Likely Outcomes)
    • Chapter 3: Independence
    • Chapter 4: Conditional Probability and Independence (Bayes's Formula)
    • Chapter 5: Random Variables (Discrete, Expected Value, C.D.F., Variance)
    • Chapter 6: Discrete Distributions (Bernoulli, Binomial, Poisson)
    • Chapter 7: Continuous Random Variables (Uniform, Expectation, Variance)
    • Chapter 8: Normal Distributions
    • Chapter 9: Normal Approximations to the Binomial
    • Chapter 10: Continuous Distributions (Exponential)
    • Chapter 11: Multivariate Distributions (Joint, Independent, Conditional, Joint PDF)
    • Chapter 12: Expectations (Covariance, Correlations)
    • Chapter 13: Moment Generating Functions
    • Chapter 14: Limit Laws (Central Limit Theorem)

  3. Locate deployment assets for Knowhere

    main

    The deploy/ directory contains the necessary assets for deploying and running Knowhere services.

    • Use deploy/docker/ to find Dockerfiles for the API and worker services.
    • Use deploy/local-dev/ for Docker Compose assets intended for local development environments.

    For instructions on running the full stack locally, refer to the deploy/local-dev/README.md file.

  4. Identify core workflow ownership in apps/worker

    main

    The apps/worker service is responsible for the heavy lifting of data processing. Key areas include:

    • Worker Document Processing: Manages the actual processing runs, source preparation, and parsing execution.
    • Worker Document Parsing: The core parsing engine. It uses a format router and various adapters to handle different file types.
    • Heading Hierarchy: Logic for inferring structure and layout from parsed documents.

  5. Identify RAG ecosystem tools and technologies

    main

    The RAG ecosystem consists of several layers of technology:

    • Orchestration Frameworks: LangChain and LLamaIndex provide extensive APIs for building RAG pipelines.
    • Low-Code/No-Code Platforms: Flowise AI allows deploying RAG applications via a drag-and-drop interface.
    • Specialized/Alternative Stacks: HayStack, Meltano, and Cohere Coral.
    • Personal/Enterprise Services: Weaviate’s Verba (for personal assistants) and Amazon’s Kendra (for enterprise search with built-in connectors).

  6. Identify core workflow ownership in apps/api

    main

    The apps/api service acts as the coordinator for various workflows. If you need to extend or debug specific areas, refer to these functional groupings:

    • Document Ingestion: Handles the creation, scope, and dispatching of ingestion jobs.
    • Job Read/Admission: Manages reading job status and enforcing system limits/billing via job_admission.py.
    • Document Lifecycle: Manages the state and lifecycle of documents.
    • Retrieval: Handles agentic retrieval, search, hydration, and graph-based queries.
    • Billing & API Keys: Manages payment records, billing workflows, and API key management.
    • Webhooks & Callbacks: Manages webhook registration, secrets, and QStash async callbacks.
    • Storage Events: Handles S3-compatible storage event intake and upload handoffs.

  7. What is a Masked Transformer layer?

    main
    A masked transformer layer is a variation of the standard transformer layer where the attention mechanism is restricted. In self-attention, masking ensures that a token can only attend to itself and preceding tokens, preventing it from 'looking ahead' at future tokens in a sequence. This is critical for tasks like autoregressive next-token prediction.

  9. What is a Reproducing Kernel Hilbert Space (RKHS)?

    main

    A Reproducing Kernel Hilbert Space (RKHS) is a specific type of Hilbert space $\mathcal{B}$ consisting of functions $f: \mathcal{X} \to \mathbb{R}$ where, for every $x \in \mathcal{X}$, there exists a function $k_x \in \mathcal{B}$ such that the 'reproducing property' holds:

    $f(x) = \langle k_x, f \rangle$ for all $f \in \mathcal{B}$.

    The function $k(x, x') = \langle k_x, k_{x'} \rangle$ is the reproducing kernel of the space. Every kernel $k$ is associated with a unique RKHS.

    Key Properties:

    • Feature Map: For every kernel $k$, there exists a feature map $\phi$ into an inner product space $H$ such that $k(x, x') = \langle \phi(x), \phi(x') \rangle$.
    • Completeness: An RKHS is a complete inner product space (a Hilbert space) formed by the completion of the space of functions spanned by the kernel.

  11. Understand Transformer architecture and contextual embeddings

    main

    Transformers build contextual embeddings by integrating information from surrounding words to create richer representations of token meanings. Unlike static embeddings (like word2vec) where a word has a fixed vector, a transformer's representation of a token changes based on its context.

    How it works:

    1. Input Encoding: Tokens are converted into initial embeddings.
    2. Stacked Transformer Blocks: The embeddings pass through multiple layers. At each layer $k$, the model computes a representation for a token by combining its previous layer's information with information from neighboring tokens.
    3. Attention Mechanism: This is the core mechanism that weighs and combines representations from appropriate other tokens from layer $k$ to build the representation for layer $k+1$.
    4. Language Modeling Head: The final representations are passed to a head that predicts the next token in the sequence (logits).

  12. Understand Lasso prediction and estimation error

    main

    In the context of the Lasso model $Y = X\beta^{0} + \varepsilon - \bar{\varepsilon}1$, the prediction and estimation error can be bounded under a specific condition called the compatibility condition.

    If the compatibility condition holds and the Lasso solution $\hat{\beta}$ is obtained using a regularization parameter $\lambda = A\sigma \sqrt{\log(p) / n}$ (where $A > 0$), then with probability at least $1 - 2p^{-(A^2 /8 - 1)}$, the following error bound applies:

    $$\frac {1}{n} | X (\beta^ {0} - \hat{\beta}) | _ {2} ^ {2} + \lambda | \hat{\beta} - \beta^ {0} | _ {1} \leq \frac {1 6 \lambda^ {2} s}{\phi^ {2}} = \frac {1 6 A^ {2} \log (p)}{\phi^ {2}} \frac {\sigma^ {2} s}{n}$$

    Where:

    • $s$ is the size of the support set $S$.
    • $\phi^2$ is the compatibility constant.
    • $\sigma$ is the common parameter of the sub-Gaussian noise $\varepsilon$.
    • $n$ is the number of observations.
    • $p$ is the number of predictors.
    $\lambda = A\sigma \sqrt{\log(p) / n}$