DOCSEARCHbyAlgolia
Submit Docs

NLWeb Documentation

repository·main·Indexed 27 days ago

https://github.com/nlweb-ai/nlweb

A framework and set of protocols for building conversational web interfaces using Schema.org structured data. NLWeb enables humans and AI agents to interact with websites via natural language and the Model Context Protocol (MCP). The ecosystem includes an incremental website crawler for extracting schema markup, a WHO Standalone Handler for agent discovery via REST and MCP, and a high-performance aiohttp webserver supporting SSE and WebSockets.

Tokens
66.1K
Snippets
129
Records
341
Agent score
91%

What's inside NLWeb

  1. Overview of NLWeb

    main

    NLWeb is a collection of open protocols and tools designed to simplify building conversational interfaces for websites. It enables natural language interaction by leveraging Schema.org and other semi-structured formats (like RSS) as a semantic layer.

    Key features include:

    • MCP/A2A Support: Every NLWeb instance acts as an MCP (Model Context Protocol) server, supporting a core ask method that allows AI agents and humans to pose natural language questions to a website.
    • Structured Responses: Responses are returned in JSON using Schema.org vocabulary.
    • Platform Agnostic: Supports Windows, macOS, and Linux, and integrates with various LLMs (OpenAI, DeepSeek, Gemini, Anthropic, etc.) and vector stores (Qdrant, Snowflake, Milvus, etc.).

  2. Overview of NLWeb Webserver components

    main

    The NLWeb webserver architecture consists of three main layers:

    1. Servers: Includes the modern aiohttp_server.py (async, high-performance, supports WebSockets/SSE) and the legacy WebServer.py (Flask-based WSGI implementation).
    2. Middleware: Handles cross-cutting concerns including auth.py (authentication), cors.py (CORS headers), error_handler.py (centralized error formatting), logging_middleware.py (request/response logging), and streaming.py (SSE support).
    3. Utilities: Includes static_file_handler.py for MIME-type aware file serving and mcp_wrapper.py for MCP protocol abstraction.

  3. Overview of MCP + Apps SDK integration

    main

    The Model Context Protocol (MCP) connects LLM clients to external tools and UIs. In the context of the Apps SDK (e.g., ChatGPT), an MCP server provides three core capabilities:

    1. List tools: Advertises supported tools via JSON Schema input/output contracts and optional annotations like readOnlyHint.
    2. Call tools: Executes actions when a model issues a call_tool request and returns structured content.
    3. Return widgets: Returns embedded resources in response metadata, allowing the Apps SDK to render rich UI components (widgets) inline using _meta.openai/outputTemplate metadata.

    The protocol is transport agnostic and supports both Server-Sent Events and streaming HTTP.

  4. Overview of satisficer (nlweb-router)

    main
    The satisficer package (part of nlweb-router) is a model router designed to implement the principle of 'satisficing' in AI decision-making. Instead of always aiming for the most optimal (and often most expensive or slowest) model, satisficer helps you pick a model that is 'barely adequate' for a specific task, prioritizing efficiency and adequacy over pure optimality.

  5. Overview of the Enterprise Semantic Layer POC

    main

    The Enterprise Semantic Layer Proof of Concept (POC) demonstrates how to enable cross-system querying without hand-coded glue. The system allows users to ask natural language questions (e.g., 'which deals are at risk based on open support tickets?') and translates them into queries against a common ontology, which are then compiled to SQL for specific source databases using TMCF mappings.

    The POC architecture consists of four primary components:

    1. Source databases: Realistic tables for enterprise applications (HubSpot CRM, Jira, Dynamics 365 Sales) populated with synthetic data.
    2. Ontology: A shared enterprise vocabulary based on Schema.org.
    3. TMCF mappings: Declarative mappings that connect each application's native schema to the shared ontology.
    4. NL-to-semantic translator: A system that converts English questions into ontology-based queries, which are subsequently compiled into SQL for the target source tables.

  6. Explore the NLWeb Repository Structure

    main

    The repository is organized into several functional modules:

    • AskAgent: The core query agent. Handles natural language queries against websites using Schema.org, featuring connectors for LLMs, vector databases, and data ingestion tools.
    • AgentFinder: A service for discovering and routing to NLWeb agents across the web.
    • DataFinder: Translates natural language to SQL for enterprise data sources (e.g., HubSpot, Dynamics 365, Jira) using Schema.org ontology mappings.
    • ModelRouter: Routes LLM requests and scores them to select cost-effective models that meet quality thresholds.
    • NLWebScorer: Neural scorer models for ranking and evaluating search result quality.

    Supporting Directories:

    • config/: YAML configuration for LLM providers, embedding models, retrieval backends, and web server settings.
    • static/: Frontend web UI assets (HTML, CSS, JS).
    • demo/: Demo scripts and example data sources.
    • scripts/: CLI utilities and helper scripts.
    • docs/: Full documentation.

  7. Understand the NLWeb Project Structure

    main

    The NLWeb project is organized into several functional directories:

    • core/: Contains request handlers (baseHandler.py), state management (state.py), and ranking logic (ranking.py).
    • embedding/: Provides various embedding implementations (e.g., openai_embedding.py, azure_oai_embedding.py, ollama_embedding.py).
    • llm/: Contains LLM providers (e.g., anthropic.py, openai.py, azure_oai.py).
    • pre_retrieval/: Handles query analysis, decontextualization, and relevance detection.
    • retrieval/: Integrates with vector databases like Azure AI Search, Qdrant, and Snowflake.
    • tools/: Utility scripts for database loading and data processing.
    • webserver/: Contains the web server implementation, including streaming support.
    • config/: Stores YAML configuration files for LLMs, embeddings, and retrieval settings.

  8. Understand the NLWeb Mixed Mode Programming model

    main
    NLWeb utilizes a 'Mixed Mode Programming' model to balance the precision of traditional programming with the intelligence of LLMs. Instead of relying solely on large, unpredictable prompts, the system makes many small, precise calls to the LLM (e.g., checking if a query refers to a place or a previous item) while the Python code retains final control over the application flow and UI. This allows for reliable, constrained design patterns in transactional interactions while leveraging LLM background knowledge for conversational nuances.

  9. Project structure for Enterprise Semantic Layer POC

    main

    The Enterprise Semantic Layer Proof of Concept (POC) is organized into several functional directories:

    • databases/: Contains generated SQLite databases (hubspot.db, jira.db, dynamics365.db).
    • ontology/: Contains the core ontology (enterprise_schema.mcf) and enumeration mappings (enum_mappings.json).
    • mappings/: Contains TMCF files mapping specific databases to the ontology (hubspot.tmcf, jira.tmcf, dynamics365.tmcf).
    • translator/: The core logic engine:
      • nl_to_semantic.py: LLM-based translation from English to SemanticQuery JSON.
      • semantic_to_sql.py: Deterministic compilation of SemanticQuery JSON to SQL.
      • tmcf_parser.py: Parses TMCF files into lookup structures.
      • execute.py: Executes SQL against SQLite databases.
      • summarize_results.py: LLM-based translation from query results back to English.
    • demo.py: The CLI entry point for the system.
    • generate_data.py: Script to generate synthetic data for the databases.

  10. Understand the Template-Based Semantic Query Architecture

    main
    NLWeb uses a two-phase template-based architecture to translate natural language questions into execution plans across heterogeneous enterprise data sources (CRMs, ticketing systems, ERPs, etc.). Unlike general-purpose Text-to-SQL or pure agentic approaches, this architecture uses LLMs strategically for template matching and value mapping, while using deterministic machinery for execution and joins. This prevents LLM failure at scale when dealing with massive enterprise schemas.

  11. Understand the NLWeb to ChatGPT Integration Architecture

    main

    The integration connects the ChatGPT Desktop App to NLWeb through a multi-layered stack using the Model Context Protocol (MCP).

    Architecture Layers:

    1. ChatGPT Desktop App: Acts as the client, calling the nlweb-list tool via MCP and rendering results using interactive widgets.
    2. NLWeb MCP Server: (Node.js/TypeScript) Located at /openai-apps-sdk-integration/nlweb_server_node/. It registers the nlweb-list tool and provides the UI widget for search results.
    3. NLWeb AppSDK Adapter: (Python aiohttp) Located at AskAgent/python/webserver/appsdk_adapter_server.py. It proxies requests to the NLWeb Core and transforms responses into the AppSDK format.
    4. NLWeb Core Server: (Python aiohttp) Located at AskAgent/python/webserver/. It processes natural language queries and retrieves data from sources like Schema.org websites or vector databases (Milvus, Elasticsearch, OpenSearch).

    Data Flow Summary:

    • Query: ChatGPT → MCP Server (nlweb-list tool) → AppSDK Adapter (/ask endpoint) → NLWeb Core → Data Sources.
    • Response: NLWeb Core → AppSDK Adapter (format transformation) → MCP Server (adds UI metadata) → ChatGPT (renders widget).

  12. Understand the NL-to-Semantic Query Translation Architecture

    main

    The NL-to-Semantic Query Translator converts natural language questions into a structured intermediate representation (Semantic Query) using an LLM. This intermediate JSON format acts as a common language between natural language and SQL, allowing for deterministic compilation to SQL via TMCF (Table Mapping Configuration) mappings. The workflow is:

    1. NL -> Semantic Query: LLM call translating English to JSON.
    2. Semantic -> SQL: Deterministic compiler using TMCF and enum mappings.
    3. SQL Execution: Running the query against source databases.
    4. Result -> English: LLM call summarizing findings in natural language.