OpenMetadata

repository·main·Indexed 9 days ago

https://github.com/open-metadata/openmetadata

An open-source context and memory layer for AI that provides a unified metadata knowledge graph connecting technical metadata, business semantics, lineage, and organizational memory.

Tokens
426.3K
Snippets
799
Records
1.8K
Agent score
96%

What's inside OpenMetadata

  1. Overview of OpenMetadata E2E Test Coverage

    main
    OpenMetadata uses Playwright for End-to-End (E2E) testing. The test suite is organized into several functional domains: Governance, Platform, Discovery, Observability, and Integration. This documentation provides a high-level summary of the test coverage across these domains, including the number of components, test files, individual test cases, and total scenarios covered.
  2. What is OpenMetadata and its role in AI context

    main

    OpenMetadata is an open platform designed to provide a unified metadata knowledge graph for data users, AI assistants, and agents. Unlike raw database connectors, OpenMetadata provides the context and memory required for AI to reason safely over enterprise data. It connects technical metadata, data quality signals, lineage, ownership, usage, policies, conversations, memories, glossaries, classifications, metrics, domains, data contracts, and data products.

    Key capabilities include:

    1. Context: Technical, operational, trust, usage, and lineage metadata.
    2. Semantics: Business meaning via glossaries, metrics, classifications, and domains.
    3. Knowledge Graph: Relationships connecting assets, people, quality, lineage, and policies.
    4. Memory: Preserving tribal knowledge like conversations, AI threads, and decisions as reusable entities.
    5. Activation: Making context usable via MCP, Semantic Search, APIs, SDKs, and webhooks.
  3. Overview of OpenMetadata ORM

    main

    The OpenMetadata ORM is a utility layer designed to convert OpenMetadata Table Entities into SQLAlchemy ORM objects. It provides the necessary mapping and building tools to bridge the gap between OpenMetadata's data models and SQLAlchemy's object-relational mapping system.

    Key capabilities include:

    • Type Mapping: A defined mapping between OpenMetadata types and SQLAlchemy types.
    • ORM Builder: A specialized builder used to transform OpenMetadata (OMeta) entities into SQLAlchemy-compatible objects.
    • Custom Type Definitions: Support for specialized types, such as UUID, to ensure data fidelity during conversion.
  4. Overview of OpenMetadata MCP OAuth Implementation

    main

    The OpenMetadata MCP OAuth implementation provides an OAuth 2.0 Authorization Code Flow with PKCE (Proof Key for Code Exchange) specifically designed for Model Context Protocol (MCP) clients like Claude Desktop.

    Key Distinction: This is user SSO authentication for MCP clients. It allows users to authenticate using their existing OpenMetadata credentials (via SSO providers like Google, Okta, Azure AD, etc., or Basic Auth). Once authenticated, MCP tools execute using that specific user's permissions within OpenMetadata. This is not intended for connector-based OAuth for data sources.

  5. Review Module Quality Grades

    main

    OpenMetadata maintains a quality grading system for its modules based on a repository audit. Grades are assigned based on architectural discipline, convention adherence, and structural debt.

    Grading Scale:

    • A: Exemplary
    • B: Solid with bounded debt
    • C: Works but carries structural debt
    • Not assessed: Insufficient evidence (audit did not measure this module)
    ModuleGradeSummary
    ingestionB+Strongest architectural discipline; cleanest architecture in the repo.
    openmetadata-specBClean codegen foundation; minor POM hygiene issues.
    openmetadata-serviceC+Excellent surface hygiene but highly tangled internals/cyclic imports.
    openmetadata-uiCDisciplined component model but heavy architectural and i18n debt.
    openmetadata-mcpNot assessedOnly known to have a clean DAG position.
    openmetadata-sdkNot assessedOnly known to have a clean DAG position.
  6. Supported Service Connectors and Ingestion Workflows

    main

    OpenMetadata supports a wide range of service connectors for metadata ingestion. For most services, the standard lifecycle includes:

    1. Create & Ingest: Creating the service instance and triggering the first ingestion run.
    2. Update Metadata: Modifying service descriptions or updating ingestion schedule options.
    3. Delete: Removing the service from the platform.

    Supported services include:

    • Databases: MySQL, Postgres, Redshift, BigQuery.
    • Data Tools/Platforms: Metabase, Superset, Airflow, Kafka, MlFlow.
    • APIs: REST-based API services.

    Note on Database Services: Services like MySQL, Postgres, and Redshift include additional service-specific ingestion behaviors beyond standard metadata collection.

  7. Data Quality UI Features: Pagination and Filtering

    main

    The Data Quality interface provides several tools to manage large lists of test cases:

    • Filtering: Use the built-in filters to narrow down test cases based on specific criteria.
    • Pagination: Navigate through large datasets using pagination controls. You can verify the first page state, navigate to the next/previous pages, and adjust the number of items displayed using the page size dropdown.
    • Breadcrumbs: When navigating from a test case to an incident, the UI provides breadcrumb navigation to help you track your location within the observability workflow.
  8. Use the TextArea component for multi-line text input

    main

    The TextArea component from @openmetadata/ui-core-components is designed for collecting multi-line free text, such as descriptions, comments, or SQL/config snippets.

    When to use:

    • For inputs that require wrapping or growing beyond a single line.

    When NOT to use:

    • For single-value inputs (use Input instead).
    • For selecting from a set (use Select instead).
    • For bounded numeric scales (use Slider instead).
  9. Goals of the Incident Lifecycle Workflow

    main

    The redesign aims to achieve the following:

    • Move incident Thread/task lifecycle into a governance workflow.
    • Enable configurable auto-assign on incident creation (default is off).
    • Provide a default branching workflow for handling open/close operations.
    • Introduce reusable openTask and closeTask node types.
    • Extend the event pipeline to broadcast TCRS events to workflows.
    • Support re-opening an incident from Resolved to any non-Resolved status.
  10. OpenMetadata Core Platform Capabilities

    main

    OpenMetadata provides several key functional areas for data and AI management:

    • AI Context and Memory: Manages memory nuggets, conversations, agent threads, and decisions.
    • Discovery and Understanding: Asset search, semantic search, and usage tracking.
    • Governance and Semantics: Glossaries, classifications, tags, and domain management.
    • Data Contracts and Standards: Support for ODCS 3.1, DCAT/DPROD, PROV-O, and JSON Schema.
    • Data Quality and Observability: Tests, profiling, freshness, and incident workflows.
    • Lineage and Impact Analysis: Table, column, dashboard, and pipeline lineage (including OpenLineage support).
    • Security and Access Control: Authentication, authorization, roles, and policies.
    • Extensibility: 130+ connectors, APIs, SDKs, webhooks, and custom ingestion frameworks.
  11. Understand the OpenMetadata UI folder structure

    main

    The UI codebase follows a modular structure organized by responsibility. Key directories include:

    • @src/components/: Reusable UI components, each in its own folder.
    • @src/pages/: Top-level views mapped to routes.
    • @src/api/: API service functions and clients, grouped by resource.
    • @src/interface/: Shared TypeScript interfaces and types.
    • @src/hooks/: Custom React hooks.
    • @src/store/: Global state management (Redux, Zustand, etc.).
    • @src/routes/: Route definitions and helpers.
    • @src/assets/: Static files like images, fonts, and icons.
    • @src/styles/: Global LESS styles, variables, and mixins.
    • @src/utils/: Utility and helper functions.
  12. Use the Badge component for status and labels

    main

    The Badge component from @openmetadata/ui-core-components is used for labeling status, categories, or small counts inline (e.g., entity types, test status, or tag counts).

    Usage Guidelines:

    • Use when: You need to display non-interactive labels or small counts.
    • Don't use when: The element needs to trigger navigation or a primary action (use a Button instead), or when you need selectable/removable chips (use Tags instead).

    Variants available in the package:

    • Badge (Base)
    • BadgeWithDot
    • BadgeWithIcon
    • BadgeWithButton (includes a dismiss button)
    • BadgeWithFlag
    • BadgeWithImage
    • BadgeIcon
    import { Badge } from '@openmetadata/ui-core-components';
    
    <Badge color="success" size="md" type="pill-color">
      {t('label.active')}
    </Badge>;