OpenMetadata
repository·main·Indexed 9 days ago
https://github.com/open-metadata/openmetadataAn open-source context and memory layer for AI that provides a unified metadata knowledge graph connecting technical metadata, business semantics, lineage, and organizational memory.
What's inside OpenMetadata
- OpenMetadata uses Playwright for End-to-End (E2E) testing. The test suite is organized into several functional domains: Governance, Platform, Discovery, Observability, and Integration. This documentation provides a high-level summary of the test coverage across these domains, including the number of components, test files, individual test cases, and total scenarios covered.
What is OpenMetadata and its role in AI context
mainOpenMetadata is an open platform designed to provide a unified metadata knowledge graph for data users, AI assistants, and agents. Unlike raw database connectors, OpenMetadata provides the context and memory required for AI to reason safely over enterprise data. It connects technical metadata, data quality signals, lineage, ownership, usage, policies, conversations, memories, glossaries, classifications, metrics, domains, data contracts, and data products.
Key capabilities include:
- Context: Technical, operational, trust, usage, and lineage metadata.
- Semantics: Business meaning via glossaries, metrics, classifications, and domains.
- Knowledge Graph: Relationships connecting assets, people, quality, lineage, and policies.
- Memory: Preserving tribal knowledge like conversations, AI threads, and decisions as reusable entities.
- Activation: Making context usable via MCP, Semantic Search, APIs, SDKs, and webhooks.
Overview of OpenMetadata ORM
mainThe OpenMetadata ORM is a utility layer designed to convert OpenMetadata Table Entities into SQLAlchemy ORM objects. It provides the necessary mapping and building tools to bridge the gap between OpenMetadata's data models and SQLAlchemy's object-relational mapping system.
Key capabilities include:
- Type Mapping: A defined mapping between OpenMetadata types and SQLAlchemy types.
- ORM Builder: A specialized builder used to transform OpenMetadata (OMeta) entities into SQLAlchemy-compatible objects.
- Custom Type Definitions: Support for specialized types, such as
UUID, to ensure data fidelity during conversion.
Overview of OpenMetadata MCP OAuth Implementation
mainThe OpenMetadata MCP OAuth implementation provides an OAuth 2.0 Authorization Code Flow with PKCE (Proof Key for Code Exchange) specifically designed for Model Context Protocol (MCP) clients like Claude Desktop.
Key Distinction: This is user SSO authentication for MCP clients. It allows users to authenticate using their existing OpenMetadata credentials (via SSO providers like Google, Okta, Azure AD, etc., or Basic Auth). Once authenticated, MCP tools execute using that specific user's permissions within OpenMetadata. This is not intended for connector-based OAuth for data sources.
Review Module Quality Grades
mainOpenMetadata maintains a quality grading system for its modules based on a repository audit. Grades are assigned based on architectural discipline, convention adherence, and structural debt.
Grading Scale:
- A: Exemplary
- B: Solid with bounded debt
- C: Works but carries structural debt
- Not assessed: Insufficient evidence (audit did not measure this module)
Module Grade Summary ingestionB+ Strongest architectural discipline; cleanest architecture in the repo. openmetadata-specB Clean codegen foundation; minor POM hygiene issues. openmetadata-serviceC+ Excellent surface hygiene but highly tangled internals/cyclic imports. openmetadata-uiC Disciplined component model but heavy architectural and i18n debt. openmetadata-mcpNot assessed Only known to have a clean DAG position. openmetadata-sdkNot assessed Only known to have a clean DAG position. Supported Service Connectors and Ingestion Workflows
mainOpenMetadata supports a wide range of service connectors for metadata ingestion. For most services, the standard lifecycle includes:
- Create & Ingest: Creating the service instance and triggering the first ingestion run.
- Update Metadata: Modifying service descriptions or updating ingestion schedule options.
- Delete: Removing the service from the platform.
Supported services include:
- Databases: MySQL, Postgres, Redshift, BigQuery.
- Data Tools/Platforms: Metabase, Superset, Airflow, Kafka, MlFlow.
- APIs: REST-based API services.
Note on Database Services: Services like MySQL, Postgres, and Redshift include additional service-specific ingestion behaviors beyond standard metadata collection.
Data Quality UI Features: Pagination and Filtering
mainThe Data Quality interface provides several tools to manage large lists of test cases:
- Filtering: Use the built-in filters to narrow down test cases based on specific criteria.
- Pagination: Navigate through large datasets using pagination controls. You can verify the first page state, navigate to the next/previous pages, and adjust the number of items displayed using the page size dropdown.
- Breadcrumbs: When navigating from a test case to an incident, the UI provides breadcrumb navigation to help you track your location within the observability workflow.
Use the TextArea component for multi-line text input
mainThe
TextAreacomponent from@openmetadata/ui-core-componentsis designed for collecting multi-line free text, such as descriptions, comments, or SQL/config snippets.When to use:
- For inputs that require wrapping or growing beyond a single line.
When NOT to use:
- For single-value inputs (use
Inputinstead). - For selecting from a set (use
Selectinstead). - For bounded numeric scales (use
Sliderinstead).
Goals of the Incident Lifecycle Workflow
mainThe redesign aims to achieve the following:
- Move incident Thread/task lifecycle into a governance workflow.
- Enable configurable auto-assign on incident creation (default is off).
- Provide a default branching workflow for handling open/close operations.
- Introduce reusable
openTaskandcloseTasknode types. - Extend the event pipeline to broadcast TCRS events to workflows.
- Support re-opening an incident from
Resolvedto any non-Resolvedstatus.
OpenMetadata Core Platform Capabilities
mainOpenMetadata provides several key functional areas for data and AI management:
- AI Context and Memory: Manages memory nuggets, conversations, agent threads, and decisions.
- Discovery and Understanding: Asset search, semantic search, and usage tracking.
- Governance and Semantics: Glossaries, classifications, tags, and domain management.
- Data Contracts and Standards: Support for ODCS 3.1, DCAT/DPROD, PROV-O, and JSON Schema.
- Data Quality and Observability: Tests, profiling, freshness, and incident workflows.
- Lineage and Impact Analysis: Table, column, dashboard, and pipeline lineage (including OpenLineage support).
- Security and Access Control: Authentication, authorization, roles, and policies.
- Extensibility: 130+ connectors, APIs, SDKs, webhooks, and custom ingestion frameworks.
Understand the OpenMetadata UI folder structure
mainThe UI codebase follows a modular structure organized by responsibility. Key directories include:
@src/components/: Reusable UI components, each in its own folder.@src/pages/: Top-level views mapped to routes.@src/api/: API service functions and clients, grouped by resource.@src/interface/: Shared TypeScript interfaces and types.@src/hooks/: Custom React hooks.@src/store/: Global state management (Redux, Zustand, etc.).@src/routes/: Route definitions and helpers.@src/assets/: Static files like images, fonts, and icons.@src/styles/: Global LESS styles, variables, and mixins.@src/utils/: Utility and helper functions.
Use the Badge component for status and labels
mainThe
Badgecomponent from@openmetadata/ui-core-componentsis used for labeling status, categories, or small counts inline (e.g., entity types, test status, or tag counts).Usage Guidelines:
- Use when: You need to display non-interactive labels or small counts.
- Don't use when: The element needs to trigger navigation or a primary action (use a
Buttoninstead), or when you need selectable/removable chips (useTagsinstead).
Variants available in the package:
Badge(Base)BadgeWithDotBadgeWithIconBadgeWithButton(includes a dismiss button)BadgeWithFlagBadgeWithImageBadgeIcon
import { Badge } from '@openmetadata/ui-core-components'; <Badge color="success" size="md" type="pill-color"> {t('label.active')} </Badge>;