Knowledge Catalog

Open Knowledge Format (OKF) is a universal, vendor-neutral format for representing knowledge as plain markdown files with YAML frontmatter, organized in a directory hierarchy. It is designed to be human-readable, agent-readable, version-controllable via Git, and portable across any filesystem or tool that supports markdown (e.g., Obsidian, Notion, MkDocs).

Key features include:

• Structured & Unstructured Data: Uses YAML frontmatter for queryable fields (type, resource, tags, generated, status) and markdown bodies for prose and schemas.
• Provenance: Includes signals for source credibility, production/verification status, and freshness (stale_after).
• Graph-shaped: Uses standard markdown links to express relationships between concepts.
• Progressive Disclosure: Uses auto-generated index.md files to allow hierarchical navigation.

The kc_enrich_sample_data.ga_events table uses the Google Analytics 4 (GA4) sample ecommerce dataset from the BigQuery Public Datasets program. This dataset contains raw, event-level data exported from the Google Merchandise Store's GA4 web e-commerce setup.

Dataset Details

• Dataset Name: bigquery-public-data.ga4_obfuscated_sample_ecommerce
• Table Format: events_YYYYMMDD (Daily partitioned tables)
• Time Range: November 1, 2020, to January 31, 2021

Important Note on Data Obfuscation

To protect privacy and business metrics, the data is deliberately obfuscated. You will encounter placeholder values such as <Other>, NULL, and empty strings ''. Metrics in this sample dataset may not be directly comparable to the live GA4 Demo Account due to this obfuscation.

The Knowledge Catalog Toolbox provides a suite of tools for managing and enhancing metadata. It consists of two primary components:

1. Metadata as Code (kcmd): Enables managing metadata as source code artifacts that can be synchronized with the metadata stored in Knowledge Catalog.
2. Enrichment Agent (kcagent): Provides a ready-to-use agent and a customizable harness designed to produce, evolve, improve, and maintain metadata within Knowledge Catalog, ensuring it is optimized for consumption by other agents.

The acme_retail/metrics bundle contains definitions for key financial metrics used in the Acme Retail context. You can access detailed documentation for specific metrics via their respective files:

• Revenue: Recognized revenue based on Acme's FY2026 policy.
• Gross Margin: Gross margin calculated according to Acme's FY2026 Cost Allocation Standard.
• Gross Margin (legacy, pre-FY2026): A deprecated formula maintained for historical reproducibility of data prior to 2026.

The posts_orphaned_tag_wiki table is part of the Stack Overflow BigQuery public dataset. It contains posts that serve as wiki entries for tags that have become orphaned or no longer exist on Stack Overflow. These posts typically provide definitions or explanations for specific tags.

Resource Location: bigquery-public-data.stackoverflow.posts_orphaned_tag_wiki

The Metadata as Code project uses two distinct disk organization layouts depending on the scope type defined in your catalog.yaml manifest:

1. Standard Layout (used for bq-dataset and entryGroup scopes):

   • Structured Data: Metadata (entry info, schema, profiles, etc.) is stored in a main YAML file per entry.
   • Unstructured Data: Rich-text fields (like Overview) are stored in dedicated sidecar Markdown files.

2. Documents Layout (used for kb scopes):

   • Single File per Entry: Each entry is represented by a single .md file.
   • Structured Data: Metadata is stored within the YAML frontmatter of the Markdown file.
   • Unstructured Data: The primary overview.content aspect is promoted to the main Markdown body of the file. Other unstructured aspects may still use sidecars.

The crypto_bitcoin bundle provides several BigQuery tables containing Bitcoin blockchain data. You can query the following tables to analyze blocks, transactions, inputs, and outputs:

• Bitcoin Blocks Table: Contains all blocks from the Bitcoin blockchain, including block headers, transaction counts, sizes, and timestamps.
• Bitcoin Outputs Table: Contains outputs from all Bitcoin transactions, including script details and values in Satoshis.
• Bitcoin Transaction Inputs: Details the UTXOs (Unspent Transaction Outputs) spent in transactions.
• Bitcoin Transactions Table: Contains all Bitcoin transactions, including inputs, outputs, block metadata, and fee structures.

OKF is a minimal, human- and agent-friendly format for representing knowledge (metadata, context, and curated insight) surrounding data and systems. It is implemented as a directory of Markdown files with YAML frontmatter.

Key characteristics:

• Readable: Human-readable without special tools.
• Parseable: Agent-friendly without bespoke SDKs.
• Diffable: Works natively with version control (e.g., Git).
• Portable: Can be exchanged across organizations and tools.

OKF v0.2 prioritizes first-class support for provenance, trust, lifecycle, and attestation to make agent-maintained corpora reliable.

The comments table in the Stack Overflow dataset contains records of all comments posted on questions and answers. Each row represents a single comment, including its text, creation date, the associated post, and the user who made it. Data is available from September 2008 onwards.

Resource Link: https://bigquery.googleapis.com/v2/projects/bigquery-public-data/datasets/stackoverflow/tables/comments

The ga4_obfuscated_sample_ecommerce dataset is a publicly accessible BigQuery dataset containing obfuscated Google Analytics 4 (GA4) event data from the Google Merchandise Store. It covers three months of historical activity (November 1, 2020, to January 1, 2021) and is used for experimenting with high-volume GA4 event data, nested schemas, and ecommerce analytics.

Resource Details:

• Type: BigQuery Dataset
• Resource URL: https://bigquery.googleapis.com/v2/projects/bigquery-public-data/datasets/ga4_obfuscated_sample_ecommerce
• Primary Table: events_* (a partitioned, sharded table containing daily event export records).

The crypto_bitcoin dataset is a public Google BigQuery dataset containing the complete Bitcoin blockchain transaction ledger and block history. It is updated continuously and can be used for blockchain forensics, macroeconomic analysis, and mining research.

Resource Location: bigquery-public-data.crypto_bitcoin