DOCSEARCHbyAlgolia
Submit Docs

Central Dogma Documentation

repository·main·Indexed 20 days ago

https://github.com/line/centraldogma

A highly-available, version-controlled service configuration repository leveraging Git, ZooKeeper, and HTTP/2 to manage configuration data at scale. Includes documentation on running JMH benchmarks, using async-profiler for flame graphs, and a comprehensive Gradle build system featuring dependency management via dependencies.toml, project flags for Java and Maven publishing, and support for shaded JARs.

Tokens
44K
Snippets
114
Records
162
Agent score
71%

What's inside Central Dogma

  1. What is Central Dogma?

    main

    Central Dogma is an open-source, highly-available, version-controlled service configuration repository. It is built on top of Git, Apache ZooKeeper, and HTTP/2.

    Key capabilities include:

    • Centralized Storage: Store configuration files in formats like .json, .yaml, .xml, .ini, or even JavaScript.
    • Flexible Retrieval: Access configurations via RESTful APIs, a Java library, or a command-line client.
    • Real-time Notifications: Servers can watch files and receive immediate notifications when configurations are updated, allowing for dynamic updates without restarts.
    • Version Control & Review: Leverages Git for full history and supports pull request workflows for configuration changes.
    • Dynamic Templating: Supports variable interpolation using ${vars.varName} syntax.
    • High Availability: Uses a multi-master replicated architecture with Apache ZooKeeper acting as a replication log queue.

  2. Use client profiles for server discovery

    main

    Instead of hardcoding hosts, you can use client profiles. The ArmeriaCentralDogmaBuilder.profile(String...) method loads server lists from JSON files found in the classpath.

    Supported Profile Files

    1. centraldogma-profiles-test.json (checked first)
    2. centraldogma-profiles.json (fallback if the test file is missing)

    Profile Structure

    A profile contains a name, a priority, and a list of hosts. Each host entry includes host, protocol (http/https), and port.

    Customizing Profiles

    • Adding Custom Profiles: Add your own centraldogma-profiles.json to the classpath. Multiple files will be merged.
    • Overriding Profiles: You can override an official profile (like beta) by creating a custom profile with the same name but a higher priority value (the default priority is 0).
    [
  {
    "name": "beta",
    "priority": 100,
    "hosts": [
      { "host": "replica1.alternative-beta.example.com", "protocol": "http", "port": 36462 }
    ]
  }
]
    ArmeriaCentralDogmaBuilder builder = new ArmeriaCentralDogmaBuilder();
builder.profile("beta");
CentralDogma dogma = builder.build();

  3. Understand Central Dogma User Roles

    main

    Central Dogma uses four distinct role types to manage permissions across the system:

    RoleDescription
    System AdministratorA 'super user' with all permissions. Only they can restore removed projects. Configured in conf/dogma.json.
    OwnerThe project administrator. Owners can add/remove members (users or tokens), create repositories, remove repositories/projects, and configure permissions for roles, members, and tokens.
    MemberUsers who can view project configuration but are not allowed to change it.
    GuestLogged-in users who are neither owners nor members. They cannot see project configuration.

    Note: While the system supports four roles, the Web UI currently allows you to choose between Owner and Member when adding users.

  4. How templates and variables work in Central Dogma

    main

    Central Dogma uses templates and variables to enable dynamic configuration. A template is a configuration file (JSON or YAML) containing placeholders using the syntax ${vars.varName}. When a file is fetched with template rendering enabled, Central Dogma replaces these placeholders with values from defined variables.

    This mechanism allows you to:

    • Manage multiple environments (dev, staging, prod) using a single template.
    • Centralize and reuse common configuration values.
    • Separate sensitive data from the configuration structure.
    • Update configurations without modifying the template files themselves.
    // Example JSON template
{
  "apiEndpoint": "${vars.api.url}",
  "timeout": "${vars.api.timeout}"
}

  5. Configure Client IP retrieval via Proxy Headers

    main

    To ensure the access logs show the actual client IP instead of the proxy's IP, configure trustedProxyAddresses and clientAddressSources:

    1. trustedProxyAddresses: A list of IP addresses or CIDR blocks (e.g., 10.0.0.1 or 10.0.0.0/8) representing your trusted proxies.
    2. clientAddressSources: A list of HTTP header names to check for the client IP.
      • Use the reserved keyword PROXY_PROTOCOL to support the PROXY protocol.
      • By default, if trustedProxyAddresses is configured, Central Dogma uses forwarded, x-forwarded-for, and PROXY_PROTOCOL.

    If neither is configured, the remote connection address is used as the client address.

  6. Use absolute and relative revisions to reference history

    main

    You can reference specific points in a repository's history using two types of revision numbers:

    • Absolute Revision: A positive integer (e.g., 1, 2, 3) that refers to a specific commit in the history. Smaller numbers refer to older commits.
    • Relative Revision: A negative integer used to reference commits relative to the latest state.
      • -1 refers to the latest commit (equivalent to Git's HEAD).
      • -2 refers to the commit immediately preceding the latest commit.
      • Smaller (more negative) numbers refer to older commits.

  7. Variable precedence and merging rules

    main

    Variables can be defined at multiple levels. When multiple variables with the same ID exist, they are merged using a specific precedence order. Higher levels override lower levels.

    Precedence Hierarchy (Lowest to Highest):

    1. Project-level variables: Available to all repositories in the project.
    2. Repository-level variables: Available only within the specific repository.
    3. Root variable file: Defined in the repository root (e.g., /.variables.json).
    4. Entry path variable file: Defined in the same directory as the template (e.g., /configs/.variables.json).
    5. Client-specified variable file: Explicitly provided during the fetch request (e.g., via renderTemplate("/vars/prod.json")).

  8. Variable types: STRING and JSON

    main

    Central Dogma supports two variable types:

    1. STRING: Stores text values.

      • In JSON templates, you must wrap the placeholder in quotes: "${vars.myVar}".
      • In YAML templates, quotes are optional: ${vars.myVar}.

    2. JSON: Stores structured data (objects, arrays, numbers, booleans, null).

      • Use dot notation to access nested properties: ${vars.database.host}.
      • In JSON templates, placeholders must be quoted: "${vars.database.host}".
      • In YAML templates, quotes are optional: ${vars.database.host}.

    Note on Type Coercion: When using JSON templates, placeholders are rendered as strings (e.g., "5432"). However, when deserializing into Java objects using Jackson, these string values are automatically coerced into the correct types (like int or boolean) if the target field type matches.

    // JSON template usage for a JSON variable
{
  "dbHost": "${vars.database.host}",
  "dbPort": "${vars.database.port}",
  "useSsl": "${vars.database.ssl}"
}

  9. Use Application Tokens for Client Authentication

    main

    For automated clients (rather than human users in a browser), use Application Tokens instead of session tokens to avoid the complexity of managing session expiration.

    Key Characteristics:

    • Virtual Users: An Application Token acts like a virtual user and can be assigned any role in a project.
    • Permissions: Like members, tokens can have specific permissions set in repository configurations.
    • Creation: Anyone logged into Central Dogma can create an Application Token via the Application Tokens menu in the Web UI. Each token requires a unique Application ID to identify the client source.
    • Management: Only the token creator or a System Administrator can deactivate or remove a token.

    Token Levels:

    • User level: Standard tokens for client requests.
    • System Admin level: Created only by System Administrators. These tokens allow access to system administrator-level APIs.

  10. Template syntax and interpolation

    main

    Use the ${vars.varName} syntax for basic variable interpolation. For JSON variables containing objects, use dot notation to access nested properties.

    Warning: Avoid using FreeMarker directives (like <#if>, <#list>, or <#assign>) directly in .json or .yaml files. Doing so will cause Central Dogma to fail validation when storing the file. If you need FreeMarker logic, use the .ftl extension (e.g., config.json.ftl), which bypasss standard JSON/YAML validation.

    // Basic interpolation
Application: ${vars.appName}
Version: ${vars.version}

  11. Import Maven BOM (Bill of Materials)

    main

    You can import Maven BOMs by adding them to the [boms] section in your dependencies.toml. Once imported, you can declare dependencies that automatically use the versions managed by that BOM.

    [boms]
armeria = { module = "com.linecorp.armeria:armeria-bom", version = "1.17.0" }

[libraries.armeria]
module = "com.linecorp.armeria:armeria"

  12. Understand Central Dogma path syntax

    main

    The dogma client uses Unix-like paths to reference resources. The hierarchy is determined by the number of path components:

    • foo: Refers to Project foo.
    • foo/bar: Refers to Repository bar in Project foo.
    • foo/bar/alice: Refers to Directory /alice in Repository bar in Project foo.
    • foo/bar/alice/bob.json: Refers to File bob.json under Directory /alice in Repository bar in Project foo.