DOCSEARCHbyAlgolia
Submit Docs

Tungsten Clustering (for MySQL) 8.0

website·Indexed 33 days ago

https://docs.continuent.com/

Documentation for Tungsten Clustering (for MySQL) 8.0, covering cluster operations, backup and recovery procedures, Manager and Replicator APIs, and command-line tools such as tpm, trepctl, and thl. Includes guidance on failover internals, composite cluster switching, and JVM configuration for the Manager.

Tokens
5M
Snippets
25.9K
Records
37.1K
Agent score
43%

What's inside Tungsten Clustering

  1. Overview of the tpm Deployment Command

    The tpm command is the primary deployment tool for Tungsten Clustering (for MySQL). It is used to manage the installation, upgrade, and configuration of clustering services across hosts. The tool supports two primary configuration methods: INI files and Staging directories. Key capabilities include processing installs and upgrades, managing certificates, configuring firewalls, and performing validation checks on the deployment environment.

  2. Overview of MongoDB Replication in Tungsten Replicator 7.2

    Replicating data from MySQL to MongoDB involves a two-part process using Service Alpha. First, Service Alpha on the Extractor reads MySQL binary logs and stores the information in the Transaction History Log (THL). Second, Service Alpha on the Applier reads the THL from the remote replicator and applies the changes to MongoDB. These services can reside on the same machine or separate machines.

  3. Overview of cctrl command-line tool commands

    The cctrl tool provides a variety of commands to manage Tungsten Clustering. Key operations include:

    • Cluster Management: cluster (issue commands across the cluster), cluster topology validate (report on topology and health), cluster validate (validate quorum configuration), and show topology (display configured topology).
    • Service & Data Management: datasource (single datasource commands), replicator (specific replicator commands), router (specific router/connector commands), and service (run service scripts).
    • Cluster State & Health: ls (show cluster status), members (list dataservice managers), and ping (test host availability).
    • Operational Control: failover (primary to replica), switch (promote replica to primary), and set master (set primary within a datasource).
    • Modes: admin (metadata repair), expert (expert mode), and physical (physical mode).

  4. Overview of Batch Loading for Data Warehouses in Tungsten Replicator

    Tungsten Replicator provides a batch apply mechanism specifically for data warehouses (such as Vertica or Redshift) where standard JDBC SQL execution is inefficient. Instead of executing individual SQL statements, Tungsten uses the SimpleBatchApplier to group thousands of transactions and load them using the target database's native batch loading commands (e.g., the COPY command in Vertica).

    Unlike traditional ETL tools that run on a fixed schedule, Tungsten builds batches automatically as transactions arrive in the log. The system is self-adjusting: if small batches cause loading delays, Tungsten automatically increases the batch size to maintain performance and reduce lag.

  5. Overview of Tungsten Replicator Kafka Applier

    The Tungsten Replicator Kafka Applier converts incoming data streams from an upstream replicator into JSON documents and publishes them as messages to Apache Kafka topics. Each row from a source transaction is converted into an individual Kafka message, regardless of whether the original transaction contained multiple rows.

    Data Flow Process:

    1. Extraction: Data is extracted from MySQL binary logs using a standard extractor.
    2. Enrichment:
      • The ColumnNameFilter is used to tag row-change information with corresponding column names.
      • The PrimaryKeyFilter is used to add primary key information to the row-based replication data.
    3. Application: The resulting Transaction History Log (THL) information is applied to Kafka via the Kafka Applier.

    Deployment Architecture:

    • Extractor Side: Service Alpha extracts information from the MySQL binary log into THL.
    • Applier Side: Service Alpha reads the THL from the remote replicator and applies it to Kafka.
    • These services can reside on the same machine or on separate machines.

  6. Overview of Tungsten Replicator 7.2 Deployment Topologies

    Tungsten Replicator 7.2 supports several deployment topologies to manage data replication between sources and targets. The available topologies include:

    • Simple Primary/Replica: Replicates data from one single source host to one single target.
    • Active/Active: Enables bi-directional replication between two or more hosts. This specific topology is only supported between MySQL hosts.
    • Fan-Out: A Primary/Replica variation where a single source host replicates data to multiple targets. Targets can be of different types and can utilize advanced filtering.
    • Fan-In: The reverse of Fan-Out, where multiple source hosts replicate data into a single target. This allows for flexibility such as remapping schemas via advanced filtering.
    • Tungsten Cluster Integration: Using the replicator as a Cluster-Extractor allows the use of Transaction History Logs (THL) generated within an existing Tungsten Cluster to replicate data to a standalone target.

  7. Overview of Tungsten Replicator Architecture

    Tungsten Replicator is a high-performance replication engine that enhances native replication (e.g., MySQL) by providing global transaction IDs, advanced topology support (Composite Active/Active, star, and fan-in), and enhanced latency identification. It supports heterogeneous replication, allowing data to be transformed to match the structure of different target servers (e.g., MySQL to Oracle or MongoDB).

    The system consists of three core components:

    1. Extractor: Reads data from a MySQL data server and writes it into the Transaction History Log (THL). It records complete transactions, each assigned a unique internal transaction ID.
    2. Transaction History Log (THL): Acts as the central storage for extracted data. It stores transactions as SQL statements or row-based information, along with metadata, options, and an incremental global transaction ID. This allows for precise tracking and retrieval of specific transactions.
    3. Applier: Reads information from the THL and applies it to a destination data server. Because it reads from the THL, the applier can reformat row-based data to match the target database's format.

  8. Overview of composite datasource removal methods

    There are three supported methods to remove an entire composite datasource (cluster) service from an existing composite cluster (CAP, DAA, or CAA):

    1. tpm update: Uses the --replace-release flag after modifying the tungsten.ini file.
    2. tpm delete-service: A dedicated command for service deletion.
    3. Manual removal: Using individual commands for each required step.

  9. Overview of Tungsten Replicator Deployment Topologies

    Tungsten Replicator supports several deployment topologies to manage data movement between source and target hosts. The available topologies include:

    • Simple Primary/Replica: Replicates data from one single source host to one single target.
    • Active/Active: Enables bi-directional replication between two or more hosts. This specific topology is only supported between MySQL hosts.
    • Fan-Out: Replicates data from a single source host into multiple targets. Targets can be of different types, and advanced filtering can be applied.
    • Fan-In: Replicates data from multiple source hosts into a single target. This allows for flexibility such as remapping schemas via advanced filtering.
    • Tungsten Cluster Integration: Using the replicator as a Cluster-Extractor allows the use of Transaction History Logs (THL) generated within an existing Tungsten Cluster to replicate data to a standalone target.

  10. Overview of Tungsten Manager API Cluster Control

    The Tungsten REST APIv2 provides programmatic access to cluster control operations that are otherwise performed via the cctrl command-line utility. Supported operations include performing backups, initiating switches, shunning or welcoming nodes, and toggling the replicator online/offline status.

  11. Overview of cctrl command-line tool commands

    The cctrl tool provides a suite of commands for managing Tungsten Clustering. Key operational commands include:

    • ls: Show cluster status
    • members: List the managers of the dataservice
    • ping: Test host availability
    • show topology: Display the currently configured cluster topology
    • failover: Perform a failover operation from a primary to a replica
    • switch: Promote a Replica to a Primary
    • set master: Set the Primary within a datasource
    • recover master using: Recover the Primary within a datasource using a specified Primary
    • datasource: Issue a command on a single datasource
    • replicator: Issue a command on a specific replicator
    • router: Issue a command on a specific router (connector)
    • service: Run a service script
    • set: Set management options
    • create composite: Create a composite dataservice
    • physical: Enter physical mode
    • help: Display help information
    • quit or exit: Exit the cctrl shell

  12. Overview of Tungsten Dashboard

    Tungsten Dashboard is a Web-based user interface designed for monitoring and managing Tungsten Clustering deployments of MySQL databases. The documentation covers all General Availability (GA) versions up to and including version 8.0.3.