DOCSEARCHbyAlgolia
Submit Docs

TDengine Time-Series Database Documentation

website·Indexed 34 days ago

https://docs.tdengine.com

Documentation for TDengine TSDB, a purpose-built time-series database. Includes guides on TDengine SQL, stream processing, cluster management, and data ingestion. Covers native and REST connection methods for Java, C#, Go, Python, and C/C++, as well as enterprise components like taosX, taosX-Agent, and the PERFORMANCE SCHEMA.

Tokens
483.5K
Snippets
1.9K
Records
3.3K
Agent score
66%

What's inside TDengine

  1. Overview of PI connector deployment architectures

    The PI connector is a taosX plugin that reads data from a PI system and writes it to TDengine. Because it depends on the PI AF SDK, which is Windows-only, the connector must always execute on a Windows host. There are three primary deployment options:

    1. taosX Direct Connection (Option A): taosX is installed directly on a Windows host in the same network segment as the PI system. The connector runs as a built-in plugin.
    2. taosX-Agent Proxy Mode (Option B): taosX is deployed elsewhere (e.g., Linux, Cloud, or IT data center), and a taosX-Agent is deployed on a Windows host to act as a proxy to the PI system. This is the recommended mode for production and OT/IT isolated networks.
    3. Multi-PI System Aggregation (Option C): An enterprise-level setup where multiple plants each have their own PI systems and independent taosX-Agents, aggregating all data into a single unified TDengine cluster.

  2. Overview of TDengine TSDB Stream Processing

    TDengine TSDB (starting from v3.3.7.0) includes a built-in stream processing engine that allows users to define real-time transformations using SQL. When data is written into a source table, it is automatically processed and the results are pushed to target tables based on a trigger mode. This system is designed to handle high-throughput data ingestion with millisecond-level latency, serving as a lightweight alternative to complex external stream processing frameworks like Kafka and Flink.

    Key capabilities include:

    • Tiered Storage & Downsampling: Reducing storage costs and improving query efficiency for high-frequency raw data.
    • Precomputation: Accelerating real-time dashboards and reports by avoiding scans of billions of records.
    • Anomaly Detection: Enabling low-latency alerts based on predefined rules.
    • Decoupled Trigger-Compute Strategy: The trigger source (event driver) can be separate from the computation source table, and computations can be performed across different databases and tables using any supported query statement.

  3. Overview of TDengine MQTT Data Subscription

    Starting from v3.3.7.0, TDengine allows users to subscribe to data from existing topics using an MQTT client connected to a Bnode service.

    Key Specifications:

    • Protocol: MQTT 5.0 is recommended (required for sub-offset user properties). MQTT 3.1 and 3.1.1 are supported.
    • Authentication: Uses native TDengine authentication.
    • Topic Management: Topics must be pre-created; dynamic creation via message publishing is not supported.
    • Shared Subscriptions: Use the format $share/group_id/topic_name for load balancing and high availability.
    • Subscription Position:
      • latest (default): Subscribe to new data.
      • earliest: Subscribe from the earliest WAL position (set via user property sub-offset=earliest).
    • Quality of Service: Supports QoS 0 and QoS 1.

  4. Overview of taosX-Agent Store and Forward

    The Store and Forward feature ensures zero data loss in industrial data collection scenarios where the network between the taosX-Agent (deployed on the factory/OT side) and taosX (deployed in the data center) may be interrupted. When a network outage occurs, the Agent persists collected data to a local disk cache. Once the network recovers, the cached data is automatically resent to taosX.

  5. Overview of TDengine Active-Active Mode

    Active-active mode (available in TDengine TSDB-Enterprise only) allows for high availability and disaster recovery by deploying two separate TDengine instances (primary and secondary). Data is replicated in real-time between them via taosX. If the primary node fails, the client driver automatically and transparently fails over to the secondary node.

    Important Constraints:

    • Not supported in TDengine 3.4.x.
    • Each node can be a single instance or a full cluster.
    • WAL retention period affects fault tolerance: if the secondary node is unreachable longer than the WAL retention period, data loss occurs and must be recovered manually.

  6. Overview of supported Time-Series Foundation Models (TSFMs) in TDgpt

    Since v3.3.6.4, TDgpt supports six TSFMs: TDtsfm v1.0, Time-MoE, Chronos, Moirai, TimesFM, and Moment. TDtsfm and Time-MoE are included in the installation packages, while others require a local service. Third-party model service scripts are located in <TDgpt_root_directory>/lib/taosanalytics/tsfmservice/>.

  7. Overview of data writing methods for TDengine Cloud

    TDengine Cloud supports several methods for importing external data into instances. Before using any write method, you must first create a database in the cloud service's data browser due to permission restrictions.

    Available methods include:

    1. Data Sources: Used for importing large volumes of remote data. Supported types include TDengine subscription (local, cloud vendor servers, or other TDengine Cloud instances), InfluxDB, MQTT, PI, OPC-UA, and OPC-DA.
    2. Data Collection Agents: Uses third-party proxy tools and schema-less REST interfaces. Supported proxies include Telegraf and Prometheus; supported protocols include InfluxDB, OpenTSDB JSON, and OpenTSDB Telnet.
    3. Direct Writing: Data can be written directly using TDengine SQL or programmatically via TDengine client libraries.
    4. Stress Testing: The taosBenchmark tool can be used to write data for performance testing.

  8. Overview of TDengine Stream Processing

    TDengine (v3.3.7.0+) provides built-in stream processing to perform real-time aggregation, downsampling, and anomaly detection using SQL. It separates the trigger (when to compute) from the compute (what to calculate and where to send results). Typical use cases include reducing data scanned by later queries via downsampling, precomputing statistics for dashboards, and triggering early alerts.

  9. Overview of TDengine Cloud capabilities

    TDengine Cloud is a fully managed, elastic time-series data processing service. Key capabilities include:

    • Data Ingestion: Supports various data sources and collection agents.
    • Data Exploration: Web-based browser for databases and SQL query execution.
    • Programming Interfaces: Client libraries for Java, Python, Go, Rust, and Node.js, as well as a REST API.
    • Stream Processing: Built-in event-driven stream processing and continuous queries, removing the need for external tools like Flink or Spark.
    • Data Subscription (Topics): Kafka-like API for subscribing to tables with support for filter conditions and sharing across users/groups.
    • Management: Tools for user/group management, database replication across regions/clouds, backups, IP whitelisting, and operation logs.
    • DB Mart: A marketplace for published public databases within TDengine Cloud.

  10. Overview of OPC UA Alarm and Event (A&E) Collection

    OPC UA A&E collection allows TDengine to subscribe to event notifications pushed by an OPC UA Server using the Subscription + MonitoredItem + EventFilter mechanism. Unlike point collection (which monitors a node's attribute value), A&E collection captures state changes (alarms, state transitions, operator logs) and writes every state change as a new row in TDengine to reconstruct the alarm lifecycle.

    There are two primary data forms collected:

    1. Alarms: Stateful objects with a lifecycle (e.g., equipment trip). Each state transition pushes an event.
    2. Events (ordinary): Stateless, one-shot notifications (e.g., door opening, system start/stop).

    Key advantages over simulating alarms via point subscriptions include higher efficiency (one subscription for all alarms of a type) and complete semantics (capturing EventId, severity changes, and ClientUserId).

  11. Overview of taosdump backup and restore tool

    taosdump is a high-performance backup and restore tool for TDengine TSDB that stores data in a compressed columnar format. It supports full backups, database-level, table-level, time-range, and metadata-only backups, as well as resumable execution. Starting from v3.4.2.0, the tool was upgraded for better performance and smaller backup sizes. Note that while the new version can import Avro format data generated by older versions, it no longer generates Avro format backup data.

  12. Overview of taosgen performance benchmarking tool

    taosgen is a performance benchmarking tool for time-series data products used for data generation and write performance testing. It supports Windows, Linux, and macOS. Unlike taosBenchmark, taosgen supports job orchestration via Directed Acyclic Graphs (DAG), multiple targets (TDengine, MQTT, Kafka, InfluxDB), Lua expressions for data generation, real-time data generation, and TDengine connection pooling.