DOCSEARCHbyAlgolia
Submit Docs

Apache IoTDB Documentation

website·Indexed 20 days ago

https://iotdb.apache.org/

Documentation for Apache IoTDB, featuring guides on data writing and querying, SQL manual, and authority management. It includes API references for Java Native API, Python Native API, JDBC, and REST API V2, as well as tools for data import and export, data subscription, and technical details on encoding, compression, and cluster data partitioning.

Tokens
864.7K
Snippets
4.3K
Records
5.9K
Agent score
96%

What's inside Apache IoTDB

  1. Overview of Apache IoTDB AINode

    V1.3.x
    AINode is an endogenous node type in Apache IoTDB (alongside Configurable Node and DataNode) designed to perform machine learning analysis on time series data. It allows users to register external machine learning models and execute analysis tasks (such as prediction and anomaly detection) using SQL statements. The creation, management, and inference of models are integrated directly into the database engine. AINode is delivered as an additional package and is installed independently of the main IoTDB cluster.

  2. Overview of Apache IoTDB UDF types

    V1.3.x

    Apache IoTDB supports three main types of User Defined Functions (UDF) to extend computing capabilities:

    • UDTF (User Defined Table Function): Custom scalar functions. They take k columns of time series and 1 row of data as input and output 1 column and 1 row. These can be used in any clause where scalar functions are permitted (e.g., SELECT, WHERE).
    • Custom Time Series Generation Functions: These take k columns and m rows of data and output 1 column and n rows of data (where m can differ from n). These are restricted to SELECT clauses. Supported access strategies include ROW_BY_ROW, SLIDING_TIME_WINDOW, SLIDING_SIZE_WINDOW, SESSION_TIME_WINDOW, and STATE_WINDOW.
    • UDAF (User Defined Aggregate Function): Custom aggregation functions. They take k columns and m rows of data and output 1 column and 1 row. These can be used in clauses where aggregation functions are permitted (e.g., SELECT, HAVING).

  3. Overview of IoTDB Data Import Methods

    IoTDB provides three primary methods for importing data: the Data Import Tool (a script-based utility for manual imports), the TsFile Auto-Loading Feature (which monitors directories for new TsFiles), and Load TsFile SQL commands. The Data Import Tool specifically supports CSV, SQL, and TsFile formats for single or batch imports.

  4. Overview of Apache IoTDB

    Apache IoTDB is a low-cost, high-performance IoT-native time-series database designed to handle massive data volumes, high sampling frequencies, and frequent out-of-order data. It provides a native time-series computation engine with over 100 built-in aggregation and time-series functions, supporting high-speed ingestion for millions of devices.

  5. Overview of the IoTDB Stream Computing Framework

    V1.2.x

    The IoTDB stream processing framework enables the implementation of customized stream processing logic to monitor storage engine changes, transform data, and push it to external systems. A stream processing task, known as a 'Pipe', consists of three sequential subtasks implemented via plugins:

    1. Pipe Extractor: Extracts data from the system.
    2. Pipe Processor: Processes and transforms the extracted data.
    3. Pipe Connector: Sends the processed data to an external system.

    These pipelines can be used for edge-to-cloud synchronization, remote disaster recovery, and read/write load balancing across multiple databases. Configuration is handled declaratively using SQL statements.

  6. Overview of Apache IoTDB Data Synchronization

    Data synchronization in Apache IoTDB enables data sharing between instances for use cases such as edge-to-cloud synchronization, data migration, and backup. A synchronization task is composed of three stages:

    1. Source Stage: Extracts data from the source IoTDB.
    2. Process Stage: Processes the extracted data.
    3. Sink Stage: Sends the processed data to the target IoTDB.

    Users can define the synchronization scope during task creation. The default scope is data.insert (newly written data).

  7. Overview of Hive-TsFile-Connector

    The Hive-TsFile-Connector allows Apache Hive to support external data sources of the TsFile type. It enables users to load single TsFiles or entire directories of TsFiles from the local file system or HDFS into Hive and query them using HQL. Note that write operations (such as the INSERT operation in HQL) are currently not supported.

  8. Overview of the IoTDB Full Backup Tool

    The IoTDB Full Backup Tool creates a full backup of a single IoTDB node's data. The backup is designed so that it can be directly started and joined to the original cluster.

    Critical Requirements:

    • You must stop the IoTDB service before starting the backup.
    • The script runs in the background by default; check the log files for execution progress.

  9. Overview of the IoTDB Stream Computing Framework

    V1.3.x

    The IoTDB stream processing framework allows users to implement customized stream processing logic to monitor storage engine changes, transform data, and push it to external systems. A stream processing task, called a 'Pipe', consists of three sequential subtasks implemented as plugins:

    1. Pipe Source: Extracts data.
    2. Pipe Processor: Processes/transforms data.
    3. Pipe Sink: Sends data to an external system.

    These tasks are configured declaratively using SQL statements to enable ETL capabilities for use cases such as edge-to-cloud synchronization, remote disaster recovery, and read/write load balancing.

  10. Overview of Flink-TsFile-Connector

    The Flink-TsFile-Connector enables Apache Flink to support external data sources of the TsFile type. It allows users to read and write TsFiles using the DataStream and DataSet APIs. Supported capabilities include loading a single TsFile, multiple TsFiles (DataSet API only), or all files within a specific directory from either the local file system or HDFS.

  11. Overview of AINode capabilities

    AINode is an endogenous node in IoTDB (alongside ConfigNode and DataNode) that extends the database's capability for machine learning analysis on time series. It integrates model management, training, and inference directly within the database engine. Users can perform time series analysis tasks (such as forecasting) using registered models via SQL statements or by registering custom machine learning models.

  12. Overview of Apache IoTDB core features

    V1.2.x

    Apache IoTDB is designed for time-series data from intelligent networking devices with the following key capabilities:

    • Storage Efficiency: High disk compression ratios to reduce hardware costs.
    • Data Organization: Efficient directory structures for complex time-series data, including support for devices of the same type and fuzzy searching across massive directories.
    • Performance: High-throughput read/write capabilities supporting millions of low-power device connections.
    • Query Capabilities: Supports time alignment across devices/sensors, frequency domain transformations in time-series fields, and rich aggregation functions in the time dimension.
    • Developer Experience: Uses a SQL-like language, JDBC standard API, and provides built-in import/export tools.
    • Ecosystem Integration: Integrates with Hadoop, Spark, and Grafana for analysis and visualization.
    • Unified Access: Eliminates the need for manual database partitioning or sharding and treats historical and real-time data uniformly.