DOCSEARCHbyAlgolia
Submit Docs

Apache IoTDB Documentation

repository·master·Indexed 27 days ago

https://github.com/apache/iotdb

A high-performance, lightweight data management system designed for time series data in industrial IoT applications. This documentation covers Docker deployment (standalone, confignode, datanode, and ainode), MQTT service customization, JDBC and ODBC connectivity, C REST client implementation, and integration testing in simple and cluster modes.

Tokens
25.8K
Snippets
49
Records
172
Agent score
91%

What's inside Apache IoTDB

  2. Overview of Apache IoTDB

    master
    Apache IoTDB (Internet of Things Database) is a specialized data management system for time series data. It provides services for data collection, storage, and analysis. It is designed to be lightweight and high-performance, featuring seamless integration with the Hadoop and Spark ecosystems. IoTDB utilizes TsFile, a columnar storage file format optimized for time series data.

  3. Understand the Apache IoTDB Security Model and Trust Boundaries

    master

    Apache IoTDB is designed to be deployed as operator-managed infrastructure within a trusted network by default. Understanding the trust boundaries is critical for secure deployment:

    • Client RPC Surface (Primary Boundary): The main entry point for untrusted input. Clients connect via Thrift session protocol, JDBC, or SQL. Clients are considered untrusted and are constrained by Role-Based Access Control (RBAC) privileges.
    • Cluster/Inter-node Surface (Secondary Boundary): Communication between ConfigNodes and DataNodes (RPC and consensus channels) is assumed to occur on a trusted network. Currently, these channels have no transport encryption. Operators are responsible for network segmentation to protect these channels.
    • Operator/Admin: The root superuser and those controlling the deployment/host are considered trusted.
    • Peer Nodes: Other nodes in the cluster are trusted to the extent the cluster security model trusts its members.

  4. Core Concepts of the IoTDB Consensus Layer

    master

    The Consensus Layer provides an abstraction for managing multiple copies of application data to ensure fault tolerance and data integrity. It hides the complexities of specific consensus algorithms (like Raft) behind a unified interface.

    Key Components:

    • IStateMachine: The user application component that manages a local copy of the data.
    • Peer: The smallest consensus unit within a process, which contains an IStateMachine.
    • ConsensusGroup: A collection of Peer instances that all manage the same copy of data.
    • IConsensus: The interface defining the core functionality of the Consensus Layer.
    • ConsensusFactory: The entry point used to instantiate specific consensus implementations.

    Data Flow:

    When writing data to a ConsensusGroup via IConsensus::write, the operation is sent to the group leader's IStateMachine::write. The leader decides on the write, applies it to its local state machine, and forwards the operation to other members in the group.

  5. Select and extract the correct Apache IoTDB C++ SDK package

    master

    Choose the pre-built SDK zip file based on your target environment's operating system and glibc/MSVC version. Once downloaded, unzip it and set the IOTDB_SESSION_HOME environment variable to the extraction directory to simplify integration.

    Available Classifiers:

    • Linux x86_64 (glibc >= 2.28): linux-x86_64-glibc2.28
    • Linux aarch64 (glibc >= 2.28): linux-aarch64-glibc2.28
    • macOS x86_64: macos-x86_64
    • macOS arm64: macos-aarch64
    • Windows + VS 2017: windows-x86_64-msvc14.1
    • Windows + VS 2019: windows-x86_64-msvc14.2
    • Windows + VS 2022: windows-x86_64-msvc14.3
    • Windows + VS 2026: windows-x86_64-msvc14.4

    The SDK includes include/ (public API headers), lib/ (shared/dynamic libraries), cmake/ (package config), and pkgconfig/ (metadata). Thrift and Boost are already encapsulated and do not require separate installation.

    unzip iotdb-session-cpp-2.0.11-SNAPSHOT-linux-x86_64-glibc2.28.zip
export IOTDB_SESSION_HOME=$PWD/iotdb-session-cpp-2.0.11-SNAPSHOT-linux-x86_64-glibc2.28

  7. Build the Apache IoTDB C++ Client from source

    master

    You can build the C++ client using either Maven (as a wrapper) or standalone CMake.

    Using Maven

    Use Maven to build the SDK and package it into a zip file. The Maven build sets the installation prefix to target/install/.

    Linux/macOS (Library only):

    mvn -P with-cpp -pl iotdb-client/client-cpp -am -DskipTests package

    Windows (MSVC): You must provide the path to your Boost installation.

    mvn -P with-cpp -pl iotdb-client/client-cpp -am -DskipTests "-Dboost.include.dir=C:\boost_1_88_0" package

    Using Standalone CMake

    If Maven is not available, use CMake directly.

    Linux/macOS:

    cmake -S iotdb-client/client-cpp -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --target install

    Windows (Visual Studio):

    cmake -S iotdb-client/client-cpp -B build -G "Visual Studio 17 2022" -A x64
cmake --build build --config Release --target install

  8. Build the Apache IoTDB C++ SDK from source

    master

    You can build the C++ SDK using Maven or directly via CMake.

    Using Maven

    To build the package using Maven, use the with-cpp profile. To perform an offline build, set iotdb.offline=ON and ensure all required platform-specific tarballs are pre-populated in the third-party/ sub-directory.

    Using CMake

    To build using CMake, you can use the IOTDB_OFFLINE flag for offline builds.

    Offline Build Requirements

    Before building offline, you must manually place the following files in the third-party/ directory corresponding to your platform:

    • Linux: thrift-0.23.0.tar.gz, boost_1_60_0.tar.gz, m4-1.4.19.tar.gz, flex-2.6.4.tar.gz, bison-3.8.tar.gz, and optionally openssl-3.5.0.tar.gz.
    • macOS: thrift-0.23.0.tar.gz, boost_1_84_0.tar.gz, and optionally openssl-3.5.0.tar.gz.
    • Windows: thrift-0.23.0.tar.gz, boost_1_60_0.tar.gz, and optionally openssl-3.5.0.tar.gz.
    # Maven offline build
mvn -P with-cpp -pl iotdb-client/client-cpp -am -DskipTests \
    -Diotdb.offline=ON package

# CMake offline build
cmake -S iotdb-client/client-cpp -B build -DIOTDB_OFFLINE=ON
cmake --build build --config Release --target install

  9. Run Integration Tests in Simple Consensus Mode

    master

    Integration tests in Simple mode (1 config node and 1 data node running tree model ITs) can be executed via Maven.

    To run only the integration tests, use the following command:

    mvn clean verify -DskipUTs -pl integration-test -am -P with-integration-tests

    Note for IDE users (IntelliJ): If running in an IDE for the first time or after changing dependent module code, you must first generate the integration-test/target/template-node directory using this command:

    mvn clean package -DskipTests -pl integration-test -am -P with-integration-tests

    Once generated, you can run individual test cases directly from the IDE.

  10. Register an AINode to an existing IoTDB cluster

    master

    To add an AINode to an existing cluster, you can use Docker Compose or a docker run command with specific environment variables to point to the ConfigNode and cluster ingress settings.

    # Using Docker Compose (requires docker-compose-ainode.yml)
docker compose -f docker-compose-ainode.yml up -d

# Using Docker Run (v2.0.7+)
docker run -d \
  --name iotdb-ainode \
  --network host \
  -p 10810:10810 \
  -p 8080:8080 \
  -e AIN_SEED_CONFIG_NODE=127.0.0.1:10710 \
  -e AIN_RPC_ADDRESS=127.0.0.1 \
  -e AIN_RPC_PORT=10810 \
  -e AIN_CLUSTER_INGRESS_ADDRESS=127.0.0.1 \
  -e AIN_CLUSTER_INGRESS_PORT=6667 \
  -e AIN_CLUSTER_INGRESS_USERNAME=root \
  -e AIN_CLUSTER_INGRESS_PASSWORD=root \
  apache/iotdb:2.0.11-SNAPSHOT-ainode

  11. Migrate RPC Schema from 0.8.x to 0.9.x (version-1)

    master

    When upgrading from IoTDB 0.8.x to 0.9.x (version-1), note the following API additions and removals:

    New Methods and Structs

    • Batch Insertion: Added TSBatchInsertionReq and the insertBatch method (returning TSExecuteBatchStatementResp).
    • Timeseries Management: Added TSCreateTimeseriesReq and the createTimeseries method. Also added TSCreateMultiTimeseriesReq and createMultiTimeseries.
    • Data Operations: Added TSInsertReq and the insertRow method. Added TSDeleteDataReq and the deleteData method. Added deleteTimeseries and deleteStorageGroups methods.
    • Status: Added TSStatusType.

    Removals

    • TSSetStorageGroupReq has been removed.
    • TSDataValue and TSRowRecord have been removed.
    • Several optional fields in TSFetchMetadataResp have been removed, including version, childPaths, nodesList, storageGroups, devices, nodeTimeseriesNum, timeseriesList, and timeseriesNum.

  12. Migrate RPC Schema from 0.9.x (version-1) to 0.10.x (version-2)

    master

    When upgrading from IoTDB 0.9.x (version-1) to 0.10.x (version-2), note these significant API renames and changes:

    API Renames

    • insertRows $\rightarrow$ insertRecords
    • insert $\rightarrow$ insertRecord
    • insertBatch $\rightarrow$ insertTablet
    • TSStatusType $\rightarrow$ TSStatus
    • TS_SessionHandles $\rightarrow$ SessionIds
    • TSOperationHandle $\rightarrow$ queryIds

    Data Type Changes

    • In TSInsertInBatchReq and TSInsertReq, use TsDataType and binary instead of string.

    New Features and Updates

    • Session IDs: sessionId is now a required parameter in getTimeZone, getProperties, setStorageGroup, and createTimeseries.
    • Non-Aligned Data: TSExecuteStatementResp and TSFetchResultsResp now include an optional TSQueryNonAlignDataSet. TSFetchResultsReq now requires a bool isAlign field.
    • Tablet Insertion: Added TSInsertTabletsReq and the insertTablets method.
    • Record Insertion: Added inferType field to TSInsertRecordReq.

    Removals

    • TS_SessionHandle and TSHandleIdentifier have been removed.
    • TSStatus and TSExecuteInsertRowInBatchResp have been removed.