DOCSEARCHbyAlgolia
Submit Docs

LinDB Documentation

website·Indexed 32 days ago

https://lindb.io/

LinDB is a distributed time-series database featuring a high-performance HTTP REST API (v1), a SQL-like query language called LinQL, and client libraries for Go and Java. The platform includes an admin console for cluster monitoring, data exploration, and node configuration, with deployment options for standalone, cluster, and Kubernetes modes.

Tokens
23.6K
Snippets
77
Records
172
Agent score
63%

What's inside LinDB

  1. Overview of LinDB distributed time series database

    LinDB is a high-performance open-source distributed time series database designed for high availability, query optimization, and horizontal scalability. It is specifically built for medium and large-scale Internet enterprises to handle metric storage and analysis under massive data volumes. It supports multiple replicas to ensure high availability (remaining operational with only one replica alive) and supports multi-IDC deployments allowing writes to any IDC and aggregation across multiple IDCs.

  2. Overview of LinDB Time-Series Database

    LinDB is an open-source, cloud-native, horizontally scalable, distributed time-series database designed for high-performance data ingestion and real-time analytics.

  4. Design an asynchronous write client for LinDB

    To improve writing performance, implement an asynchronous client using a buffered queue mechanism. The recommended architecture involves:

    1. A unified API that writes data into a DataPointBuffer queue.
    2. Two background threads: one to move data from DataPointBuffer to a SendBuffer, and another to consume SendBuffer and send data to the LinDB Broker via HTTP.
    3. A callback mechanism to monitor exceptions during asynchronous processing.

    Buffers should be flushed based on two constraints:

    • Time: Flush data if no new data arrives within a specific period to prevent stale data.
    • Size: Flush data once the buffer reaches a specific number of metrics to optimize batch size.

  5. Understand LinDB's Multichannel Replication Strategy

    LinDB uses a multichannel replication scheme optimized for high-write throughput and eventual consistency, rather than strict consensus protocols like Paxos or Raft. In this model, a write is considered successful once it is written to the Leader. To prevent data inconsistency and loss during Leader failover, LinDB assigns a unique write channel (WAL) to each node.

    Key characteristics:

    • Channel Ownership: Each node has its own dedicated replication channel (e.g., Node 1 has 1-WAL). Data in a specific channel always flows from the node that owns that channel to the followers.
    • Failover Handling: If Node 1 (Leader) fails and Node 2 is elected, the system switches to the 2-WAL channel. When Node 1 recovers, it continues replicating any remaining data in its 1-WAL channel to followers, while Node 2 supplements data from the 2-WAL channel to Node 1.
    • Consistency Trade-off: This approach allows for out-of-order data writes, which is acceptable for time series data. It avoids conflicts because only one node has authority over a specific channel.

  6. LinDB Architecture Overview

    LinDB utilizes a decoupled architecture that separates compute from storage to allow for independent horizontal scaling. The system is divided into three primary layers:

    1. Computation Layer: A stateless layer consisting of Broker nodes that handle client requests, load balancing, and query orchestration.
    2. Storage Layer: A stateful layer responsible for persisting data and indices, performing local filtering, and executing DDL tasks.
    3. MetaData Layer: Powered by ETCD, this layer manages cluster configuration, sharding details, and node status.

  7. Manage cluster metadata and configurations

    The Meta Data Management section of the Admin Console provides tools for:

    • Storage: Managing storage cluster configurations.
    • Database: Managing database configurations.
    • Explore: Browsing cluster scheduling information and the memory scheduling result status for each component node.

  8. Understand LinDB query architecture and node roles

    LinDB uses a distributed query architecture consisting of two primary roles:

    • Broker: Acts as the entry point. It receives user query requests, generates an execution plan based on the query statement, delivers the plan to the relevant Storage nodes, and aggregates the returned results into the final response.
    • Storage: Responsible for data-level operations, including data filtering, DownSampling, and basic atomic calculations (operator push-down).

    The general query sequence follows this pipeline: Query Language Plan $\rightarrow$ Filtering $\rightarrow$ Scan Time Series $\rightarrow$ Grouping (if required) $\rightarrow$ DownSampling $\rightarrow$ Aggregation $\rightarrow$ Functions $\rightarrow$ Expressions.

  9. Choose a network endpoint to connect to LinDB

    LinDB provides three primary network endpoints for interacting with the database. By default, all three operate on port 9000, though these ports can be configured.

    • Admin Console: Used for general administration, SQL SELECT queries, charting, self-monitoring, and metadata management.
    • Client Libraries: The highest performance method for inserting data and performing SQL SELECT queries.
    • HTTP REST API: Used for inserting data (via INSERT) and querying data (via SQL SELECT).

  10. Support for ARM64 architecture in lindb

    lindb provides official support and binary packages for ARM64 architecture across multiple operating systems, including Linux and MacOS (Darwin). CI support for ARM64 images was introduced in version v0.3.0.

  11. Install LinDB on Linux via binary package

    To install LinDB on Linux, download the appropriate architecture package (amd64 or arm64), extract the archive, and run the server in standalone mode.
    # 1. Download the package (example for amd64)
# wget https://github.com/lindb/lindb/releases/download/v0.3.1/lindb-v0.3.1-linux-amd64.tar.gz

# 2. Uncompress
tar -zxvf lindb-v0.3.1-linux-amd64.tar.gz

# 3. Run LinDB in standalone mode
./lind standalone run

  12. LinDB HTTP REST API Overview

    LinDB provides a public HTTP REST API for interacting with the system. The current stable version is v1, and all public API endpoints are prefixed with /api/v1/.