Milvus Java SDK

repository·master·Indexed 19 days ago

https://github.com/milvus-io/milvus-sdk-java

A client library for interacting with the Milvus vector database, enabling Java applications to perform vector searches and data management. The SDK is available as a core package (milvus-sdk-java) and an optional BulkWriter package (milvus-sdk-java-bulkwriter). It supports Milvus server versions from 2.0 to 3.0.x and provides V1 and V2 APIs, along with performance benchmarking tools for measuring search, query, and insert latencies.

Tokens
3.6K
Snippets
13
Records
17
Agent score
67%

What's inside milvus-sdk-java

  1. Explore Milvus Java SDK example types

    master

    The examples project is organized by SDK version and specific feature sets:

    • V2 SDK Examples: Located in src/main/java/io/milvus/v2/. These use the MilvusClientV2 API.
    • V1 SDK Examples: Located in src/main/java/io/milvus/v1/. These use the MilvusServiceClient API.
    • BulkWriter Examples: Located in src/main/java/io/milvus/v2/bulkwriter/.

    Important: Some examples (like BulkWriter or external table examples) may require additional services such as MinIO. Always check the comments at the top of each individual example file for specific environmental requirements.

  2. Install the Milvus Java SDK

    master

    Add the Milvus Java SDK to your project using Apache Maven or Gradle.

    Starting from version 2.5.2, the SDK is split into two packages:

    1. milvus-sdk-java: The core SDK for standard operations.
    2. milvus-sdk-java-bulkwriter: Required only if you need the BulkWriter tool, as it has a larger dependency footprint.

    If you do not require BulkWriter, you only need to install milvus-sdk-java.

    ### Apache Maven
    ```xml
    <dependency>
        <groupId>io.milvus</groupId>
        <artifactId>milvus-sdk-java</artifactId>
        <version>3.0.5</version>
    </dependency>

    Gradle (Groovy)

    implementation 'io.milvus:milvus-sdk-java:3.0.5'

    Gradle (Kotlin)

    implementation("io.milvus:milvus-sdk-java:3.0.5")
  3. Set up the Milvus Java SDK development environment

    master

    To develop for the Milvus Java SDK, ensure you have Java 8 or higher and Apache Maven installed. You must clone the repository recursively to include the necessary Milvus proto files located in the sdk-core/src/main/milvus-proto submodule.

    If you have already cloned the repository without submodules, you must initialize them manually.

    # Clone the repository with submodules
    $ git clone --recursive git@github.com:milvus-io/milvus-sdk-java.git
    
    # If you already cloned without submodules, run this:
    $ git submodule update --init
  4. Run Milvus Java SDK performance benchmarks

    master

    The benchmark project provides tools to measure the performance of the Milvus Java SDK. You can run all benchmarks across multiple SDK versions using the run_benchmarks.sh script, or run specific benchmark classes using Maven.

    Prerequisites

    • Java 8+
    • Maven 3.x (must be in PATH)
    • A running Milvus instance (default: localhost:19530)

    Quick Start

    To run all benchmarks for the default SDK versions (2.6.1 and 2.6.14) against a local Milvus instance:

    cd benchmark
    ./run_benchmarks.sh

    Results are saved as timestamped markdown files in the results/ directory.

  5. Build the Milvus Java SDK

    master

    Use Apache Maven to build the project. Running the install command will also trigger the generation of protobuf-related code required for the SDK to function.

    If using IntelliJ IDEA, after cloning, right-click the project root and select Maven -> Reload Project to sync dependencies.

    $ mvn install
  6. Build the Milvus Java SDK Examples project

    master

    The examples project is a standalone Maven project. To compile the examples, navigate to the examples directory and use Maven.

    Prerequisites:

    • Java 8 or higher
    • Apache Maven
    • A running Milvus instance (default: localhost:19530)

    Note: This project depends on milvus-sdk-java and milvus-sdk-java-bulkwriter being present in your local Maven repository. If you are developing the SDK itself and have made local changes, you must run mvn install -DskipTests from the SDK project root before building the examples.

    cd examples
    mvn compile
  7. Install the milvus-sdk-java-bulkwriter package

    master

    If your application requires the BulkWriter tool, you must explicitly add the milvus-sdk-java-bulkwriter dependency to your project.

    ### Apache Maven
    ```xml
    <dependency>
        <groupId>io.milvus</groupId>
        <artifactId>milvus-sdk-java-bulkwriter</artifactId>
        <version>3.0.5</version>
    </dependency>

    Gradle (Groovy)

    implementation 'io.milvus:milvus-sdk-java-bulkwriter:3.0.5'

    Gradle (Kotlin)

    implementation("io.milvus:milvus-sdk-java-bulkwriter:3.0.5")
  8. Use the run_benchmarks.sh script with options

    master

    The run_benchmarks.sh script allows you to control which SDK versions and Milvus instances are used for the full benchmark suite.

    Command Line Options

    OptionDescriptionDefault
    -v, --versionsComma-separated SDK versions2.6.1,2.6.14
    -u, --uriMilvus server URIhttp://localhost:19530
    -t, --tokenAuthentication tokenroot:Milvus
    -h, --helpShow help message-

    Examples

    Single SDK version:

    ./run_benchmarks.sh -v 2.6.14

    Multiple SDK versions:

    ./run_benchmarks.sh -v 2.6.1,2.6.13,2.6.14

    Custom URI and Token:

    ./run_benchmarks.sh -v 2.6.1,2.6.14 -u http://myhost:19530 -t root:Milvus
    # Single SDK version
    ./run_benchmarks.sh -v 2.6.14
    
    # Multiple SDK versions (comma-separated)
    ./run_benchmarks.sh -v 2.6.1,2.6.13,2.6.14
    
    # With a custom Milvus URI
    ./run_benchmarks.sh -v 2.6.14 -u http://myhost:19530
    
    # With URI and token
    ./run_benchmarks.sh -v 2.6.1,2.6.14 -u http://myhost:19530 -t root:Milvus
  9. Run individual benchmark classes via Maven

    master

    You can run specific benchmark classes (like PoolBenchmark or SearchBenchmark) directly using mvn exec:java.

    Configuration via System Properties

    • -Drevision: Controls the SDK version to use (default: 2.6.14).
    • -Dexec.args: Accepts two positional arguments:
      1. URI: Milvus server address (default: http://localhost:19530).
      2. Token: Authentication token (default: root:Milvus).

    Always use clean (e.g., mvn clean compile) to ensure previous SDK version classes are cleared before compiling against a new version.

  10. Configure SearchBenchmark via JSON

    master

    You can customize SearchBenchmark parameters by creating a JSON file at config/SearchBenchmark.json. If the file is missing, the benchmark uses hardcoded defaults.

    Configuration Keys

    KeyDefaultDescription
    floatVectorDim768Float vector dimension
    binaryVectorDim2048Binary vector dimension
    float16VectorDim768Float16 vector dimension
    rowCount100000Number of rows to insert
    repeat10Number of iterations per search variant

    Example Config

    {
      "floatVectorDim": 768,
      "binaryVectorDim": 2048,
      "float16VectorDim": 768,
      "rowCount": 100000,
      "repeat": 10
    }
  11. Configure PoolBenchmark via JSON

    master

    You can customize PoolBenchmark parameters by creating a JSON file at config/PoolBenchmark.json. If the file is missing, the benchmark uses hardcoded defaults.

    Configuration Keys

    KeyDefaultDescription
    vectorDim128Vector dimension
    rowCount100000Number of base rows to insert
    threadCount32Number of concurrent threads
    requestCount10000Number of requests per benchmark
    topK10Search top-K parameter
    rounds10Number of benchmark rounds
    maxIdlePerKey10Pool: max idle clients per key
    maxTotalPerKey20Pool: max total clients per key
    maxTotal100Pool: max total clients across all keys
    maxBlockWaitSeconds30Pool: max seconds to wait when pool is exhausted

    Example Config

    {
      "vectorDim": 128,
      "rowCount": 100000,
      "threadCount": 32,
      "requestCount": 10000,
      "topK": 10,
      "rounds": 10,
      "maxIdlePerKey": 10,
      "maxTotalPerKey": 20,
      "maxTotal": 100,
      "maxBlockWaitSeconds": 30
    }
  12. Fix java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory

    master

    If you encounter java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory, it means the SLF4J API is missing from your application's classpath. The Milvus Java SDK requires SLF4J for logging.

    To resolve this, add the slf4j-api dependency to your project.

    ### Apache Maven
    ```xml
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-api</artifactId>
        <version>1.7.36</version>
    </dependency>

    Gradle (Groovy)

    implementation 'org.slf4j:slf4j-api:1.7.36'

    Gradle (Kotlin)

    implementation("org.slf4j:slf4j-api:1.7.36")