DOCSEARCHbyAlgolia
Submit Docs

OpenAI Java SDK

repository·main·Indexed 23 days ago

https://github.com/openai/openai-java

A type-safe, fluent Java SDK for interacting with the OpenAI REST API. It supports the Responses and Chat Completions APIs, asynchronous execution via CompletableFuture, and streaming responses. The SDK includes features for structured outputs using Java classes, workload identity authentication for K8s, Azure, and GCP, and a dedicated artifact for OpenAI-compatible APIs on Amazon Bedrock.

Tokens
22.2K
Snippets
38
Records
87
Agent score
79%

What's inside openai-java

  1. Understand the OpenAI Java SDK version support policy

    main

    The OpenAI Java SDK follows a strict version support policy regarding JVM versions, framework integrations (like Spring Boot), and dependencies. Compatibility is declared per published artifact.

    Key principles:

    • Framework-neutral core: The core SDK evolves independently from optional integrations. An integration may require a newer JVM only if it uses a separate, generation-specific artifact.
    • Compatibility boundaries: Raising an artifact's JVM/API floor, changing its framework generation, or removing an artifact is treated as a Major version change.
    • Dependency ownership: OpenAI owns core implementation dependencies. Consumers own their application platform. For optional integrations, OpenAI declares and tests a range, but consumers choose the specific version through their platform.

  2. Define JSON schema properties from Java

    main

    When deriving a schema from Java classes, the following rules apply:

    • Inclusion: All public fields or public getter methods are included by default.
    • Exclusion: Non-public fields/getters are excluded unless annotated with @JsonProperty.
    • Getter Naming: If you use a custom name for a getter (not following the get prefix convention), you must annotate it with @JsonProperty to ensure the correct property name is used in the schema.
    • Required Fields: OpenAI requires all properties in a schema to be marked as required. The SDK automatically respects this and ignores any @JsonProperty(required = false) annotations.
    • Maps: Map types are treated as separate classes with no named properties, resulting in an empty "properties" field. To include arbitrary key/value data, model it as a List of entry objects with named fields.

  3. Perform manual pagination

    main

    If you need fine-grained control over when to fetch the next page, use the manual pagination methods on the page object:

    • items(): Returns the items in the current page.
    • hasNextPage(): Returns true if there are more pages available.
    • nextPage(): Fetches and returns the next page object.
    JobListPage page = client.fineTuning().jobs().list();
while (true) {
    for (FineTuningJob job : page.items()) {
        System.out.println(job);
    }

    if (!page.hasNextPage()) {
        break;
    }

    page = page.nextPage();
}

  4. Accumulate streamed responses into full objects

    main

    If you need to process a stream but eventually want the full response object (e.g., a ChatCompletion or Response), use the provided accumulator helpers:

    • ChatCompletionAccumulator: Records ChatCompletionChunk objects and can be used to reconstruct a full ChatCompletion object.
    • ResponseAccumulator: Records ResponseStreamEvent objects and can be used to reconstruct a full Response object.

    For synchronous streams, use .peek(accumulator::accumulate) within your stream pipeline. For asynchronous streams, call .accumulate(chunk) inside the .subscribe() callback.

    // Accumulating Chat Completions (Synchronous)
ChatCompletionAccumulator chatCompletionAccumulator = ChatCompletionAccumulator.create();

try (StreamResponse<ChatCompletionChunk> streamResponse = client.chat().completions().createStreaming(params)) {
    streamResponse.stream()
            .peek(chatCompletionAccumulator::accumulate)
            .forEach(chunk -> {
                // process chunks...
            });
}
ChatCompletion chatCompletion = chatCompletionAccumulator.chatCompletion();

  5. Understand SDK Immutability and Builders

    main

    All classes in the SDK are immutable once constructed. To create objects, use the provided builder or factory methods.

    If you need to create a modified copy of an existing immutable object, use its toBuilder() method to convert it back into a builder, apply your changes, and rebuild it.

  6. Why the SDK uses JsonField and avoids Enums/Data Classes

    main

    The SDK design prioritizes forward compatibility with the OpenAI API:

    • JsonField<T> vs T: Allows the SDK to handle undocumented fields, represent the difference between a missing field and an explicit null, and perform lazy validation.
    • Avoiding enum: Standard Java enums break if the API introduces a new value. The SDK uses types that can accommodate new, unknown values without crashing.
    • Avoiding data classes: Adding new fields to a data class is a breaking change in Java. The SDK uses standard classes to allow for seamless field additions.
    • No Checked Exceptions: The SDK avoids checked exceptions to improve developer experience, compatibility with lambdas, and to prevent unnecessary verbosity.

  7. How the SDK artifacts are structured

    main

    The SDK is split into three layers to allow for different HTTP client requirements:

    1. openai-java-core: Contains the core logic and defines the OpenAIClient and OpenAIClientAsync interfaces. It has no dependency on OkHttp.
    2. openai-java-client-okhttp: A provider that implements the core interfaces using the OkHttp library. It provides OpenAIOkHttpClient.
    3. openai-java: The standard dependency that bundles both core and client-okhttp together.

    Use openai-java-core if you need to implement a completely custom HttpClient or if you want to avoid the OkHttp dependency.

  8. Iterate through paginated results with auto-pagination

    main

    The SDK provides an autoPager() method to simplify iterating through all items across multiple pages of results. This method automatically fetches subsequent pages as you consume the items.

    Synchronous Client

    When using the synchronous client, autoPager() returns an Iterable. You can use it in a standard for-each loop or convert it to a Java Stream.

    Asynchronous Client

    When using the asynchronous client, autoPager() returns an AsyncStreamResponse. You can subscribe to the stream using a simple lambda or a full AsyncStreamResponse.Handler to manage lifecycle events like onNext (for each item) and onComplete (to handle errors or successful completion).

    // Synchronous Example
JobListPage page = client.fineTuning().jobs().list();
for (FineTuningJob job : page.autoPager()) {
    System.out.println(job);
}

// Asynchronous Example
CompletableFuture<JobListPageAsync> pageFuture = client.async().fineTuning().jobs().list();
pageFuture.thenRun(page -> page.autoPager().subscribe(job -> {
    System.out.println(job);
}));

  9. Handle streaming responses

    main

    The SDK provides streaming methods (identified by the Streaming suffix, e.g., createStreaming) that return response chunks as they arrive. This is useful for SSE or JSONL responses.

    Synchronous Streaming

    For synchronous clients, streaming methods return a StreamResponse<T>. You should use a try-with-resources block to ensure the stream is closed properly.

    Asynchronous Streaming

    For asynchronous clients, streaming methods return an AsyncStreamResponse<T>. You can process chunks using .subscribe() with a lambda or a custom AsyncStreamResponse.Handler to manage onNext and onComplete (including error handling).

    // Synchronous streaming
try (StreamResponse<ChatCompletionChunk> streamResponse = client.chat().completions().createStreaming(params)) {
    streamResponse.stream().forEach(chunk -> {
        System.out.println(chunk);
    });
}

// Asynchronous streaming with Handler
client.async().chat().completions().createStreaming(params).subscribe(new AsyncStreamResponse.Handler<ChatCompletionChunk>() {
    @Override
    public void onNext(ChatCompletionChunk chunk) {
        System.out.println(chunk);
    }

    @Override
    public void onComplete(Optional<Throwable> error) {
        if (error.isPresent()) {
            throw new RuntimeException(error.get());
        }
    }
});

  10. Install the OpenAI Amazon Bedrock artifact

    main

    To use the OpenAI-compatible Amazon Bedrock Mantle endpoint, add the openai-java-bedrock artifact to your project. This artifact includes the necessary AWS SDK for Java 2.x dependencies and configures the client to sign requests with SigV4.

    implementation("com.openai:openai-java-bedrock:4.49.0")
    <dependency>
  <groupId>com.openai</groupId>
  <artifactId>openai-java-bedrock</artifactId>
  <version>4.49.0</version>
</dependency>

  11. Configure Azure OpenAI client

    main

    To use the library with Azure OpenAI, use the OpenAIOkHttpClient builder with Azure-specific credentials and endpoint configuration. You can also customize the AzureUrlPathMode via ClientOptions to control how deployment or model names are handled in the URL path.

    URL Path Modes:

    • AzureUrlPathMode.AUTO (Default): Automatically detects the path mode based on the base URL.
    • AzureUrlPathMode.LEGACY: Forces the deployment or model name into the path.
    • AzureUrlPathMode.UNIFIED: For newer endpoints ending in /openai/v1. Matches OpenAI behavior; AzureOpenAIServiceVersion becomes optional and the model is passed in the request object.
    OpenAIClient client = OpenAIOkHttpClient.builder()
        .fromEnv()
        .credential(BearerTokenCredential.create(AuthenticationUtil.getBearerTokenSupplier(
                new DefaultAzureCredentialBuilder().build(), "https://cognitiveservices.azure.com/.default")))
        .build();

  12. Use Structured Outputs with the Responses API

    main

    Structured Outputs are supported in the Responses API. Instead of responseFormat(Class<T>), use text(Class<T>) when building parameters.

    Two ways to configure:

    1. Directly via Builder: Use ResponseCreateParams.Builder.text(Class<T>). This automatically switches the builder type to StructuredResponseCreateParams.Builder.
    2. Via StructuredResponseTextConfig: Build a StructuredResponseTextConfig object. This allows you to configure verbosity before attaching it to the ResponseCreateParams using the text(StructuredResponseTextConfig) method.