DOCSEARCHbyAlgolia
Submit Docs

Ollama JavaScript Library

repository·main·Indexed 26 days ago

https://github.com/ollama/ollama-js

A lightweight JavaScript client for the Ollama API. It supports local model execution, streaming responses, and cloud-based model offloading. Key features include text generation via `generate()`, multi-turn conversations with `chat()`, model management (`pull`, `push`, `copy`, `delete`), embedding generation, and cloud capabilities like `webSearch` and `webFetch`. The library provides support for both browser and Node.js environments, including multimodal image encoding and experimental image generation on macOS.

Tokens
3.3K
Snippets
7
Records
20
Agent score
88%

What's inside ollama-js

  1. Run Ollama JS examples

    main

    To run the provided examples in the repository, you must first ensure the project is built. Use npx tsx to execute the TypeScript files directly.

    Prerequisites:

    1. Run npm run build to compile the project.
    2. Use the following command structure to run a specific example: npx tsx <folder-name>/<file-name>.ts
    npx tsx <folder-name>/<file-name>.ts

  2. Use experimental Image Generation

    main

    The library includes experimental support for image generation.

    Limitations:

    • This feature is currently only available on macOS.
    • It is marked as experimental and may change.

    You can find the implementation details in image-generation/image-generation.ts.

  3. Initialize the Ollama client for browser usage

    main

    To use the Ollama library in a browser environment, instantiate the Ollama class. You can provide a Config object to specify the host, custom headers, or a custom fetch implementation. If no host is provided, it defaults to the standard Ollama host.

    Note: The library uses whatwg-fetch to ensure compatibility in browser environments.

  4. Configure the Ollama client

    main

    When initializing an Ollama client, you can provide a Config object to customize the connection.

    Key configuration options:

    • host: The base URL of the Ollama server.
    • fetch: A custom fetch implementation (useful for polyfills or specialized environments).
    • proxy: A boolean to enable proxy support.
    • headers: Custom HeadersInit to include with requests.
    interface Config {
  host: string
  fetch?: Fetch
  proxy?: boolean
  headers?: HeadersInit
}

  5. Initialize the Ollama client

    main
    The ollama-js library provides a default instance of the Ollama class that can be imported directly. This client is used to interact with the Ollama API, supporting both local Ollama instances and cloud models. It inherits capabilities from OllamaBrowser but adds Node.js-specific features like file system access for image encoding.

  6. List and inspect models with `list`, `ps`, and `show`

    main

    Retrieve information about models on the server:

    • list(): Returns a list of all models available on the server (ListResponse).
    • ps(): Returns a list of models currently running on the server (ListResponse).
    • show(request: ShowRequest): Returns metadata for a specific model (ShowResponse).

  7. Manage models with `pull`, `push`, `copy`, and `delete`

    main

    Use these methods to manage the model library on your Ollama server:

    • pull(request: PullRequest): Downloads a model from the registry. Supports stream: true to track progress via AbortableAsyncIterator<ProgressResponse>.
    • push(request: PushRequest): Uploads a model to the registry. Supports stream: true for progress tracking.
    • copy(request: CopyRequest): Creates a copy of a model with a new name. Returns Promise<StatusResponse>.
    • delete(request: DeleteRequest): Removes a model from the server. Returns Promise<StatusResponse>.

  8. Use Ollama Cloud APIs: `webSearch` and `webFetch`

    main

    The client provides access to Ollama's cloud-based web capabilities:

    • webSearch(request: WebSearchRequest): Performs a web search using the Ollama web search API. Requires a query.
    • webFetch(request: WebFetchRequest): Fetches content from a specific URL using the Ollama web fetch API. Requires a url.

  9. Generate text responses with `generate()`

    main

    The generate method creates a response from a text prompt. It supports both streaming and non-streaming modes.

    • Non-streaming: Returns a Promise<GenerateResponse>.
    • Streaming: Set stream: true in the request to return an AbortableAsyncIterator<GenerateResponse>.

    If images are provided in the request, they can be passed as Uint8Array or base64 strings; the client will automatically encode Uint8Array inputs to base64.

  10. Abort ongoing streamed requests

    main
    If you have active streamed requests (e.g., from chat, generate, pull, or push), you can cancel all of them simultaneously by calling the abort() method on your Ollama instance. This uses an internal AbortController for each active stream.

  11. Generate embeddings with EmbedRequest

    main

    Use EmbedRequest to generate vector embeddings for text input.

    Key Fields:

    • model: The name of the embedding model.
    • input: A single string or an array of strings to embed.
    • dimensions: The desired dimensionality of the output embeddings.
    • truncate: Whether to truncate input if it exceeds context length.
    • options: A Partial<Options> object.
    export interface EmbedRequest {
  model: string
  input: string | string[]
  truncate?: boolean
  keep_alive?: string | number
  dimensions?: number
  options?: Partial<Options>
}

  12. Chat with models using ChatRequest

    main

    Use ChatRequest for conversational interactions. It accepts a list of Message objects.

    Message Structure:

    • role: The role of the message (e.g., 'user', 'assistant', 'system').
    • content: The text content.
    • images: Optional multimodal input.
    • tool_calls: Optional tool calls returned by the assistant.

    ChatRequest Fields:

    • model: The name of the model.
    • messages: An array of Message objects.
    • tools: An array of Tool definitions for function calling.
    • stream: Whether to stream the response.
    • options: A Partial<Options> object.
    export interface Message {
  role: string
  content: string
  thinking?: string
  images?: Uint8Array[] | string[]
  tool_calls?: ToolCall[]
  tool_name?: string
}

export interface ChatRequest {
  model: string
  messages?: Message[]
  stream?: boolean
  format?: string | object
  keep_alive?: string | number
  tools?: Tool[]
  think?: boolean | 'high' | 'medium' | 'low'
  logprobs?: boolean
  top_logprobs?: number
  options?: Partial<Options>
}