Groq TypeScript SDK

The library provides mechanisms to manage request resilience:

Retries

Certain errors (Connection errors, 408 Request Timeout, 409 Conflict, 429 Rate Limit, and >=500 Internal errors) are automatically retried 2 times by default using exponential backoff. Use maxRetries to change this behavior.

Timeouts

Requests time out after 1 minute by default. On timeout, an APIConnectionTimeoutError is thrown. Use timeout to adjust this. Note that timed-out requests are also subject to the default retry logic.

You can configure these globally in the Groq constructor or override them on a per-request basis.

// Global configuration
const client = new Groq({
  maxRetries: 0, // default is 2
  timeout: 20 * 1000, // 20 seconds (default is 1 minute)
});

// Per-request override
await client.chat.completions.create(
  { 
    messages: [{ role: 'user', content: '...' }], 
    model: 'openai/gpt-oss-20b' 
  }, 
  { 
    maxRetries: 5, 
    timeout: 5 * 1000 
  }
);

If you need to access endpoints, parameters, or response properties not yet included in the typed SDK, you can bypass the type safety:

1. Undocumented Endpoints: Use HTTP verb methods directly on the client (e.g., client.post('/path', { ... })). Client options like retries are still respected.
2. Undocumented Parameters: Pass extra properties to existing methods and use // @ts-expect-error to suppress TypeScript errors. For GET requests, extra params are sent as query strings; for other verbs, they are sent in the body.
3. Undocumented Response Properties: Access properties on the returned object using // @ts-expect-error or by casting to a custom type.

// Undocumented endpoint
await client.post('/some/path', {
  body: { some_prop: 'foo' },
  query: { some_query_arg: 'bar' },
});

// Undocumented parameter
client.chat.completions.create({
  // ...
  // @ts-expect-error baz is not yet public
  baz: 'undocumented option',
});

When a request parameter requires a file upload, you can provide the file in several formats:

• A web File object.
• A fetch Response object.
• A Node.js fs.ReadStream (recommended for Node environments).
• The return value of the toFile helper (useful for Buffer or Uint8Array).

import fs from 'fs';
import Groq, { toFile } from 'groq-sdk';

const client = new Groq();

// Using Node.js fs.createReadStream()
await client.audio.transcriptions.create({
  model: 'whisper-large-v3-turbo',
  file: fs.createReadStream('/path/to/file'),
});

// Using web File API
await client.audio.transcriptions.create({
  model: 'whisper-large-v3-turbo',
  file: new File(['my bytes'], 'file'),
});

// Using fetch Response
await client.audio.transcriptions.create({
  model: 'whisper-large-v3-turbo',
  file: await fetch('https://somesite/file'),
});

// Using toFile helper for Buffers or Uint8Arrays
await client.audio.transcriptions.create({
  model: 'whisper-large-v3-turbo',
  file: await toFile(Buffer.from('my bytes'), 'file'),
});

await client.audio.transcriptions.create({
  model: 'whisper-large-v3-turbo',
  file: await toFile(new Uint8Array([0, 1, 2]), 'file'),
});

The SDK uses a global fetch by default. You can provide a custom fetch implementation or configure fetchOptions (which accept RequestInit options) during client instantiation or per request.

Configuring Proxies via fetchOptions:

• Node.js (using undici): Use the dispatcher key.
• Bun: Use the proxy key.
• Deno: Use the client key with a Deno.HttpClient.

// Node.js Proxy Example
import Groq from 'groq-sdk';
import * as undici from 'undici';

const proxyAgent = new undici.ProxyAgent('http://localhost:8888');
const client = new Groq({
  fetchOptions: {
    dispatcher: proxyAgent,
  },
});

// Bun Proxy Example
const clientBun = new Groq({
  fetchOptions: {
    proxy: 'http://localhost:8888',
  },
});

// Deno Proxy Example
import { Groq } from 'npm:groq-sdk';
const httpClient = Deno.createHttpClient({ proxy: { url: 'http://localhost:8888' } });
const clientDeno = new Groq({
  fetchOptions: {
    client: httpClient,
  },
});