Groq Python API Library

In API responses, a field that is explicitly null or missing entirely will both result in None in Python. To distinguish between them, check the .model_fields_set attribute on the response object. If the field name is not in .model_fields_set, the key was missing from the JSON; otherwise, it was explicitly set to null.

if response.my_field is None:
  if 'my_field' not in response.model_fields_set:
    print('Got json like {}, without a "my_field" key present at all.'.format(response))
  else:
    print('Got json like {"my_field": null}.')

Install the Groq client from PyPI using pip. The library requires Python 3.10 or higher.

pip install groq

While the library closes connections upon garbage collection, it is best practice to manually manage the lifecycle. Use a context manager to ensure the client and its underlying connections are closed correctly when exiting the block.

from groq import Groq

with Groq() as client:
  # make requests here
  ...

# HTTP client is now closed

The library uses the standard Python logging module. You can enable logging by setting the GROQ_LOG environment variable to info or debug (for more verbose output).

$ export GROQ_LOG=info

By default, the async client uses httpx. To use aiohttp for improved concurrency performance, install the aiohttp extra and pass DefaultAioHttpClient() to the http_client parameter.

pip install groq[aiohttp]

import os
import asyncio
from groq import DefaultAioHttpClient
from groq import AsyncGroq

async def main() -> None:
    async with AsyncGroq(
        api_key=os.environ.get("GROQ_API_KEY"),
        http_client=DefaultAioHttpClient(),
    ) as client:
        chat_completion = await client.chat.completions.create(
            messages=[
                {
                    "role": "user",
                    "content": "Explain the importance of low latency LLMs",
                }
            ],
            model="openai/gpt-oss-20b",
        )
        print(chat_completion.id)

asyncio.run(main())

You can customize the underlying httpx client by passing a DefaultHttpxClient to the Groq constructor. This allows for configuring proxies, custom transports, and base URLs. You can also apply these configurations to a specific client instance using .with_options().

import httpx
from groq import Groq, DefaultHttpxClient

client = Groq(
    # Or use the `GROQ_BASE_URL` env var
    base_url="http://my.test.server.example.com:8083",
    http_client=DefaultHttpxClient(
        proxy="http://my.test.proxy.example.com",
        transport=httpx.HTTPTransport(local_address="0.0.0.0"),
    ),
)

# Per-request customization
client.with_options(http_client=DefaultHttpxClient(...))

The status field in a BatchRetrieveResponse indicates the current lifecycle stage of a batch job. You can use this to determine if a batch is ready for processing or if you need to inspect error files.

Valid status values:

• validating: The batch is being checked for validity.
• failed: The batch job encountered an error.
• in_progress: The batch is currently being processed.
• finalizing: The batch is in the final stages of processing.
• completed: The batch has finished successfully.
• expired: The batch has exceeded its processing window.
• cancelling: A cancellation request is in progress.
• cancelled: The batch was successfully cancelled.

To use the synchronous client, import Groq and instantiate it. By default, it looks for the GROQ_API_KEY environment variable. You can also provide the api_key explicitly as a keyword argument.

import os
from groq import Groq

client = Groq(
    api_key=os.environ.get("GROQ_API_KEY"),  # This is the default and can be omitted,
)

chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Explain the importance of low latency LLMs",
        }
    ],
    model="openai/gpt-oss-20b",
)
print(chat_completion.choices[0].message.content)

When performing file uploads (e.g., for audio transcriptions), you can pass the file as bytes, a PathLike instance, or a tuple of (filename, contents, media type). If using a PathLike instance with the async client, the file is read asynchronously automatically.

from pathlib import Path
from groq import Groq

client = Groq()

client.audio.transcriptions.create(
    model="whisper-large-v3-turbo",
    file=Path("/path/to/file"),
)