DOCSEARCHbyAlgolia
Submit Docs

Cohere Python SDK

repository·main·Indexed 18 days ago

https://github.com/cohere-ai/cohere-python

A unified interface to access Cohere's generative AI models across multiple cloud platforms, including Cohere's own platform, AWS (Bedrock, SageMaker), Azure, GCP, and Oracle Cloud Infrastructure (OCI). The SDK provides tools for chat (via ClientV2), streaming responses, audio transcription (via AudioClient), and asynchronous batch processing (via BatchesClient). Version 7.0.8.

Tokens
23K
Snippets
72
Records
90
Agent score
63%

What's inside cohere-python

  1. Install and use Cohere with Oracle Cloud Infrastructure (OCI)

    main

    To use Cohere models on Oracle Cloud Infrastructure, install the OCI-specific extra:

    pip install 'cohere[oci]'

    Then, use cohere.OciClient or cohere.OciClientV2. You must provide the oci_region and oci_compartment_id.

    import cohere

# Using OCI config file authentication (default: ~/.oci/config)
co = cohere.OciClient(
    oci_region="us-chicago-1",
    oci_compartment_id="ocid1.compartment.oc1...",
)

response = co.embed(
    model="embed-english-v3.0",
    texts=["Hello world"],
    input_type="search_document",
)

print(response.embeddings)

  2. Use the Cohere ClientV2 for Chat

    main

    To interact with Cohere models (like command-r-plus-08-2024), instantiate cohere.ClientV2() and use the .chat() method. You can provide your API key via the CO_API_KEY environment variable to avoid hardcoding it in your scripts.

    import cohere

co = cohere.ClientV2()

response = co.chat(
    model="command-r-plus-08-2024",
    messages=[{"role": "user", "content": "hello world!"}],
)

print(response)

  4. Migrate streaming usage to chat_stream and generate_stream

    main

    The streaming: boolean parameter is no longer supported in the new SDK. Instead, use the dedicated streaming methods:

    • Replace chat() with chat_stream()
    • Replace generate() with generate_stream()

    These methods automatically handle the streaming parameter in the underlying request.

    stream = co.chat_stream(
    message="Tell me a short story"
)

for event in stream:
    if event.event_type == "text-generation":
        print(event.text, end='')

  6. Replace deprecated num_workers with httpx_client configuration

    main

    The num_workers parameter in the Client constructor is deprecated. To control connection limits, pass a configured httpx.Client via the httpx_client parameter.

    import httpx
import cohere

limits = httpx.Limits(max_connections=10)
co = cohere.Client(httpx_client=httpx.Client(limits=limits))

  7. Understand the Connector object

    main
    A Connector is an abstraction used to integrate external data sources with the /chat endpoint. This allows for grounded generations where the model can provide citations to the data source. Connectors are identified by a unique id (automatically generated from the name upon registration) and can be configured with specific authentication methods and behavior settings.

  8. Initialize the AsyncCohere Client

    main

    For asynchronous applications, use the AsyncClient class. It follows a similar initialization pattern to the synchronous Client but uses httpx.AsyncClient and supports async with context management.

    Key initialization parameters:

    • api_key: Your Cohere API key.
    • httpx_client: An optional existing httpx.AsyncClient instance.
    • log_warning_experimental_features: Set to False to suppress warnings for experimental features.
    import asyncio
from cohere import AsyncClient

async def main():
    async with AsyncClient(api_key="YOUR_API_KEY") as client:
        # make async calls
        pass

asyncio.run(main())

  9. Initialize the Cohere Client

    main

    The Client class is the primary synchronous entrypoint for interacting with Cohere APIs. You can initialize it by providing an api_key. If no key is provided, the SDK will attempt to retrieve it from the CO_API_KEY or COHERE_API_KEY environment variables.

    Key initialization parameters:

    • api_key: Your Cohere API key (string or a callable returning a string).
    • base_url: The API base URL (defaults to CO_API_URL env var).
    • environment: The deployment environment (e.g., ClientEnvironment.PRODUCTION).
    • timeout: Request timeout in seconds.
    • max_retries: Number of retry attempts for failed requests.
    • log_warning_experimental_features: Set to False to suppress warnings when using experimental parameters like response_format.schema in chat methods.
    from cohere import Client

client = Client(api_key="YOUR_API_KEY")

# Using as a context manager to ensure httpx client is closed
with Client(api_key="YOUR_API_KEY") as client:
    # make calls
    pass

  10. Stream chat responses with chat_stream

    main

    For real-time text generation, use the chat_stream method. This returns an iterable of events. You can check for event.type == "content-delta" to access the incremental text chunks via event.delta.message.content.text.

    import cohere

co = cohere.ClientV2()

response = co.chat_stream(
    model="command-r-plus-08-2024",
    messages=[{"role": "user", "content": "hello world!"}],
)

for event in response:
    if event.type == "content-delta":
        print(event.delta.message.content.text, end="")

  11. Configure fine-tuning hyperparameters

    main

    When performing fine-tuning, you can use the Hyperparameters class to control the training process. This includes settings for standard training (epochs, batch size, learning rate), early stopping logic, and Low-Rank Adaptation (LoRA) specific parameters.

    Standard Training Parameters

    • train_batch_size (int): The number of training examples included in a single training pass.
    • train_epochs (int): The total number of epochs to train for.
    • learning_rate (float): The learning rate used during training.

    Early Stopping

    To prevent overfitting, you can configure early stopping:

    • early_stopping_patience (int): Stops training if the loss metric does not improve beyond the early_stopping_threshold after this many evaluation cycles.
    • early_stopping_threshold (float): The minimum amount the loss must improve to prevent early stopping.

    LoRA (Low-Rank Adaptation) Parameters

    If using LoRA, you can tune the following:

    • lora_alpha (int): Controls the scaling factor for LoRA updates. Higher values make updates more impactful.
    • lora_rank (int): Specifies the rank for low-rank matrices. Lower ranks reduce parameter count but may limit model flexibility.
    • lora_target_modules (LoraTargetModules): The specific combination of LoRA modules to target.
    from cohere.finetuning.finetuning.types import Hyperparameters

hyperparameters = Hyperparameters(
    train_batch_size=32,
    train_epochs=3,
    learning_rate=0.0001,
    lora_rank=8,
    lora_alpha=16,
    early_stopping_patience=2,
    early_stopping_threshold=0.01
)

  12. Removed functions in cohere v5

    main

    The following functions are no longer supported in the new SDK and have been removed:

    • check_api_key
    • loglikelihood
    • batch_generate
    • codebook
    • batch_tokenize
    • batch_detokenize
    • detect_language
    • generate_feedback
    • generate_preference_feedback
    • create_cluster_job
    • get_cluster_job
    • list_cluster_jobs
    • wait_for_cluster_job
    • create_custom_model
    • wait_for_custom_model
    • get_custom_model
    • get_custom_model_by_name
    • get_custom_model_metrics
    • list_custom_models