DOCSEARCHbyAlgolia
Submit Docs

replicate-python

repository·main·Indexed 21 days ago

https://github.com/replicate/replicate-python

Python client for Replicate (v1.0.7) providing a high-level interface to run machine learning models. It supports synchronous and asynchronous execution, streaming outputs for LLMs, background processing, model pipelining, and fine-tuning via replicate.trainings.create(). The library includes a FileOutput object for streaming model results and provides namespaces for managing accounts, collections, deployments, hardware, and predictions.

Tokens
17.5K
Snippets
69
Records
88
Agent score
75%

What's inside replicate-python

  1. Handle model output files with FileOutput

    main

    When a model outputs files, replicate.run() returns FileOutput objects. These are file-like objects that allow you to read data directly into memory or stream it in chunks.

    Key features:

    • .read(): Reads the entire file into memory (synchronous).
    • .aread(): Reads the entire file into memory (asynchronous).
    • Iterator Protocol: Supports for chunk in output: to stream data in chunks, which is ideal for large files.
    • .url: Provides the underlying data source URL (though using the object's methods is recommended).

    To opt out of this behavior and receive raw data instead, pass use_file_output=False to replicate.run().

    import replicate
from PIL import Image

output = replicate.run(
    "stability-ai/stable-diffusion:27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478",
    input={"prompt": "wavy colorful abstract patterns, oceans"}
    )

# Read binary data
with open("my_output.png", "wb") as file:
  file.write(output[0].read())
  
# Use as an iterator (e.g., with PIL)
background = Image.open(output[0])

  2. Stream FileOutput in web frameworks

    main

    Because FileOutput implements the Iterator protocol, it can be used directly to stream responses in popular Python web frameworks.

    FastAPI

    @app.get("/")
async def main():
    output = replicate.run("black-forest-labs/flux-schnell", input={...}, use_file_output=True)
    return StreamingResponse(output)

    Flask

    @app.route('/stream')
def streamed_response():
    output = replicate.run("black-forest-labs/flux-schnell", input={...}, use_file_output=True)
    return app.response_class(stream_with_context(output))

    Django

    def stream_response(request):
    output = replicate.run("black-forest-labs/flux-schnell", input={...}, use_file_output=True)
    return HttpResponse(output, content_type='image/webp')
    import replicate
import aiofiles

# Async reading entire file
async with aiofiles.open(filename, 'w') as file:
    await file.write(await output.aread())

# Async streaming chunks
async with aiofiles.open(filename, 'w') as file:
    async for chunk in output:
        await file.write(chunk)

  4. Compose models into a pipeline

    main

    You can chain models together by passing the output of one model's predict method as an input to the next.

    import replicate

laionide = replicate.models.get("afiaka87/laionide-v4").versions.get("b21cbe271e65c1718f2999b038c18b45e21e4fba961181fbfae9342fc53b9e05")
swinir = replicate.models.get("jingyunliang/swinir").versions.get("660d922d33153019e8c263a3bba265de882e7f4f70396546b6c9c8f9d47a021a")

image = laionide.predict(prompt="avocado armchair")
upscaled_image = swinir.predict(image=image)

  6. Deployment and Release data models

    main

    The following classes represent the data structures returned by the Replicate API for deployments:

    Deployment

    Represents a hosted deployment.

    • id: The qualified name owner/name.
    • owner: The user or organization owning the deployment.
    • name: The deployment name.
    • configuration: A Configuration object containing hardware and scaling settings.
    • current_release: The Release currently active in the deployment.
    • predictions: A namespace to access predictions for this specific deployment.

    Release

    Represents a specific versioned release of a deployment.

    • number: The release number.
    • model: The model identifier (owner/name).
    • version: The specific model version ID.
    • created_at: Timestamp of creation.
    • created_by: The Account that created the release.

    Configuration

    Represents the scaling and hardware settings.

    • hardware: The hardware SKU.
    • min_instances: Minimum number of instances for scaling.
    • max_instances: Maximum number of instances for scaling.

  7. Understand the Version and Versions objects

    main

    In the Replicate Python client, model versions are managed through two primary classes:

    1. Version: A data object representing a specific snapshot of a model. It contains metadata like id, created_at, cog_version, and the openapi_schema.
    2. Versions: A namespace object used to perform operations on the versions of a specific model. It is initialized with a Client and a reference to a model (which can be a Model object or a string in the format owner/name).

  8. Initialize a custom Client

    main

    While the replicate package provides a default shared client using the REPLICATE_API_TOKEN environment variable, you can instantiate a replicate.client.Client to use a different API token or add custom HTTP headers.

    import os
from replicate.client import Client

replicate = Client(
    api_token=os.environ["SOME_OTHER_REPLICATE_API_TOKEN"],
    headers={
        "User-Agent": "my-app/1.0"
    }
)

  9. Create a new model

    main

    Create a model for a user or organization by specifying the owner, name, visibility, and hardware SKU.

    import replicate

model = replicate.models.create(
    owner="your-username",
    name="my-model",
    visibility="public",
    hardware="gpu-a40-large"
)

  10. List predictions

    main

    Use replicate.predictions.list() to retrieve a list of your previous predictions. The results are paginated. To fetch subsequent pages, pass the next property from the previous page result back into the list() method.

    import replicate

page1 = replicate.predictions.list()

if page1.next:
    page2 = replicate.predictions.list(page1.next)

  11. List and paginate models

    main

    You can list models created by a user or organization using replicate.models.list(). Results are paginated.

    Automatic Pagination (Recommended) Use replicate.paginate to fetch all pages automatically.

    Manual Pagination Use the next property from the current page to fetch the next page of results.

    # Automatic pagination
models = []
for page in replicate.paginate(replicate.models.list):
    models.extend(page.results)
    if len(models) > 100:
        break

# Manual pagination
page = replicate.models.list()
while page:
    models.extend(page.results)
    if len(models) > 100:
          break
    page = replicate.models.list(page.next) if page.next else None