DOCSEARCHbyAlgolia
Submit Docs

Transformers

website·Indexed 40 days ago

https://gradio.app/

A model-definition framework for state-of-the-art machine learning models in text, computer vision, audio, and video.

Tokens
25.4K
Snippets
165
Records
183
Agent score
71%

What's inside Transformers

  1. Overview of the Transformers library

    Transformers is a model-definition framework for state-of-the-art machine learning models across text, computer vision, audio, video, and multimodal domains. It serves as a central pivot for model definitions, ensuring compatibility across various training frameworks (e.g., Axolotl, Unsloth, DeepSpeed, FSDP, PyTorch-Lightning), inference engines (e.g., vLLM, SGLang, TGI), and modeling libraries (e.g., llama.cpp, mlx).

  2. Gather event-specific data using gr.SelectData

    To access detailed information about a selection event (e.g., which cell in a DataFrame or which pixel in an Image was clicked), add gr.SelectData as a type hint to an argument in your event listener function. The SelectData object provides the value of the selection and the index where it occurred.
    import gradio as gr

with gr.Blocks() as demo:
    board = gr.Dataframe(value=[["", "", ""] * 3], interactive=False)

    def place(board, turn, evt: gr.SelectData):
        if evt.value:
            return board, turn
        # Use evt.index to find the specific cell
        board[evt.index[0]][evt.index[1]] = turn
        return board, "O" if turn == "X" else "X"

    board.select(place, [board, gr.State("X")], [board, gr.State()], show_progress="hidden")

demo.launch()

  3. Create chatbot UIs with gr.ChatInterface

    The gr.ChatInterface class is a high-level abstraction specifically designed for building Chatbot UIs. Like gr.Interface, it allows you to supply a function and automatically generates a fully working chat interface.

  4. Access event metadata using gr.EventData and subclasses

    When gr.EventData or one of its subclasses is used as a type hint for a prediction function argument, Gradio automatically passes an object containing information about the event that triggered the listener. All these objects contain a .target attribute referring to the component that triggered the event.

    Common subclasses include:

    • gr.SelectData: For .select() events.
    • gr.KeyUpData: For .key_up() events.
    • gr.DeletedFileData: For .delete() events.
    • gr.LikeData: For .like() events.
    • gr.RetryData: For .retry() events.
    • gr.UndoData: For .undo() events.
    • gr.EditData: For .edit() events.
    • gr.DownloadData: For .download() events.
    • gr.CopyData: For .copy() events.

  5. Build custom layouts with gr.Blocks

    For applications requiring more customizable layouts, complex data flows (where outputs serve as inputs to other functions), or dynamic visibility/property updates based on user interaction, use the gr.Blocks class instead of gr.Interface.

  6. Understand the Transformers library design architecture

    The Transformers library is built on two main design principles:

    1. Simplicity: Every model is implemented using only three main classes: configuration, model, and preprocessor. This allows for rapid deployment via the Pipeline or Trainer classes.
    2. Pretrained Model Focus: The library emphasizes the use of pretrained models to reduce compute costs, time, and carbon footprints while maintaining state-of-the-art performance.

  7. Stream audio and video media

    Gradio supports streaming audio and video directly from a generator. To enable this:

    1. Set streaming=True in the gr.Audio or gr.Video output component.
    2. Set autoplay=True to ensure media plays automatically as chunks arrive.
    3. Use a generator that yields chunks of the media.

    Supported Formats:

    • Audio: .mp3, .wav, or bytes sequences.
    • Video: .mp4 or .ts files (using h.264 codec).

    For smooth playback, ensure chunks are consistent in length and larger than 1 second.

    import gradio as gr
from time import sleep

def stream_media(file):
    for _ in range(10):
        sleep(0.5)
        yield file

# Audio Streaming
gr.Interface(
    stream_media, 
    gr.Audio(sources=["microphone"], type="filepath"), 
    gr.Audio(streaming=True, autoplay=True)
).launch()

# Video Streaming
gr.Interface(
    stream_media, 
    gr.Video(sources=["webcam"], format="mp4"), 
    gr.Video(streaming=True, autoplay=True)
).launch()

  8. Organize components vertically using gr.Column

    Components within a gr.Column are placed vertically. Columns are typically nested within gr.Row to create complex grid layouts. The scale parameter controls the relative width of columns within a row, and min_width ensures a minimum pixel width.
    import gradio as gr

with gr.Blocks() as demo:
    with gr.Row():
        with gr.Column(scale=1, min_width=300):
            text1 = gr.Textbox(label="prompt 1")
            text2 = gr.Textbox(label="prompt 2")
        with gr.Column(scale=2, min_width=300):
            img1 = gr.Image("images/cheetah.jpg")
            btn = gr.Button("Go")

demo.launch()

  9. Use JavaScript in Gradio event listeners

    Gradio event listeners support a js argument that accepts a JavaScript function as a string. This allows for client-side processing. You can provide both a JavaScript function and a Python function (fn), in which case the JavaScript function executes first. If only JavaScript is needed, set the Python fn to None.
    import gradio as gr

with gr.Blocks() as demo:
    subject = gr.Textbox(placeholder="subject")
    verb = gr.Radio(["ate", "loved", "hated"])
    object = gr.Textbox(placeholder="object")
    output = gr.Textbox()

    # JavaScript function runs first, then Python (if provided)
    # Here, only JS is used, so fn is None
    btn = gr.Button("Reverse")
    btn.click(
        fn=None, 
        inputs=[subject, verb, object], 
        outputs=output, 
        js="(s, v, o) => o + ' ' + v + ' ' + s"
    )

demo.launch()

  10. Share a Gradio demo publicly

    To create a publicly accessible URL for a Gradio demo without manual web hosting, set the share parameter to True within the launch() method.
    import gradio as gr

def greet(name):
    return "Hello " + name + "!"

demo = gr.Interface(fn=greet, inputs="textbox", outputs="textbox")

demo.launch(share=True)

  11. Install Gradio via pip

    Gradio requires Python 3.10 or higher. It is recommended to install Gradio within a virtual environment using pip.
    pip install --upgrade gradio

  12. Embed a Gradio app using the <gradio-app> web component

    The <gradio-app> web component allows you to embed Gradio demos into a webpage. It supports several attributes to control loading, appearance, and behavior.
    <gradio-app
	space="gradio/Echocardiogram-Segmentation"
	eager="true"
	initial_height="0px"
></gradio-app>