promptic

promptic is designed to be a productive and pythonic library for LLM application development. It leverages LiteLLM to prevent provider lock-in, allowing you to switch LLM providers by changing a single line of code.

Key features include:

• Type-safe structured outputs: Uses Pydantic for reliable data extraction.
• Agent building: Easy implementation of agents using function calling.
• Streaming: Built-in support for real-time response streaming.
• Prompt caching: Automatic caching for supported models to reduce latency and cost.
• Conversation memory: Built-in mechanisms to manage chat history.

By default, promptic functions are stateless. You can enable conversation memory in two ways:

1. Built-in Memory: Set memory=True in the @llm decorator. This allows the LLM to maintain context across multiple calls. Use the .message() method on the decorated function to send follow-up messages without re-executing the original prompt template.
2. Custom State: Extend the State class to implement custom persistence (e.g., saving history to Redis or a database) and pass it to the state parameter in @llm.

from promptic import llm

@llm(memory=True)
def chat(message):
    """{message}"""

# Initial call
print(chat("Hi, I'm Alice"))

# Follow-up using .message() to maintain context
print(chat.message("What is my name?"))

You can build agents by decorating functions with @llm.tool. These functions become tools that an LLM can invoke. When an LLM function (the agent) is called, it can automatically execute these tools to perform actions or retrieve information based on the user's request.

from promptic import llm

@llm(model="gpt-4o")
def scheduler(command):
    """{command}"""

@scheduler.tool
def get_current_time():
    """Get the current time"""
    return "10:00 AM"

print(scheduler("What time is it?"))

The @llm decorator turns a Python function into an LLM prompt. The function's docstring serves as the prompt template, where curly braces {} denote variables that will be replaced by the function's arguments. When the function is called, promptic generates the prompt and returns the LLM's response.

You can also pass configuration options to the decorator, such as model and system prompts.

from promptic import llm

@llm
def translate(text, language="Chinese"):
    """Translate '{text}' to {language}"""

print(translate("Hello world!"))
# 您好，世界！

@llm(
    model="claude-3-haiku-20240307",
    system="You are a customer service analyst. Provide clear sentiment analysis with key points.",
)
def analyze_sentiment(text):
    """Analyze the sentiment of this customer feedback: {text}"""

print(analyze_sentiment("The product was okay but shipping took forever"))

VCR.py integration allows you to record HTTP interactions (like API calls to OpenAI or Anthropic) into YAML files called "cassettes". Subsequent test runs replay these recordings instead of making real API calls, providing speed, cost savings, and determinism.

To use it, mark your test functions with the @pytest.mark.vcr decorator. On the first run, VCR records the live API responses; on subsequent runs, it uses the recorded data.

import pytest
from promptic import llm

@pytest.mark.vcr
def test_llm_call():
    @llm(model="gpt-4o-mini", temperature=0)
    def get_greeting(name: str) -> str:
        """Say hello to {name}"""

    result = get_greeting("Alice")
    assert "Alice" in result

To trace LLM interactions and function calls, you can use the Promptic class with a weave client. This allows you to visualize and debug your LLM application flows.

import weave
from promptic import Promptic

client = weave.init("my-project")
promptic = Promptic(weave_client=client)

@promptic.llm
def translate(text, language="Chinese"):
    """Translate '{text}' to {language}"""

print(translate("Hello"))

You can force the LLM to return data in a specific format using two methods:

1. Pydantic Models: Use a Pydantic model as the function's return type annotation. The LLM response is automatically validated and returned as a Pydantic object.
2. JSON Schema: Pass a dictionary representing a JSON Schema to the json_schema parameter in the @llm decorator.

from pydantic import BaseModel
from promptic import llm

class Forecast(BaseModel):
    location: str
    temperature: float
    units: str

@llm
def get_weather(location, units: str = "fahrenheit") -> Forecast:
    """What's the weather for {location} in {units}?"""

print(get_weather("San Francisco"))

To ensure reliable and maintainable tests when using VCR:

• Use deterministic settings: Always set temperature=0 in your @llm calls to ensure the LLM responses are consistent across recordings.
• Organize by recording needs: Use @pytest.mark.vcr for tests making API calls and @pytest.mark.no_vcr for pure logic tests to avoid unnecessary overhead.
• Update cassettes when prompts change: If you modify a prompt or expected output, use --record-mode=rewrite to refresh the cassette.
• Handle sensitive data: API keys are automatically filtered and replaced with REDACTED in cassettes (configured in tests/conftest.py).
• Commit cassettes to version control: This allows CI/CD to run tests without API keys and ensures other developers can run tests immediately.

Promptic integrates with VCR.py to record and replay HTTP interactions. This allows you to run tests without making real API calls after the first run, making tests faster and cheaper.

To use it, apply the @pytest.mark.vcr decorator to your test function.

import pytest
from promptic import llm

@pytest.mark.vcr
def test_my_llm_function():
    @llm(model="gpt-4o-mini", temperature=0)
    def get_capital(country: str) -> str:
        """What is the capital of {country}?"""

    assert get_capital("France") == "Paris"

Install the promptic package using pip to begin building LLM applications.

pip install promptic

"Cassette not found"

Cause: The test is trying to replay a cassette that doesn't exist. Solution: Run the test with --record-mode=once to create the cassette.

"Request not found in cassette"

Cause: The test is making a request that wasn't previously recorded. Solution:

1. Re-record the cassette using --record-mode=rewrite.
2. Use --record-mode=new_episodes to append the new request to the existing cassette.