repository·main·Indexed 23 days ago
https://github.com/generative-computing/melleaA Python library for writing generative programs to build predictable, testable, and structured AI workflows. mellea replaces brittle prompting with type-annotated programs using Pydantic for schema enforcement and automatic retries. It features task decomposition via Mellea Decomp, support for aLoRA (Adaptive Low-Rank Adapters) training and deployment, asynchronous execution with lazy compute, and integrations for AWS Bedrock, Ollama, and audio-text-to-text workflows.
The m_decompose pipeline is organized into the following components:
decompose.py: Responsible for generating refined subtasks from the user request.pipeline.py: Orchestrates the full workflow (decomposition $\rightarrow$ execution $\rightarrow$ aggregation).prompt_modules/: Contains reusable prompt components used by the pipeline.m_decomp_result_v1.py.jinja2: A Jinja2 template used to format the final output.The tutorial demonstrates how to build a predictable generative pipeline using the following core components:
instruct(): The fundamental method for calling an LLM with structured instructions.simple_validate: A method for adding deterministic checks, such as word count or specific formatting rules.RejectionSamplingStrategy: A strategy used to control the retry budget and manage SamplingResult when constraints are not met.@generative decorator: A way to create typed functions that use LLM-backed implementations.Mellea provides several features for building predictable generative programs:
@generative and Pydantic schemas to enforce output types at generation time.mify to integrate Mellea into existing codebases.The following notebooks are available for exploring specific Mellea capabilities:
example.ipynb: General introduction and basic examples.compositionality_with_generative_stubs.ipynb: Tutorial on composing generative functions.context_example.ipynb: Context management and working with contexts.document_mobject.ipynb: Text processing using document MObjects.instruct_validate_repair.ipynb: Walkthrough of the instruct-validate-repair paradigm.m_serve_example.ipynb: Deploying Mellea programs as services.mcp_example.ipynb: Integration with the Model Context Protocol (MCP).model_options_example.ipynb: Configuring model parameters and options.sentiment_classifier.ipynb: Building a sentiment classification system.table_mobject.ipynb: Working with table data structures.simple_email.ipynb: Email generation based on requirements.georgia_tech.ipynb: Domain-specific research/academic use case.The Component Protocol is the fundamental unit of composition in Mellea. Every high-level API call (like m.instruct(), @generative, or m.chat()) is backed by a Component that handles input formatting for the LLM and output parsing into a typed result.
You should implement a custom Component instead of using the standard API when:
@generative docstrings or instruct() templates.ModelOutputThunk from one LLM call directly into the formatted input of another.MELP is an experimental lazy evaluation system for Mellea programs. It allows developers to define computations that are deferred until their results are explicitly required. This enables declarative workflow composition, optimization by avoiding unnecessary computations, and the implementation of advanced control flow patterns.
Core Concepts:
⚠️ Experimental: MELP is an experimental feature; APIs are subject to change and should be used with caution in production environments.
A generative program is any program that contains calls to an LLM, ranging from simple prompt wrappers to complex multi-step reasoning systems.
The core challenge of generative programming is the interleaving of deterministic code (which is predictable and testable) with stochastic LLM operations (which are non-deterministic and may produce varying outputs for the same input).
Mellea acts as a reliable execution layer. It is designed to manage the boundary between these two modes by ensuring that stochastic parts are constrained, failures are handled gracefully, and uncertainty does not accumulate unchecked. It is not an orchestration framework (like LangChain or smolagents) but rather a tool that those frameworks can use to ensure individual LLM calls or groups of calls meet specific requirements.
A CBlock (content block) is Mellea's atomic unit of content and the fundamental unit understood by backends. While chat() uses Message components, mfuncs.act() allows you to pass any component or CBlock directly.
Note on Tokenization: CBlocks are tokenization boundaries. The tokenization of concatenate(CBlock(str_a), CBlock(str_b)) is concatenate(tokenize(str_a), tokenize(str_b)), which may differ from tokenize(concatenate(str_a, str_b)). This affects KV caching.
import mellea.stdlib.functional as mfuncs
from mellea.stdlib.base import SimpleContext, CBlock
from mellea.backends.ollama import OllamaModelBackend
response, next_context = mfuncs.act(
CBlock("What is 1+1?"),
context=SimpleContext(),
backend=OllamaModelBackend("granite4:latest"),
)
print(response.value)Mellea supports real-time streaming of LLM responses, allowing you to receive output token-by-token. To make this output useful for validation, Mellea uses Chunking to group tokens into meaningful units (like words, sentences, or paragraphs).
This allows for Stream Validation, where you can apply requirements at the chunk level to trigger an early exit (stopping generation) if a constraint is violated.
You can monitor the generation progress by processing the following events:
ChunkEvent: A new chunk of text has been received.QuickCheckEvent: An initial validation result for a chunk.FullValidationEvent: Complete validation after the full generation is finished.StreamingDoneEvent: The generation process is complete.Mellea provides two primary ways to interact with models:
instruct(): Best for one-off instructions where the prompt text varies. It uses a prompt string with {{variable}} placeholders that are filled at call time using user_variables.@generative: Best for reusable, typed, and unit-testable functions. The prompt is defined once in the function's docstring. These functions also participate in Mellea's lazy evaluation graph, allowing you to compose generative calls.| Feature | instruct() | @generative |
|---|---|---|
| Prompt Definition | At call time (string) | In function docstring |
| Variable Handling | user_variables dict | Function arguments |
| Typing | Returns raw output | Returns typed Python objects |
# instruct() example
with start_session() as m:
result = m.instruct(
"Translate the following into {{language}}: {{text}}",
user_variables={"language": "French", "text": "Hello, world!"},
)
# @generative example
@generative
def translate(text: str, language: str) -> str:
"""Translate text into the specified language.
Return only the translated text, with no explanation.
"""
with start_session() as m:
result = translate(m, text="Hello, world!", language="French")Mellea's start_session() uses SimpleContext by default, which is designed to be safe for concurrent async calls.
When to use which context:
SimpleContext. It prevents state corruption when multiple thunks are being generated at once.ChatContext from mellea.stdlib.context.Important Note on ChatContext:
ChatContext is NOT safe for concurrent writes. If you use it, you must ensure you await each call fully before starting the next one to avoid stale contexts or corruption. You may see a warning if ChatContext is used with async methods, even if called sequentially; this is safe to ignore if you are strictly awaiting each call before the next.
Sequential Chat Example:
import asyncio
import mellea
from mellea.stdlib.context import ChatContext
async def sequential_chat():
m = mellea.start_session(ctx=ChatContext())
r1 = await m.achat("Hello.")
r2 = await m.achat("Tell me more.") # safe because r1 is fully resolved
print(str(r2))
asyncio.run(sequential_chat())import asyncio
import mellea
from mellea.stdlib.context import ChatContext
async def sequential_chat():
m = mellea.start_session(ctx=ChatContext())
r1 = await m.achat("Hello.")
r2 = await m.achat("Tell me more.") # safe — r1 is fully resolved
print(str(r2))
asyncio.run(sequential_chat())When using stream_with_chunking, you can specify a chunking strategy to determine how the LLM output is split for validation. The choice involves a trade-off between latency (how fast a validator reacts) and context (how much information the validator has per chunk).
| Alias | Splits on | Good for |
|---|---|---|
"word" | Whitespace | Token-local checks: forbidden words, numeric limits |
"sentence" | ., !, ? followed by whitespace | Grammar, coherence, per-sentence content rules |
"paragraph" | Two or more consecutive newlines | Topic coherence, citation presence, heading structure |