Dify Plugin SDK

Starting from SDK version 0.9.0, plugins can support polling-based LLM invocation. This allows a model to declare a polling feature and implement methods that handle long-running provider jobs. Instead of a single long request, the plugin submits a job and returns status updates through a short request/response flow.

Polling results transition through three specific states:

• running: Returns plugin-owned state to be used for the next polling check.
• succeeded: Returns the final LLM result.
• failed: Returns a terminal error state.

The manifest specification uses two key fields to manage compatibility between plugins and the Dify platform:

1. meta.version: Specifies the version of the manifest specification. This ensures backward compatibility. If Dify detects an unsupported version, it will attempt to use only the supported parts of the plugin.
2. meta.minimum_dify_version: Specifies the minimum version of Dify required to run the plugin. This ensures forward compatibility and helps users understand if they need to upgrade Dify to use specific plugin features.

The Dify Plugin SDK uses just as a command runner. To develop with the SDK, install just using your preferred package manager or via uv:

uv tool install rust-just

Common development tasks can be executed using the following commands:

• just dev: Sync default development dependencies.
• just fmt: Apply formatting fixes.
• just lint: Format and apply linting fixes.
• just check: Check lockfile, formatting, and linting.
• just test: Run SDK tests.
• just build: Build source and wheel distributions.
• just docs: Generate schema documentation.

The Dify Plugin SDK follows Semantic Versioning (a.b.c). When adding the SDK to your project, it is recommended to use version constraints that allow patch and minor updates (for bug fixes and new features) but prevent major version updates to avoid breaking changes.

Example dependency constraint:

dify_plugin >= 0.8

When implementing an AIModel, you must provide an _invoke_error_mapping property. This allows the SDK to transform raw exceptions from your provider (e.g., a specific HTTP error from an API) into standardized Dify errors like InvokeAuthorizationError or a general InvokeError.

The internal helper _transform_invoke_error uses this mapping to wrap the original error with a provider-specific prefix (e.g., [provider_name]) and a descriptive message.

When an LLM request is finished (non-streaming), use LLMResult. It contains the full message, the total usage, and the model name.

• LLMResult: Standard full response.
• LLMResultWithStructuredOutput: A variant that includes a structured_output mapping for JSON-mode or tool-calling responses.

Note: The prompt_messages field in these classes is deprecated and will always be returned as an empty list for backward compatibility.

# Example of a complete result
result = LLMResult(
    model="gpt-4",
    message=AssistantPromptMessage(content="Full response text"),
    usage=LLMUsage.empty_usage()
)

The agent's reasoning process is captured in AgentScratchpadUnit objects. These represent the 'thought' process and the resulting actions.

An Action object within a scratchpad unit contains:

• action_name: The name of the tool to call.
• action_input: A dictionary or string containing the arguments for the tool.

You can check if an agent has reached a conclusion by calling is_final() on a scratchpad unit. An agent is considered finished if the action_name is exactly "final answer" (case-insensitive).

# Example of checking if an agent is finished
if scratchpad_unit.is_final():
    print("Agent reached final answer:", scratchpad_unit.agent_response)

For streaming LLM responses, use LLMResultChunk and its variants. A chunk contains a delta (the incremental change in the message), the model name, and an optional system_fingerprint.

• LLMResultChunkDelta: Contains the incremental message, usage (if provided in the chunk), and finish_reason.
• LLMResultChunkWithStructuredOutput: A variant used when the LLM returns structured data (via the structured_output field).

# Example of a chunk structure
chunk = LLMResultChunk(
    model="gpt-4",
    delta=LLMResultChunkDelta(
        index=0,
        message=AssistantPromptMessage(content="Hello")
    )
)

The ModelPosition class allows you to control the display order of models in the Dify UI. You provide lists of model names (strings) for different categories. Models in the list are sorted in ascending order based on their position in the array.

Supported categories:

• llm
• text_embedding
• rerank
• tts
• speech2text
• moderation

from dify_plugin.entities.model.provider import ModelPosition

position = ModelPosition(
    llm=["gpt-4", "gpt-3.5-turbo"],
    text_embedding=["text-embedding-3-small"]
)

The meta.version field tracks changes to the manifest specification.