ImageSorcery MCP

repository·master·Indexed 43 days ago

https://github.com/sunriseapps/imagesorcery-mcp

A Model Context Protocol (MCP) server providing local computer vision and image manipulation tools for LLMs. Version 0.12.0 allows AI assistants to perform image editing (crop, resize, rotate, blur), drawing and annotation, object detection via Ultralytics, OCR via EasyOCR, and background removal locally on the user's machine to ensure privacy.

Tokens
13.6K
Snippets
48
Records
66
Agent score
54%

What's inside ImageSorcery MCP

  1. Overview of ImageSorcery MCP

    master

    ImageSorcery MCP is a Model Context Protocol (MCP) server that provides AI assistants with local, computer-vision-based image recognition and editing capabilities. It allows an AI to perform tasks like cropping, resizing, object detection, OCR, and background removal locally on your machine without uploading images to external servers.

    Key Capabilities:

    • Image Editing: Crop, resize, rotate, blur, and change color palettes.
    • Drawing & Annotation: Draw text, shapes (circles, lines, rectangles, arrows), and overlays.
    • Computer Vision: Object detection (via Ultralytics), finding objects by text description, and OCR (via EasyOCR).
    • Advanced Processing: Background removal and masking.

    Best Practices for AI Assistants:

    • Use full paths to files when requesting tasks.
    • Specify models and confidence thresholds for detection tasks.
    • Add "use imagesorcery" to your prompt to ensure the assistant uses the correct tools.
  2. How prompts work in ImageSorcery MCP

    master

    Prompts in ImageSorcery MCP are parameterized message templates designed to guide LLMs through specific image processing workflows. Instead of raw instructions, prompts provide structured guidance that helps the AI use the server's tools effectively for tasks like background removal or object manipulation.

    Prompts are registered using a register_prompt function that accepts a FastMCP instance and utilizes the @mcp.prompt() decorator. This architecture allows the server to expose these templates as callable resources to the LLM.

    # Conceptual implementation pattern
    def register_prompt(mcp: FastMCP):
        @mcp.prompt()
        def my_new_prompt(param: str) -> str:
            return f"Process this image with {param}"
  3. How the ImageSorcery MCP configuration system works

    master

    The configuration lifecycle follows these steps:

    1. Creation: Running imagesorcery-mcp --post-install creates a config.toml file in the root directory with default values.
    2. Loading: The system automatically loads config.toml on startup. If the file is missing, one is created with defaults. Configuration is validated using Pydantic.
    3. Integration: Tools check for configuration defaults when parameters are omitted. For example, calling detect(input_path="image.jpg") will automatically use the detection.confidence_threshold and detection.default_model from the config. Explicitly provided parameters will always override these defaults.
  4. What the post-installation script does

    master

    Running imagesorcery-mcp --post-install is a critical setup step that performs the following:

    • Configuration: Creates a config.toml file in the current directory for customizing tool parameters.
    • Model Management: Creates a models directory and generates an initial models/model_descriptions.json.
    • YOLO Models: Downloads default YOLO models (yoloe-11l-seg-pf.pt, yoloe-11s-seg-pf.pt, yoloe-11l-seg.pt, yoloe-11s-seg.pt) required for the detect tool.
    • CLIP Installation: Attempts to install the clip Python package from Ultralytics' GitHub repository (required for the find tool's text prompt functionality).
    • CLIP Model: Downloads the CLIP model file into the models directory.
  5. Understand the ImageSorcery MCP Core Architecture

    master

    The ImageSorcery MCP server is built on the FastMCP framework and is organized around a central server entry point and a middleware layer.

    • server.py: Acts as the primary entry point. It handles FastMCP initialization (name, instructions, logging), registers tools (like blur, crop, detect) and resources (like models), manages command-line arguments, and orchestrates post-installation tasks such as model downloads.
    • middleware.py: Contains logic to intercept and process requests/responses. It uses middleware to improve error clarity and provide global exception handling, ensuring that errors are transformed into standardized McpError objects for clients.
  6. Manage telemetry keys during build

    master

    Telemetry API keys are managed via environment variables or the src/imagesorcery_mcp/telemetry_keys.py module.

    Environment Variables:

    • IMAGESORCERY_AMPLITUDE_API_KEY
    • IMAGESORCERY_POSTHOG_API_KEY

    Build Workflow for CI:

    1. Set environment variables in CI.
    2. Run populate_telemetry_keys.py to write keys to the module.
    3. Build/package the project.
    4. Run clear_telemetry_keys.py to wipe keys from the working copy to prevent accidental commits.

    Note: To skip population, set SKIP_TELEMETRY_POPULATION=true.

  7. How error handling and validation work in ImageSorcery MCP

    master

    ImageSorcery MCP uses custom middleware to ensure that errors returned to MCP clients are actionable and standardized.

    • ImprovedValidationMiddleware: Intercepts ToolError exceptions from FastMCP tools. It parses generic validation failures (like missing or unexpected parameters) and converts them into user-friendly McpError messages with standardized error codes.
    • ErrorHandlingMiddleware: Provides a global catch-all for unhandled exceptions. It ensures errors are logged with tracebacks and transformed into standardized McpError objects, preventing raw server crashes from being sent to the client.
  8. Rules for using ImageSorcery MCP tools

    master

    When interacting with the ImageSorcery MCP Server tools, adhere to the following technical requirements:

    • Coordinate Format: All tools that require image coordinates use the Bounding Box format: [x1, y1, x2, y2], where (x1, y1) represents the top-left corner and (x2, y2) represents the bottom-right corner.
    • File Path Requirements: All file paths provided in tool arguments (such as input_path or output_path) must be absolute (full) paths. Relative paths (e.g., my_image.jpg) are not supported; use the full system path (e.g., /home/user/images/my_image.jpg).
  9. Run full post-installation with `post_install.py`

    master

    Automates the complete setup process for the ImageSorcery MCP server. This includes:

    • Creating the models directory.
    • Generating model descriptions.
    • Downloading default YOLO models.
    • Installing the clip Python package.
    • Downloading the CLIP model.
    • Ensuring a .user_id file exists for telemetry.

    CLI Usage:

    # As a standalone script
    python -m src.imagesorcery_mcp.scripts.post_install
    
    # Via the server entry point
    imagesorcery-mcp --post-install
    # Run post-installation as a standalone script
    python -m src.imagesorcery_mcp.scripts.post_install
    
    # Or run it through the server with the --post-install flag
    imagesorcery-mcp --post-install
  10. Adding new prompts to the server

    master

    To extend the server with a new prompt template, follow these steps:

    1. Create the file: Create a new Python file in src/imagesorcery_mcp/prompts/ (e.g., new_prompt.py).
    2. Implement logic: Define your prompt function with the necessary parameters and return types.
    3. Register the prompt: Implement a register_prompt(mcp: FastMCP) function within your new file that uses the @mcp.prompt() decorator.
    4. Server Integration: Import and call your register_prompt function in src/imagesorcery_mcp/server.py.
    5. Documentation: Add the new prompt to the Available Prompts section of the src/imagesorcery_mcp/prompts/README.md in alphabetical order.
    6. Testing: Add corresponding tests in tests/prompts/test_new_prompt.py.
  11. Set up ImageSorcery MCP for development

    master

    To contribute to or develop ImageSorcery MCP, follow these steps to set up a local environment:

    1. Clone the repository:
      git clone https://github.com/sunriseapps/imagesorcery-mcp.git
      cd imagesorcery-mcp
    2. Create and activate a virtual environment:
      python -m venv venv
      source venv/bin/activate # For Linux/macOS
      # venv\Scripts\activate    # For Windows
    3. Install in editable mode with dev dependencies:
      pip install -e ".[dev]"
      This installs the package and all development tools (like build and twine).
    git clone https://github.com/sunriseapps/imagesorcery-mcp.git
    cd imagesorcery-mcp
    python -m venv venv
    source venv/bin/activate
    pip install -e ".[dev]"
  12. Install ImageSorcery MCP via pipx (Recommended)

    master

    The recommended installation method is using pipx, which manages virtual environments automatically.

    1. Install pipx if you haven't already:
      • macOS: brew install pipx
      • Ubuntu/Debian: sudo apt update && sudo apt install pipx
      • Other: pip install --user pipx followed by pipx ensurepath
    2. Install the package: pipx install imagesorcery-mcp
    3. Run post-installation: imagesorcery-mcp --post-install. This step is mandatory as it downloads required models and attempts to install the clip package.
    pipx install imagesorcery-mcp
    imagesorcery-mcp --post-install