DOCSEARCHbyAlgolia
Submit Docs

Comic Translate

repository·main·Indexed 25 days ago

https://github.com/ogkalu2/comic-translate

A tool for translating comics, manga, and webtoons using Large Language Models like GPT-4. It automates a pipeline consisting of speech bubble detection, OCR, inpainting for text removal, translation, and re-rendering. The tool includes a GUI with manual correction modes, undo/redo functionality for bounding box and brush operations, and support for various image patching and text replacement commands.

Tokens
4.6K
Snippets
5
Records
34
Agent score
78%

What's inside comic-translate

  1. Comic Translate GUI Usage Tips

    main

    Navigation and Controls

    • Zoom/Scroll: Use Ctrl + Mouse Wheel to zoom in/out. Otherwise, use vertical scrolling.
    • Page Navigation: Use the Right and Left arrow keys to move between images.
    • Trackpad: Standard trackpad gestures are supported for viewing images.

    Translation Features

    • Manual Mode: If Automatic Mode fails (due to detection errors, poor OCR, or insufficient cleaning), you can switch to Manual Mode to correct the image/text manually.
    • Continuous Reading: In Automatic Mode, processed images are saved and loaded into the viewer (or loaded upon transition), allowing you to keep reading while other images are being translated.
    • Font Support: Ensure your selected font supports the characters of the target language.

  2. Use Manual Mode for corrections

    main

    If Automatic Mode fails (e.g., no text detected, incorrect OCR, or insufficient cleaning), use Manual Mode to make corrections:

    1. Select the image with issues.
    2. Click Undo the Image.
    3. Toggle Manual Mode to manually edit or correct elements.

  3. Install Comic Translate from Source

    main

    To run the source code directly, follow these steps:

    1. Prerequisites:

      • Install Python 3.12 (ensure you check "Add python.exe to PATH" during installation).
      • Install git.
      • Install uv (installation guide).

    2. Setup Commands:

    git clone https://github.com/ogkalu2/comic-translate
cd comic-translate
uv init --python 3.12
uv add -r requirements.txt --compile-bytecode
    1. GPU Acceleration (NVIDIA users only): If you have an NVIDIA GPU, install onnxruntime-gpu for better performance:
    uv pip install onnxruntime-gpu
    1. Updating: To update your local installation:
    git pull
uv init --python 3.12
uv add -r requirements.txt --compile-bytecode

    (Note: Only run uv init if you did not use uv during the initial installation.)

  4. Install Comic Translate via Desktop Application or Browser Extension

    main

    You can use Comic Translate without manual setup through these methods:

    • Desktop Application: Download and install for Windows or macOS from https://www.comic-translate.com.

      • Windows: If prompted by Smart Screen, click More info > Run anyway.
      • macOS: If the app fails to open, go to Settings > Privacy & Security and click Open Anyway at the bottom.
      • Note: GPU acceleration is currently only available when running from source.

    • Browser Extension: Install the Comic Translate extension from https://www.comic-translate.com/download for Chromium-based browsers (Chrome, Edge, Brave, etc.) to read directly on websites.

  5. Usage Tips and Controls

    main

    Interface Controls

    • Zooming: Use Ctrl + Mouse Wheel to zoom. Otherwise, the wheel performs vertical scrolling.
    • Navigation: Use Left and Right mouse buttons to navigate between images.
    • Touchpad: Standard touchpad gestures are supported for viewing images.

    Translation Features

    • Manual Mode: If automatic mode fails (e.g., text not detected, incorrect OCR, or poor cleaning), you can switch to Manual Mode to make corrections. To do this, undo the image and switch modes.
    • Continuous Reading: In automatic mode, once an image is processed, it is loaded into the viewer or stored so you can continue reading while other images are being translated.
    • Font Support: Ensure the selected font supports the characters of your target language.

  6. Comic Translate Application Tips and Shortcuts

    main

    Navigation and View

    • Zoom: Ctrl + Mouse Wheel.
    • Vertical Scroll: Use the Mouse Wheel without Ctrl.
    • Touchpad: Standard touchpad gestures are supported for viewing.
    • Page Navigation: Use the Left and Right arrow keys to move between images.

    Translation Modes

    • Automatic Mode: Processes images in the background so you can continue reading while translation occurs.
    • Manual Mode: Introduced in v2.0. If Automatic Mode fails (e.g., no text detected, incorrect OCR, or poor cleaning), switch to Manual Mode to manually correct the translation and segmentation.

    Configuration

    • Fonts: Ensure the selected font supports the characters of your target language.

  7. Install Comic Translate via Desktop App or Browser Extension

    main

    You can install Comic Translate using pre-built binaries or extensions:

    • Desktop App: Download the Windows or macOS installer from https://www.comic-translate.com.

      • Windows: If prompted by Smart Screen, click 'More info' > 'Run anyway'.
      • macOS: If the app fails to open, go to 'System Settings' > 'Privacy & Security', scroll down, and click 'Open Anyway'.
      • Note: GPU acceleration is currently only available when running from source.

    • Browser Extension: For Chromium-based browsers (Chrome, Edge, Brave, etc.), install the extension from https://www.comic-translate.com/download to read directly on websites.

  9. Troubleshoot CBR file errors

    main

    If you are attempting to use CBR files and encounter the error raise RarCannotExec("Cannot find working tool"), it means your system cannot find a working RAR extraction tool.

    Solution:

    1. Install Winrar or 7-Zip.
    2. Add the installation directory (e.g., C:\Program Files\WinRAR on Windows) to your system's PATH environment variable.

  10. Comic Translate GUI Navigation and Controls

    main

    Use these shortcuts and gestures within the application:

    • Zoom: Ctrl + Mouse Wheel.
    • Navigation: Right and Left arrow keys to move between images.
    • Scrolling: Vertical scrolling is the default.
    • Gestures: Standard trackpad gestures are supported for viewing images.

  11. Reference the Comic Translate Technical Pipeline

    main

    Comic Translate uses the following pipeline for comic translation:

    • Bubble & Text Detection: Uses an RT-DETR-v2 model (trained on 11,000 comic images) via bubble-and-text-detector.
    • OCR (Optical Character Recognition):
      • Japanese: manga-ocr (default)
      • Korean: Pororo (default)
      • Other Languages: PPOCRv5 (default)
      • Optional Providers: Gemini 2.0 Flash, Microsoft Azure Vision.
    • Image Inpainting (Restoration): Removes detected text using lama (with Anime/Manga fine-tuning) or AOT-GAN models.
    • Translation: Supports LLMs including GPT-4.1, Claude-4.5, and Gemini-2.5. It provides full page text to the LLM and offers an option to provide the image itself for additional context.
    • Text Rendering: Renders translated text into the detected bounding boxes with appropriate line breaks.

  12. Reference: OCR and Translation Engines

    main

    Comic Translate uses various engines for OCR and translation:

    Default OCR Engines

    • Japanese: manga-ocr
    • Korean: Pororo
    • All other languages: PPOCRv5

    Optional OCR Engines (Supports any language)

    • Gemini 2.0 Flash
    • Microsoft Azure Vision

    Supported LLM Translators

    • GPT-4.1
    • Claude-4.5
    • Gemini-2.5

    Note: All LLMs receive the full page text to assist translation. You can also provide the image itself to provide more context to the model.