repository·master·Indexed 27 days ago
https://github.com/hect0x7/jmcomic-crawler-pythonA Python API and CLI tool for crawling and downloading content from JMComic (禁漫天堂), supporting both web and mobile interfaces. It features robust anti-bot bypassing, plugin support, and integration with GitHub Actions. The library provides synchronous and asynchronous methods for downloading albums and photos, customizable directory rules, and support for image decoding and format conversion.
jmcomic library provides a Python API designed to crawl and download content from JMComic. Developers can use it to access JMComic interfaces and automate the downloading of albums using a few lines of code.The plugin mechanism (introduced in v2.2.0) allows for flexible, non-intrusive functional enhancements. Plugins are automatically executed when specific events occur. You can configure plugins in your option configuration file.
Built-in events include:
after_init: Triggered after the option object is created.before_image: Before downloading an image.before_album: Before downloading an album.before_photo: Before downloading a chapter.after_image: After downloading an image.after_album: After downloading an album.after_photo: After downloading a chapter.jmcomic is a Python-based crawler designed to download content from JMComic. It supports both Async and Sync APIs and includes features to bypass Cloudflare protection and support the latest encryption/decryption algorithms used by the JMComic app.
Key Features:
The jmcomic crawler provides several advanced features for downloading content:
Downloader, Option, Client, and entity classes.Option objects, including request domains, disk cache settings, download paths, and metadata (headers, cookies, proxies).For advanced configurations (e.g., changing image formats, setting proxies, or using plugins), create a configuration file (e.g., option.yml) and load it using jmcomic.create_option_by_file().
Example option.yml to convert images to PNG:
download:
image:
suffix: .pngPython usage:
import jmcomic
# Create an option object from your config file
option = jmcomic.create_option_by_file('D:/option.yml')
# Download the album using the option object
jmcomic.download_album(123, option)
# Alternatively: option.download_album(123)import jmcomic
option = jmcomic.create_option_by_file('D:/option.yml')
jmcomic.download_album(123, option)Once the GitHub Action workflow has finished running:
.zip file.You can define custom download paths using the dir_rule.rule configuration. This supports f-string template syntax where you use {variable_name} to inject entity attributes into the path.
Syntax Rules:
/ as a separator (recommended): Bd / {Atitle} / {Pname}._ as a separator: Bd_{Atitle}_{Pname}./ and _. If your folder name needs an underscore, use / as the primary separator (e.g., Bd / {Aid}_{Atitle}).Bd is a special segment representing the base_dir and should typically be placed at the beginning.Variables are prefixed by A (for Album/本子) or P (for Photo/章节).
dir_rule:
base_dir: D:/a/b/c/
rule: Bd / {Atitle}For advanced logging requirements, there are two primary interception methods:
jm_logger (Recommended/Standard): Use this to change log output destinations (e.g., files, monitoring services, backends), adjust display formats, or implement custom filtering.EXECUTOR_LOG (Advanced/Deep Customization): Use this if you need to completely reshape the log distribution logic or bridge logs directly to third-party systems that do not follow the standard Python logging protocol.You can automate the export and download of your JMComic favorites data using a GitHub Actions workflow. This process involves forking the repository, configuring your credentials as GitHub Secrets, and running the specific workflow.
https://github.com/hect0x7/JMComic-Crawler-Python/fork and click Create fork.To prevent leaking your credentials in the Action logs, use GitHub Secrets instead of manual input.
https://github.com/[YOUR_USERNAME]/JMComic-Crawler-Python/settings/secrets/actions.JM_USERNAME: Your JMComic username.JM_PASSWORD: Your JMComic password.ZIP_PASSWORD: The password for the resulting compressed file.https://github.com/[YOUR_USERNAME]/JMComic-Crawler-Python/actions/workflows/export_favorites.yml.To filter categories or rankings, use categories_filter for a single page or categories_filter_gen as an asynchronous generator for automatic pagination. Use JmMagicConstants to specify time, category, and sort order.
import asyncio
from jmcomic import JmOption, JmMagicConstants
async def main():
async with JmOption.default().new_jm_async_client() as cl:
# Get the first page of albums sorted by views
page = await cl.categories_filter(
page=1,
time=JmMagicConstants.TIME_ALL,
category=JmMagicConstants.CATEGORY_ALL,
order_by=JmMagicConstants.ORDER_BY_VIEW,
)
for aid, atitle in page:
print(aid, atitle)
# Use generator for all pages
async for page in cl.categories_filter_gen(
time=JmMagicConstants.TIME_ALL,
category=JmMagicConstants.CATEGORY_ALL,
order_by=JmMagicConstants.ORDER_BY_VIEW,
):
print(page.page)
asyncio.run(main())You can download an entire album using the synchronous download_album method or the asynchronous download_album_async method. Pass the album ID as a string.
import jmcomic
# Synchronous download
jmcomic.download_album('123')
# Asynchronous download
import asyncio
asyncio.run(jmcomic.download_album_async('123'))In environments without a global proxy (like some Linux servers), you can pass a custom postman object with proxy settings to the domain fetching APIs.
JmModuleConfig.get_html_domain_all(postman=JmModuleConfig.new_postman(proxies={'http': 'http://127.0.0.1:7890', 'https': 'http://127.0.0.1:7890'}))