pixiv-shaft Documentation

repository·classic·Indexed 27 days ago

https://github.com/ceuilisa/pixiv-shaft

An open-source third-party Pixiv client for Android featuring advanced search, batch downloads, and a full novel reader. The project implements a dual-channel connection strategy using Cronet QUIC for API requests and No-SNI TLS for image loading to bypass network restrictions. It includes technical details on the ImageHostManager for proxy support, WebSocket lifecycle management with HMAC-SHA256 authentication, and a technical stack based on Kotlin, Java, and Material Design 3.

Tokens
9.5K
Snippets
9
Records
45
Agent score
93%

What's inside pixiv-shaft

  1. Overview of Shaft features

    classic

    Shaft is an open-source, third-party Pixiv client for Android. Key capabilities include:

    • Browsing & Discovery: Personalized recommendations (illustrations, manga, novels), trending tags, and daily/weekly/monthly rankings.
    • Search & Filter: Search for users and content with advanced filters and popularity sorting (no premium required).
    • Social & Interaction: Commenting, following users, multi-account support, and user muting/blocking.
    • Downloads & History: Batch downloading with queue management, customizable file naming, and local history.
    • Content Support: GIF playback, full novel reader with chapter support, and reverse image search (SauceNAO, TinEye, IQDB, Ascii2D).
    • Experience: Material Design 3, dark mode, and direct connection support for mainland China users.
  2. Switch Pixiv Image Hosts

    classic

    Pixiv-Shaft allows users to switch between different image host domains to improve loading speeds or bypass network restrictions.

    Supported modes:

    • PIXIV: The default official Pixiv hosts (i.pximg.net / s.pximg.net).
    • PIXIV_CAT: Uses the Pixiv Cat proxy (i.pixiv.cat / s.pixiv.cat).
    • CUSTOM: Allows users to input a complete custom proxy prefix (e.g., https://your.proxy[/optional/path]).

    Important Note on URL Persistence: Image rewriting happens only during the loading phase. The underlying data models always store the original pximg.net URLs. This ensures that shared links or copied URLs remain universally accessible and functional for other users.

  3. Understand the Pixiv-Shaft Dual-Channel Connection Strategy

    classic

    Pixiv-Shaft uses a dual-channel architecture to bypass network restrictions (GFW) that target Pixiv's SNI (Server Name Indication). Instead of a single connection method, it splits traffic into two distinct paths based on the target service:

    1. API Requests (Cronet QUIC): Uses the Chromium network engine (Cronet) to send HTTP/3 requests over UDP. This bypasses TCP-layer RST (Reset) packets injected by firewalls because the traffic runs on QUIC/UDP rather than TCP.
    2. Image Loading (No-SNI TLS): Uses a customized TLS configuration to strip the SNI extension from the TLS ClientHello. Since the image servers (i.pximg.net) route requests based on IP and do not require SNI, removing it prevents the firewall from identifying the target domain via the TLS handshake.

    Architecture Summary:

    • API Path: RetrofitOkHttpCronetInterceptorCronet EngineQUIC/UDPCloudflare CDN
    • Image Path: GlideOkHttpRubySSLSocketFactory (No SNI) → HttpDnsTLS/TCPPixiv Image Servers
  4. Understand the Pixiv-Shaft WebSocket Connection Lifecycle

    classic

    The WebSocket connection follows a specific lifecycle. A connection is not considered 'connected' upon the HTTP 101 Switching Protocols response; the client must receive the first frame, which is always a hello frame, to transition the UI to a 'Connected' state.

    Connection Closure Behaviors:

    • Normal/Error Closure (onClose(1006, "")): Triggered by server terminate(), heartbeats, backpressure, or TCP exceptions. The client should initiate the reconnection process.
    • Session Replacement (onClose(1008, "replaced")): Triggered when a user logs in on another device, exceeding the concurrency limit. The client should NOT immediately reconnect to avoid infinite thrashing. Instead, notify the user (e.g., "Account logged in on another device") and allow them to manually reconnect.

    Note on State: The WebSocket is stateless. Upon reconnection, the server does not replay missed messages. Clients must use the /history?room=...&before=... endpoint to fetch missed data.

  5. Configure download paths and filenames using templates

    classic

    Shaft uses a template system to define how downloaded files are named and where they are stored. You can configure these settings in Settings → Download Path / Filename.

    Template Syntax Rules

    1. Plain text: Appears exactly as written (e.g., Shaft/Illusts/).
    2. {variable_name}: Replaced by actual content (e.g., {title}).
    3. [?condition:content]: The content only appears if the condition is met.
    4. /: Represents a sub-directory.

    Quick Start: Presets

    You can use one of the 4 built-in presets by clicking the purple "Apply" button on their respective cards:

    • Shaft Classic: Pictures/Shaft/Illusts/Author (ID)/Title ID p1.jpg
    • Flat: Pictures/Shaft/Title ID p1.jpg
    • Group by Date: Pictures/Shaft/YYYY/YYYY-MM/Title ID p1.jpg
    • Group by Author: Pictures/Shaft/Author (ID)/Title ID p1.jpg
    Shaft/Illusts/[?R18:R18/][?AI:AI/]{author} ({author_id})/{title} {id}[?p>1: p{page}].{ext}
  6. Manage Downloads and Batch Operations

    classic

    Downloading

    • R18 Storage: If Save R18 files separately is enabled, they will be stored in the ShaftImages-R18 directory within ShaftImages.
    • Deleting Records: Deleting a download record does not delete the actual file from storage.

    Batch Downloading

    • Tap Preview to enter batch mode.
    • Tap the bottom-right corner of individual items to select/deselect.
    • Long-press and drag to perform batch selection/deselection.
  7. Use Advanced UI Gestures and Shortcuts

    classic
    • Reload: Tap the navigation icon at the bottom to reload the current page.
    • Scroll to Top: Tap the top tab to scroll to the top of the page.
    • Side Drawer: Swipe right on a blank area (only on the first page; subsequent pages require edge-triggering).
    • Global Back Gesture: Swipe from the left edge of the screen to go back. Disable this in Settings -> Personalization -> Global Swipe Back if it conflicts with system gestures.

    Content Interaction

    • Waterfall Flow: Long-press an item to bring up a menu.
    • Quick Favorite: Long-press the heart icon to favorite by specific tags.
    • Work Details:
      • Toggle between New and Old UI.
      • Top-right menu: Copy work link or load original image.
      • Long-press work name, author name, or tags to copy or pin to search history.
      • Clickable Work ID and Author ID for quick copying.
    • Downloads: Long-press the download button to select image quality. Enable Settings -> Personalization -> Auto-favorite on Download to automatically favorite downloaded works.
    • Author Page: Click the author's name to copy their ID; long-press to copy the name.
  8. Implement WebSocket Reconnection Strategy

    classic

    The server does not replay missed messages upon reconnection. Clients must implement the following logic:

    1. Message Recovery: After reconnecting, manually fetch missed messages using GET /chat/history?room=...&before=<last_seen_id>.
    2. Unsent Messages: Do not automatically flush unsent local messages after reconnection to avoid duplicates. While the server supports idempotency via client_msg_id, it is better for UX to let the user manually confirm re-sending.
    3. Exponential Backoff: Use the following table for reconnection attempts. Reset the counter upon receiving a hello frame.
    AttemptBase WaitActual (±20% jitter)
    11s0.8–1.2s
    22s1.6–2.4s
    34s3.2–4.8s
    48s6.4–9.6s
    5+16–30sjitter ±20%
    1. Immediate Retry Scenarios:
    • NetworkMonitor transitions from OFFLINE to ONLINE.
    • App moves from background to foreground while in Disconnected state.
    • Received 503 shutting_down (wait 5–10s for the first retry).
  9. Handle Message Deduplication and Idempotency

    classic

    Because the server broadcasts messages immediately before they are committed to the database, network retries can cause the same message to be delivered multiple times. Clients must perform deduplication in the UI/local store layer using client_msg_id.

    Implementation Strategy

    1. Generate a unique ID: Use UUID v4, ULID, or nanoid (8–64 characters: [A-Za-z0-9_-]).
    2. Optimistic UI: When sending, immediately insert the message into the local store with a Sending state using the client_msg_id as the primary key.
    3. Upsert on Receipt: When a msg frame arrives via WebSocket, use an UPSERT operation (not INSERT) based on the client_msg_id. This will overwrite the 'Sending' entry with the actual delivered data, preventing duplicate rows.

    Example (Kotlin)

    // Use client_msg_id as the local unique key
    val localKey = entity.clientMsgId ?: "server:${entity.id}"
    
    // Use UPSERT to handle duplicates
    daos.upsertByKey(localKey, entity)
  10. Force uninstall Shaft using ADB

    classic

    If standard uninstallation fails, you can force a complete removal of Shaft using the Android Debug Bridge (ADB).

    Warning: Using ADB and enabling USB Debugging carries inherent risks to your device. Proceed at your own risk.

    Prerequisites

    1. Enable Developer Mode: Enable 'Developer Options' on your Android device.
    2. Enable USB Debugging: Find and turn on USB Debugging within Developer Options.
    3. Download Platform Tools: Download the SDK Platform Tools on a computer.
    4. Connect Device: Connect your phone to the computer via USB cable (ensure MTP mode is selected if prompted) and allow the USB debugging authorization on the phone's popup.

    Steps to Force Uninstall

    1. Open powershell or cmd in the directory containing adb.exe.
    2. Verify the connection by running:
      .
      adb devices
      You should see a device ID listed under list of devices attached.
    3. Run the following command to completely uninstall Shaft:
      .
      adb uninstall "ceui.lisa.pixiv"

    Post-Uninstallation Cleanup

    1. Once uninstalled, download the desired app-release.apk from the latest release assets.
    2. Install the new APK.
    3. Disconnect the USB cable and close the terminal.
    4. Security Cleanup: Disable USB Debugging, revoke USB debugging authorizations on the phone, and turn off Developer Options.
    .
    adb uninstall "ceui.lisa.pixiv"
  11. Configure OkHttp for WebSocket Connectivity

    classic

    To maintain a stable WebSocket connection and avoid premature timeouts or NAT idle disconnects, use the following OkHttp configuration.

    Key Recommendations:

    • Set readTimeout to 0 (infinite) because the connection is kept alive via heartbeats, not data flow.
    • Use pingInterval (e.g., 30 seconds) to allow the library to handle RFC-compliant control frames automatically.
    • Do not rely on application-layer {"kind":"ping"} frames for connection health; let the library handle RFC pings.
    val ws: OkHttpClient = OkHttpClient.Builder()
        .connectTimeout(10, TimeUnit.SECONDS)
        .readTimeout(0, TimeUnit.MILLISECONDS)    // WS 永不超时,靠 ping 探活
        .writeTimeout(10, TimeUnit.SECONDS)
        .pingInterval(30, TimeUnit.SECONDS)        // 跟服务端同 30s,任意一边 NAT 先 idle 时另一边的 ping 撞醒它
        .retryOnConnectionFailure(true)
        .build()