BiliSum Documentation

repository·master·Indexed 19 days ago

https://github.com/lycohana/bilisum

An AI-powered video summarization and knowledge base tool that transforms Bilibili, YouTube, and local videos into searchable interactive notes, mind maps, and visual notes using VLM and RAG. Supports local ASR via Alibaba FunASR (QwenASR), Twelve Labs Pegasus for video understanding, and various LLM providers. Available as a browser service via npx/Docker or as a desktop application.

Tokens
34K
Snippets
93
Records
142
Agent score
65%

What's inside BiliSum

  1. Use Alibaba FunASR (QwenASR) for local speech recognition

    master
    Starting from version v1.18.0, BiliSum supports the Alibaba FunASR (QwenASR) local speech recognition engine. This engine is optimized for Chinese recognition, offering performance that exceeds Whisper in accuracy, and operates at approximately 34x the CPU speed of Whisper.
  2. BiliSum Light UI Design Principles

    master

    The Light Mode design for BiliSum focuses on a bright, professional, and consumer-grade aesthetic, avoiding the heavy feel of traditional back-office interfaces.

    Key Design Directions:

    • Brand Identity: Uses a warm coral brand axis extracted from the logo (#FB7299, #F85D8E, #FF9ABA) with low-saturation light backgrounds.
    • Workflow Emphasis: The primary operation area highlights the core loop: Input Link -> Local Processing -> AI Summary.
    • Information Hierarchy: Data overviews are secondary to the main Call to Action (CTA). Low-frequency information (service status, version, hardware usage) is relegated to a status panel at the bottom of the sidebar to minimize distraction.
    • Layout Structure:
      • Sidebar: Narrow sidebar containing a Brand Card, navigation items with rounded selection states, and a bottom status panel.
      • Main Content: Organized into a Header (Title + Service Badge), Primary Action Card (Input + CTA), Overview Card (Statistics), and Video Library (Search + Filters + Grid).
  3. Understand BiliSum packaging structure and runtime directories

    master

    When using the onedir packaging, the application is distributed as a directory containing the executable and its dependencies.

    Output Structure (dist/BiliSum/)

    • BiliSum.exe: The main executable.
    • _internal/: Internal PyInstaller dependencies.
    • web/static/: Web assets.
    • runtime/base/: The packaged seed runtime.
    • bin/ffmpeg.exe: Bundled FFmpeg.

    User Runtime and Data Directories

    BiliSum uses specific directories in %LOCALAPPDATA% for persistent data and managed runtimes:

    • User Data: %LOCALAPPDATA%/bilisum/data (stores tasks, products, cookies, etc.).
    • Managed Runtime: %LOCALAPPDATA%/bilisum/runtime (stores installed components like CUDA torch or ASR models).
    • GPU/CUDA Runtime: %LOCALAPPDATA%/bilisum/runtime/gpu-cu12x (default path for CUDA-enabled components).
  4. Understand BiliSum UI Design Principles

    master

    BiliSum's UI is designed for the Electron desktop app and web interfaces, focusing on a lightweight, professional, and transparent user experience. When building new features or pages, adhere to these core principles:

    • Main Flow Priority: The core path is "Input Link $\rightarrow$ Local Processing $\rightarrow$ View Results". Ensure the primary action area is more prominent than overview data or system info.
    • Lightweight Professionalism: Avoid heavy, dense "admin dashboard" styles. Aim for a clean, consumer-grade aesthetic without excessive decorative effects.
    • State Transparency: All asynchronous processes must clearly communicate what is happening (e.g., running, success, failure, waiting) so the user knows the next step.
    • Dual-Theme Consistency: Light mode is the default. Every new component must define its appearance for both light and dark themes (backgrounds, text, borders, and status colors).
    • Extensibility: Reuse existing shells, cards, forms, and status patterns instead of creating new UI paradigms for settings, details, or task progress pages.
  5. How VLM-based visual notes work

    master

    BiliSum offers a 'VLM Understanding' mode for visual notes. Unlike standard notes that simply insert screenshots, this mode uses a Vision Language Model (VLM) to read the original notes and screenshots to reconstruct the article structure objectively.

    Key features:

    • Structure: Paragraphs and images alternate (3-6 selected images) instead of stacking images at the end.
    • Content: Uses frame descriptions to create objective fact lists rather than narrative summaries.
    • Configuration: You can enable this in Settings → Visual Note Format → VLM Understanding Type.
    • Cost Control: Since VLM calls are more expensive, you can adjust 'Maximum number of screenshots' or 'Minimum screenshot interval' in the settings.
    • Supported Models: OpenAI, Anthropic, compatible interfaces, or custom endpoints.
  6. Configure Bilibili subtitle priority and ASR fallback

    master

    BiliSum automatically handles subtitle acquisition with a specific priority hierarchy. If a video has available subtitles, the system will skip audio downloading and ASR (Automatic Speech Recognition) transcription to speed up the process.

    Subtitle Priority Order:

    1. Subtitles uploaded by the UP loader
    2. Bilibili AI Chinese subtitles
    3. Other Chinese subtitles
    4. The first available subtitle

    If no subtitles are found or if subtitle acquisition fails, the system automatically falls back to local ASR transcription. It is recommended to ensure the "优先使用 B 站字幕" (Prefer Bilibili subtitles) option is enabled in the settings (this is enabled by default).

  7. Follow BiliSum Layout and Responsive Rules

    master

    Adhere to these structural rules for application shells and page templates.

    Application Shell

    • Top Title Bar: 40px height, fixed at top.
    • Left Sidebar: 248px width, fixed position.
    • Main Content Area: Expands from the right of the sidebar, vertically scrollable.
    • Content Container: max-width: 1400px with default padding 24px 32px 40px.
    • Settings Container: max-width: 1560px with larger horizontal margins.

    Page Templates

    • Home / Video Library: page header + hero/overview + list toolbar + video grid. Search/filter must be near the list title.
    • Settings Page: Dual-column layout (left settings nav + right content area).
    • Details Page: Top content summary/cover $\rightarrow$ Middle main info stream $\rightarrow$ Right auxiliary info bar (status, metadata, actions).
    • Dialogs / Overlays: Use semi-transparent dark mask with light blur. Dialog width $\approx$ 540px.

    Responsive Breakpoints

    BreakpointRule
    1260pxSettings sidebar shrinks to 280px
    1200pxHome dual-column becomes single-column; container narrows
    1024pxSettings becomes single-column
    900pxSidebar switches from fixed to top block
    768pxVideo grid min-card width 200px; local forms become single-column
    640pxDialog bottom buttons stack vertically
  8. Migrate from BriefVid to BiliSum

    master

    If you are upgrading from the older BriefVid version, BiliSum performs an automatic migration on the first launch.

    • Source Directory: %LOCALAPPDATA%/briefvid
    • Destination Directory: %LOCALAPPDATA%/bilisum
    • Migration Strategy: The process uses a "fill missing only" approach. It copies missing data, task products, Cookies, and managed runtimes from the old directory to the new one without overwriting existing data in the new directory or deleting the old directory.
    • Electron Data: Desktop preferences and login sessions are migrated from the old Electron BriefVid userData directory to the new BiliSum userData directory.
  9. Configure CUDA and ASR runtimes

    master

    The base onedir package is configured for CPU execution by default. To enable GPU acceleration or local ASR, users must install these components via the application's settings page.

    • CUDA Support: Installing via the settings page adds the CUDA version of torch to the user's managed runtime directory (%LOCALAPPDATA%/bilisum/runtime/gpu-cu12x).
    • Local ASR: Local ASR is not distributed with the base package. It can be installed on demand via the settings page into the current runtime.
    • Note: A restart of the application is required after installing these components for changes to take effect.
  10. Follow BiliSum UI implementation constraints

    master

    When developing new components or pages, adhere to these technical constraints:

    1. Token First: Always reuse existing variables from :root. Avoid hardcoding colors except for specific platform identity colors (e.g., YouTube/Bilibili tags).
    2. Theme Consistency: Every new component must be validated for both Light and Dark themes. If a new token is added, define it for both themes.
    3. Structural Consistency: Reuse existing shell classes and layout patterns:
      • .content-frame
      • .page-header
      • .grid-card
      • .primary-button / .secondary-button
      • .settings-nav-item / .settings-toggle-row / .settings-toggle-label
      • .toggle-switch / .toggle-slider
      • .update-dialog
    4. Inline Styles: Only use inline styles for truly dynamic values (e.g., progress bar width, calculated positions). Do not use them for colors, font sizes, or shadows.
    5. Accessibility:
      • All icon-only buttons must have an aria-label.
      • Maintain text/background contrast (WCAG AA preferred).
      • Error messages must use both color and an icon/text to ensure they aren't color-dependent.
  11. Build BiliSum using PyInstaller onedir

    master

    To package BiliSum for Windows using the onedir mode, run the provided build script. This script automates dependency installation, local package setup, generation of the sidecar managed runtime, collection of ffmpeg binaries, and the PyInstaller execution.

    Prerequisites

    • Python Version: Must use 3.12.
    • Local Packages: The current environment must be able to install the three local packages from this repository.
    • FFmpeg: If you intend to distribute ffmpeg with the package, ensure ffmpeg.exe and ffprobe.exe are in your system PATH, or set the VIDEO_SUM_FFMPEG_DIR environment variable.

    Build Command

    Run the following command in PowerShell:

    python .\packaging\pyinstaller\build_onedir.py
  12. Run BiliSum via npx

    master

    You can start the local BiliSum browser service directly using npx.

    Requirements:

    • Python 3.12 must be installed on your system.

    Behavior on first run: BiliSum will automatically create and manage a Python virtual environment located in your local app data directory.

    # Start the BiliSum browser service
    npx bilisum
    
    # Start the service on a specific port
    npx bilisum start --port 3839
    
    # Run diagnostic checks
    npx bilisum doctor