BiliSum Documentation
repository·master·Indexed 19 days ago
https://github.com/lycohana/bilisumAn AI-powered video summarization and knowledge base tool that transforms Bilibili, YouTube, and local videos into searchable interactive notes, mind maps, and visual notes using VLM and RAG. Supports local ASR via Alibaba FunASR (QwenASR), Twelve Labs Pegasus for video understanding, and various LLM providers. Available as a browser service via npx/Docker or as a desktop application.
What's inside BiliSum
- Starting from version v1.18.0, BiliSum supports the Alibaba FunASR (QwenASR) local speech recognition engine. This engine is optimized for Chinese recognition, offering performance that exceeds Whisper in accuracy, and operates at approximately 34x the CPU speed of Whisper.
BiliSum Light UI Design Principles
masterThe Light Mode design for BiliSum focuses on a bright, professional, and consumer-grade aesthetic, avoiding the heavy feel of traditional back-office interfaces.
Key Design Directions:
- Brand Identity: Uses a warm coral brand axis extracted from the logo (
#FB7299,#F85D8E,#FF9ABA) with low-saturation light backgrounds. - Workflow Emphasis: The primary operation area highlights the core loop:
Input Link -> Local Processing -> AI Summary. - Information Hierarchy: Data overviews are secondary to the main Call to Action (CTA). Low-frequency information (service status, version, hardware usage) is relegated to a status panel at the bottom of the sidebar to minimize distraction.
- Layout Structure:
- Sidebar: Narrow sidebar containing a Brand Card, navigation items with rounded selection states, and a bottom status panel.
- Main Content: Organized into a Header (Title + Service Badge), Primary Action Card (Input + CTA), Overview Card (Statistics), and Video Library (Search + Filters + Grid).
- Brand Identity: Uses a warm coral brand axis extracted from the logo (
Understand BiliSum packaging structure and runtime directories
masterWhen using the
onedirpackaging, the application is distributed as a directory containing the executable and its dependencies.Output Structure (
dist/BiliSum/)BiliSum.exe: The main executable._internal/: Internal PyInstaller dependencies.web/static/: Web assets.runtime/base/: The packaged seed runtime.bin/ffmpeg.exe: Bundled FFmpeg.
User Runtime and Data Directories
BiliSum uses specific directories in
%LOCALAPPDATA%for persistent data and managed runtimes:- User Data:
%LOCALAPPDATA%/bilisum/data(stores tasks, products, cookies, etc.). - Managed Runtime:
%LOCALAPPDATA%/bilisum/runtime(stores installed components like CUDA torch or ASR models). - GPU/CUDA Runtime:
%LOCALAPPDATA%/bilisum/runtime/gpu-cu12x(default path for CUDA-enabled components).
Understand BiliSum UI Design Principles
masterBiliSum's UI is designed for the Electron desktop app and web interfaces, focusing on a lightweight, professional, and transparent user experience. When building new features or pages, adhere to these core principles:
- Main Flow Priority: The core path is "Input Link $\rightarrow$ Local Processing $\rightarrow$ View Results". Ensure the primary action area is more prominent than overview data or system info.
- Lightweight Professionalism: Avoid heavy, dense "admin dashboard" styles. Aim for a clean, consumer-grade aesthetic without excessive decorative effects.
- State Transparency: All asynchronous processes must clearly communicate what is happening (e.g., running, success, failure, waiting) so the user knows the next step.
- Dual-Theme Consistency: Light mode is the default. Every new component must define its appearance for both light and dark themes (backgrounds, text, borders, and status colors).
- Extensibility: Reuse existing shells, cards, forms, and status patterns instead of creating new UI paradigms for settings, details, or task progress pages.
How VLM-based visual notes work
masterBiliSum offers a 'VLM Understanding' mode for visual notes. Unlike standard notes that simply insert screenshots, this mode uses a Vision Language Model (VLM) to read the original notes and screenshots to reconstruct the article structure objectively.
Key features:
- Structure: Paragraphs and images alternate (3-6 selected images) instead of stacking images at the end.
- Content: Uses frame descriptions to create objective fact lists rather than narrative summaries.
- Configuration: You can enable this in Settings → Visual Note Format → VLM Understanding Type.
- Cost Control: Since VLM calls are more expensive, you can adjust 'Maximum number of screenshots' or 'Minimum screenshot interval' in the settings.
- Supported Models: OpenAI, Anthropic, compatible interfaces, or custom endpoints.
Configure Bilibili subtitle priority and ASR fallback
masterBiliSum automatically handles subtitle acquisition with a specific priority hierarchy. If a video has available subtitles, the system will skip audio downloading and ASR (Automatic Speech Recognition) transcription to speed up the process.
Subtitle Priority Order:
- Subtitles uploaded by the UP loader
- Bilibili AI Chinese subtitles
- Other Chinese subtitles
- The first available subtitle
If no subtitles are found or if subtitle acquisition fails, the system automatically falls back to local ASR transcription. It is recommended to ensure the "优先使用 B 站字幕" (Prefer Bilibili subtitles) option is enabled in the settings (this is enabled by default).
Follow BiliSum Layout and Responsive Rules
masterAdhere to these structural rules for application shells and page templates.
Application Shell
- Top Title Bar:
40pxheight, fixed at top. - Left Sidebar:
248pxwidth, fixed position. - Main Content Area: Expands from the right of the sidebar, vertically scrollable.
- Content Container:
max-width: 1400pxwith default padding24px 32px 40px. - Settings Container:
max-width: 1560pxwith larger horizontal margins.
Page Templates
- Home / Video Library:
page header + hero/overview + list toolbar + video grid. Search/filter must be near the list title. - Settings Page: Dual-column layout (
left settings nav + right content area). - Details Page: Top content summary/cover $\rightarrow$ Middle main info stream $\rightarrow$ Right auxiliary info bar (status, metadata, actions).
- Dialogs / Overlays: Use semi-transparent dark mask with light blur. Dialog width $\approx$
540px.
Responsive Breakpoints
Breakpoint Rule 1260pxSettings sidebar shrinks to 280px1200pxHome dual-column becomes single-column; container narrows 1024pxSettings becomes single-column 900pxSidebar switches from fixed to top block 768pxVideo grid min-card width 200px; local forms become single-column640pxDialog bottom buttons stack vertically - Top Title Bar:
Migrate from BriefVid to BiliSum
masterIf you are upgrading from the older
BriefVidversion, BiliSum performs an automatic migration on the first launch.- Source Directory:
%LOCALAPPDATA%/briefvid - Destination Directory:
%LOCALAPPDATA%/bilisum - Migration Strategy: The process uses a "fill missing only" approach. It copies missing data, task products, Cookies, and managed runtimes from the old directory to the new one without overwriting existing data in the new directory or deleting the old directory.
- Electron Data: Desktop preferences and login sessions are migrated from the old Electron
BriefViduserData directory to the newBiliSumuserData directory.
- Source Directory:
Configure CUDA and ASR runtimes
masterThe base
onedirpackage is configured for CPU execution by default. To enable GPU acceleration or local ASR, users must install these components via the application's settings page.- CUDA Support: Installing via the settings page adds the CUDA version of
torchto the user's managed runtime directory (%LOCALAPPDATA%/bilisum/runtime/gpu-cu12x). - Local ASR: Local ASR is not distributed with the base package. It can be installed on demand via the settings page into the current runtime.
- Note: A restart of the application is required after installing these components for changes to take effect.
- CUDA Support: Installing via the settings page adds the CUDA version of
Follow BiliSum UI implementation constraints
masterWhen developing new components or pages, adhere to these technical constraints:
- Token First: Always reuse existing variables from
:root. Avoid hardcoding colors except for specific platform identity colors (e.g., YouTube/Bilibili tags). - Theme Consistency: Every new component must be validated for both Light and Dark themes. If a new token is added, define it for both themes.
- Structural Consistency: Reuse existing shell classes and layout patterns:
.content-frame.page-header.grid-card.primary-button/.secondary-button.settings-nav-item/.settings-toggle-row/.settings-toggle-label.toggle-switch/.toggle-slider.update-dialog
- Inline Styles: Only use inline styles for truly dynamic values (e.g., progress bar width, calculated positions). Do not use them for colors, font sizes, or shadows.
- Accessibility:
- All icon-only buttons must have an
aria-label. - Maintain text/background contrast (WCAG AA preferred).
- Error messages must use both color and an icon/text to ensure they aren't color-dependent.
- All icon-only buttons must have an
- Token First: Always reuse existing variables from
Build BiliSum using PyInstaller onedir
masterTo package BiliSum for Windows using the
onedirmode, run the provided build script. This script automates dependency installation, local package setup, generation of the sidecar managed runtime, collection offfmpegbinaries, and the PyInstaller execution.Prerequisites
- Python Version: Must use
3.12. - Local Packages: The current environment must be able to install the three local packages from this repository.
- FFmpeg: If you intend to distribute
ffmpegwith the package, ensureffmpeg.exeandffprobe.exeare in your systemPATH, or set theVIDEO_SUM_FFMPEG_DIRenvironment variable.
Build Command
Run the following command in PowerShell:
python .\packaging\pyinstaller\build_onedir.py- Python Version: Must use
Run BiliSum via npx
masterYou can start the local BiliSum browser service directly using
npx.Requirements:
- Python 3.12 must be installed on your system.
Behavior on first run: BiliSum will automatically create and manage a Python virtual environment located in your local app data directory.
# Start the BiliSum browser service npx bilisum # Start the service on a specific port npx bilisum start --port 3839 # Run diagnostic checks npx bilisum doctor