LiveRecall

repository·main·Indexed 41 days ago

https://github.com/vedankpurohit/liverecall

An open-source screen recall application featuring semantic search, OCR, and image embeddings to find past screenshots using natural language queries. It operates as a privacy-first, local-only system with a FastAPI-based REST API and a system tray interface for macOS and Windows.

Tokens
5.9K
Snippets
19
Records
31
Agent score
51%

What's inside LiveRecall

  1. Understand LiveRecall's privacy and local execution model

    main
    LiveRecall is designed as a privacy-first, local-only application. It operates entirely on your local machine without requiring internet access for its core functionalities. Once initial dependencies are downloaded, all processing occurs offline, ensuring no data is transmitted to remote servers or accessed by the developers.
  2. How image encryption and security works in LiveRecall

    main

    LiveRecall secures captured screenshots using a local encryption mechanism:

    1. Encryption: Screenshots are encrypted using a password provided by the user.
    2. Password Management: The password is not stored by the application; you must remember it to access your data.
    3. Decryption: Images are only decrypted temporarily during retrieval for viewing.
    4. Secure Deletion: Decrypted images are promptly deleted after use to prevent unauthorized access.
  3. Understand LiveRecall search modes

    main

    LiveRecall uses a multi-modal approach to find screenshots. You can specify the following search modes:

    • Auto (Hybrid): The default mode. It combines all search methods using Reciprocal Rank Fusion (RRF) for the best results.
    • Image: Performs visual similarity search (e.g., searching for "cat on a keyboard").
    • Text Fuzzy: Performs fast text matching with typo tolerance using trigram search.
    • Text Semantic: Uses embeddings to find screenshots containing text with similar meaning to your query.
  4. Build LiveRecall distributable packages

    main

    Use the provided build scripts to create .dmg (macOS) or .exe (Windows) installers. The output is placed in the dist/ directory.

    # One-command build (installs deps, builds web, creates package)
    uv run python scripts/build_release.py
    
    # Quick build (skip web rebuild if exists)
    uv run python scripts/build_release.py --quick
    # One-command build (installs deps, builds web, creates package)
    uv run python scripts/build_release.py
    
    # Quick build (skip web rebuild if exists)
    uv run python scripts/build_release.py --quick
  5. Set up LiveRecall for Development

    main

    To run LiveRecall from source, ensure you have Python 3.10+, Node.js 18+, and uv installed. Follow these steps:

    1. Clone the repository and enter the directory.
    2. Use uv sync to install Python dependencies and create a virtual environment (.venv).
    3. Navigate to the web directory and run npm install to set up the web UI.
    # Clone the repository
    git clone https://github.com/VedankPurohit/LiveRecall.git
    cd LiveRecall
    
    # Install Python dependencies (creates .venv automatically)
    uv sync
    
    # Install web UI dependencies
    cd web && npm install && cd ..
    # Clone the repository
    git clone https://github.com/VedankPurohit/LiveRecall.git
    cd LiveRecall
    
    # Install Python dependencies (creates .venv automatically)
    uv sync
    
    # Install web UI dependencies
    cd web && npm install && cd ..
  6. Run LiveRecall in different modes

    main

    Depending on your needs, you can run the application in three different modes:

    • System Tray App (Recommended): Launches the full menu bar application.
    • API Server Only: Runs only the backend service.
    • Web UI Development: Runs the API in one terminal and the Next.js web interface in another.
    # Option 1: System Tray App
    uv run python main.py
    
    # Option 2: API Server Only
    uv run python main.py --api-only
    
    # Option 3: Web UI Development
    # Terminal 1: Start API
    uv run python main.py --api-only
    
    # Terminal 2: Start web UI
    cd web && npm run dev

    If running in development mode, the web UI is accessible at http://localhost:3000.

    # Option 1: System Tray App
    uv run python main.py
    
    # Option 2: API Server Only
    uv run python main.py --api-only
    
    # Option 3: Web UI Development
    # Terminal 1: Start API
    uv run python main.py --api-only
    
    # Terminal 2: Start web UI
    cd web && npm run dev
  7. Install LiveRecall on macOS (Apple Silicon)

    main

    To use the pre-built application on macOS:

    1. Download the latest DMG from the Releases page.
    2. Open the DMG and drag LiveRecall to your Applications folder.
    3. Launch the app from your Applications.
    4. Important: You must grant Screen Recording permission in System Settings > Privacy & Security > Screen Recording.

    Note: On the first launch, the app will automatically download approximately 530MB of AI models (Image embedding ~400MB and Text embedding ~130MB).

  8. Configure LiveRecall settings

    main

    Settings can be adjusted via the Web UI or through the API. Available configuration keys include:

    • Capture Mode: normal, games, fast, coding, video, presentation
    • Capture Interval: 1-10 seconds
    • Quality: 50-100%
    • Safe Mode: Filter sensitive content from search
    • Auto-compress: Compress old screenshots
    • OCR Enabled: Enable/disable text extraction (enabled by default)
    • Search Mode: auto (hybrid), image, text_fuzzy, text_semantic
  9. How TrayApp manages setup and migrations

    main

    Upon startup, TrayApp performs an enhanced setup check by querying the /setup/enhanced-status endpoint. It automatically opens the web UI setup page ({WEB_UI_URL}/setup) if any of the following conditions are met:

    1. Version Change: The needs_setup flag is true, indicating a version change was detected (e.g., last_seen_version != current_version).
    2. Missing CLIP Model: The clip_status is reported as not_downloaded, which is required for search functionality.
    3. OCR Migration: The migration_status indicates needs_migration is true (e.g., there are screenshots without OCR that need processing).
  10. Understand the LiveRecall API workflow

    main

    The LiveRecall API manages screen recording, semantic synchronization, and natural language search. The typical developer/user workflow follows these steps:

    1. Start Recording: Use POST /api/v1/recording/start to begin capturing screenshots automatically when the screen changes. Note that screenshots are saved but not immediately processed to keep the recording lightweight.
    2. Sync Data: When you are ready to enable semantic search, run POST /api/v1/sync/start. This generates CLIP embeddings for the captured screenshots.
    3. Search: Use POST /api/v1/search to find screenshots using natural language queries.

    Model Management: The CLIP model is loaded lazily upon the first sync or search operation and automatically unloads after 5 minutes of inactivity to conserve system memory.

    # Typical Workflow Summary
    # 1. POST /api/v1/recording/start
    # 2. POST /api/v1/sync/start
    # 3. POST /api/v1/search
  11. Reference the LiveRecall API endpoints

    main

    The backend provides a FastAPI-based REST API. The full interactive documentation is available at http://localhost:8742/docs when the server is running.

    EndpointMethodDescription
    /api/v1/statusGETSystem status
    /api/v1/recording/startPOSTStart capture
    /api/v1/recording/stopPOSTStop capture
    /api/v1/sync/startPOSTStart embedding/OCR sync
    /api/v1/searchPOSTHybrid search (supports search_mode parameter)
    /api/v1/screenshotsGETList screenshots
    /api/v1/screenshots/{id}/ocrGETGet OCR text for a screenshot