repository·main·Indexed 27 days ago
https://github.com/m87-labs/moondreamA family of tiny, efficient open-source vision language models (VLM) designed for image captioning, visual question answering (VQA), and object detection. Available in Moondream 2B for general-purpose understanding and Moondream 0.5B for edge devices. The project includes specialized recipes for gaze detection, promptable content moderation, and promptable video redaction, supporting deployment from local hardware to the cloud via Modal.
The CLI allows for automated batch processing of videos.
inputs directory in the script folder..mp4, .avi, .mov, .mkv, .webm) into the inputs directory.python main.py to process all videos in the folder.mkdir inputs
# Place videos in ./inputs/
python main.pyTo set up the content moderation tool, clone the repository, create a virtual environment, and install the necessary system and Python dependencies.
You must have ffmpeg and libvips installed on your system:
sudo apt-get install ffmpeg libvipsbrew install ffmpeg libvipsClone the repository and initialize a virtual environment:
git clone https://github.com/vikhyat/moondream/blob/main/recipes/promptable-video-redaction
python -m venv .venv
source .venv/bin/activate # On Windows: .venv\Scripts\activateWindows Users: You must install PyTorch with CUDA support first:
pip install torch==2.5.1+cu121 torchvision==0.20.1+cu121 --index-url https://download.pytorch.org/whl/cu121Then, install the remaining requirements:
pip install -r requirements.txtgit clone https://github.com/vikhyat/moondream/blob/main/recipes/promptable-video-redaction
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txtThe easiest way to moderate video content is via the built-in web interface.
python app.pyhttp://localhost:7860) in your browser.obfuscated-pixel).python app.pyThe Gaze Detection Video Processor automates face detection and gaze direction tracking in video files.
.mp4, .avi, .mov, or .mkv) into the input directory. The directory will be created automatically if it does not exist.output directory with the prefix processed_ (e.g., output/processed_video.mp4).Each output frame includes colored bounding boxes for faces, lines for gaze direction, and points indicating the gaze target.
Windows installation requires manual setup of libvips and FFmpeg to ensure proper GPU support and video processing.
vips-dev-w64-all-8.16.0.zip for 64-bit x64).vips-dev-8.16\bin to your project root or C:\Windows\System32.vips-dev-8.16\bin directory to your system PATH.bin folder to your system PATH or project root.# Clone and setup environment
git clone [repository-url]
cd moondream/recipes/gaze-detection-video
python -m venv venv
.\venv\Scripts\activate
# Install PyTorch with CUDA support
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
# Install other dependencies
pip install -r requirements.txtTo optimize your moderation workflow, follow these recommendations:
Censor: Use for complete content blocking.Blur styles: Use for less intrusive moderation.Bounding Box: Use for content review and manual analysis.To install the Gaze Detection Video Processor on Linux or macOS, you must first install the system dependencies libvips and ffmpeg, then set up a Python virtual environment and install the project requirements.
sudo apt-get update && sudo apt-get install -y libvips42 libvips-dev ffmpegsudo yum install vips vips-devel ffmpegbrew install vips ffmpegClone the repository, navigate to the recipe directory, and install the Python dependencies:
git clone https://github.com/vikhyat/moondream.git
cd moondream/recipes/gaze-detection-video
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt# Ubuntu/Debian
sudo apt-get update && sudo apt-get install -y libvips42 libvips-dev ffmpeg
# CentOS/RHEL
sudo yum install vips vips-devel ffmpeg
# macOS
brew install vips ffmpeg
# Project Setup
git clone https://github.com/vikhyat/moondream.git
cd moondream/recipes/gaze-detection-video
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txtTo set up the Promptable Video Redaction tool, clone the repository, create a virtual environment, and install the dependencies. You must also have ffmpeg installed on your system.
System Requirements:
ffmpeg (installed via apt-get, brew, or manual download)libvips (required for Windows users)Setup Steps:
requirements.txt.ffmpeg using your OS package manager.git clone https://github.com/vikhyat/moondream/blob/main/recipes/promptable-video-redaction
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txtThe tool provides a Gradio-based web interface for users who prefer a GUI over the command line.
python app.py.python app.pyThe Promptable Video Redaction application allows you to upload a video and use natural language to detect and redact/visualize objects (e.g., 'face', 'car', 'logo').
censor: Redacts the detected object.bounding-box: Draws a box around the object.hitmarker: Applies a hitmarker style.rows and cols to process video in a grid format.ffmpeg_preset (e.g., ultrafast, medium, slow). Faster presets result in lower quality but quicker processing.rows and cols (1-4) to control the grid layout.