DOCSEARCHbyAlgolia
Submit Docs

OpenDBM Documentation

website·Indexed 31 days ago

https://aicure.github.io/open_dbm/

Documentation for OpenDBM, a pipeline for computer vision-based movement measurements and verbal acoustic analysis. It includes guides on installing OpenFace via Docker, using the Python and REST APIs, and detailed definitions for raw and derived variables such as harmonics-to-noise ratio, shimmer, and lexical richness.

Tokens
40.6K
Snippets
152
Records
358
Agent score
71%

What's inside OpenDBM

  1. Overview of OpenDBM digital biomarker measurement

    OpenDBM is an open-source tool designed to quantify behavioral characteristics from patient video and audio. It enables the objective measurement of facial activity, vocal acoustics, speech characteristics, movement patterns, and oculomotion to derive clinically meaningful symptomatology such as emotional expressivity, voice prosody, speech valence, and tremor severity.

  2. Overview of speech biomarker extraction in open_dbm

    The open_dbm framework measures language characteristics by first transcribing detected speech from audio files into text using the open-source DeepSpeech tool. Once transcribed, various open-source natural language processing (NLP) tools are applied to the text to derive speech biomarkers. For analysis purposes, it is recommended to use the derived variables, as there is no functional difference between the raw transcribed speech and the derived variables in this pipeline.

  3. Overview of OpenDBM Digital Biomarker Library

    OpenDBM is a Python-based software package used to calculate digital biomarkers from video and audio recordings of human behavior. It integrates multiple tools to measure behavioral characteristics—including facial activity, vocal acoustics, speech characteristics, movement patterns, and oculomotion—to derive clinically meaningful symptomatology such as emotional expressivity, voice prosody, speech valence, and tremor severity.

  4. Overview of OpenDBM

    OpenDBM is an open-source tool designed to measure digital biomarkers from patient behavior captured via video and audio. It quantifies behavioral characteristics such as facial activity, vocal acoustics, speech characteristics, movement patterns, and oculomotion to measure clinically meaningful symptomatology (e.g., emotional expressivity, voice prosody, speech valence, and tremor severity).

  5. Overview of OpenDBM digital biomarker measurement

    OpenDBM is an open-source tool designed to quantify digital biomarkers from patient video and audio. It enables the objective measurement of behavioral characteristics—including facial activity, vocal acoustics, speech characteristics, movement patterns, and oculomotion—to derive clinically meaningful symptomatology such as emotional expressivity, voice prosody, speech valence, and tremor severity.

  6. Identify audio pause and speech timing variables

    The system determines the presence of voice based on the fundamental frequency (which is typically zero when no voice is detected). This is used to calculate the following raw metrics for characterizing pauses and silence in an audio file:

    • aco_pausetime: Total length of time where no speech was detected.
    • aco_totaltime: Total length of the video/audio file.
    • aco_speakingtime: Total length of time where speech was detected.
    • aco_numpauses: The total number of instances where no speech was detected.
    • aco_pausefrac: The fraction of the total time spent in pauses (aco_pausetime / aco_totaltime).

  7. Understand OpenDBM output folder structure

    When processing data with OpenDBM, the system generates an output folder at the specified location. This folder is divided into two primary categories:

    1. raw_variables: Contains frame-wise measurements based on the temporal resolution of the input file (e.g., happiness expressivity per frame in a video or audio intensity per frame in an audio file).
    2. derived_variables: Contains abstractions of the raw variables, such as averages or standard deviations calculated across the entire duration of the input file.

  8. Prerequisites for using OpenDBM

    To use OpenDBM, users should have a fundamental understanding of Python. While the documentation attempts to assume minimal prior knowledge, familiarity with Python basics is recommended for effective use of the tool.

  9. Analyze speech and pause characteristics using Raw Variables

    The system identifies pauses and speech by detecting the presence or absence of fundamental frequency (which is typically zero when no voice is detected). The following raw variables quantify these metrics for a single video:
    VariableDescription
    aco_pausetimeLength of time with no speech detected.
    aco_totaltimeThe total length of the video.
    aco_speakingtimeThe total length of time with speech detected.
    aco_numpausesNumber of instances with no speech.
    aco_pausefracPause time (aco_pausetime) divided by total time (aco_totaltime).

  10. Understand eye blink measurement using Eye Aspect Ratio (EAR)

    Eye blinks are measured by calculating the Eye Aspect Ratio (EAR), which quantifies how open the eye is. During a video, the EAR is represented as a vector where troughs typically signify individual eye blinks. These troughs are identified using a 'find peaks' function to extract specific raw variables for each detected blink.

  11. Understand the OpenDBM output directory structure

    When processing data with OpenDBM, the tool generates an output folder containing two primary categories of data: raw_variables and derived_variables.

    • Derived Variables: High-level abstractions (e.g., averages or standard deviations) of raw measurements. These are stored in a single CSV file named derived_output.csv, where each row corresponds to one processed input file.
    • Raw Variables: Frame-wise measurements based on the temporal resolution of the input file (e.g., happiness expressivity per frame). These are stored in a hierarchical folder structure: raw_variables -> <filename> -> <DBM group> (facial, acoustic, speech, or movement) -> <biomarker> -> .csv file.

  12. Manage Python environments for Open DBM development

    For developers new to Python environments, using the Conda CLI is recommended. Conda is a cross-platform (Windows, macOS, Linux, z/OS) package and environment management system that simplifies installing, running, and updating packages and their dependencies. Experienced Python developers may proceed using their existing Python environment.