DOCSEARCHbyAlgolia
Submit Docs

MIMIC Code Repository

repository·main·Indexed 25 days ago

https://github.com/mit-lcp/mimic-code

A central hub for sharing and reusing code to analyze MIMIC critical care databases, including MIMIC-III, MIMIC-IV, and MIMIC-CXR. It provides build scripts, derived concepts, and tutorials to enable reproducibility in clinical research. The repository includes guides for database optimization (indexing and partitioning), as well as deployment and loading instructions for Apache Drill, AWS Athena, and Google BigQuery.

Tokens
51.6K
Snippets
154
Records
246
Agent score
83%

What's inside mimic-code

  1. Overview of the MIMIC-III repository structure

    main

    The MIMIC-III repository provides SQL scripts to build the MIMIC-III database in various systems and extract key clinical concepts. It also includes Jupyter notebooks for data analysis and tutorials for new users.

    Key components include:

    • buildmimic: Scripts for building the database in an RDMS (PostgreSQL is the preferred system).
    • concepts: SQL views and summaries for demographics, organ failure scores, severity of illness, and treatment durations.
    • notebooks: R markdown and Jupyter notebooks for data extraction and analysis examples.
    • tutorials: Educational notebooks designed to explain concepts to new users.
    • benchmark: Speed tests for database indices.
    • tests: Test suites for the repository components.

  2. Overview of MIMIC-IV-Note

    main

    MIMIC-IV-Note contains deidentified free-text clinical notes for patients in MIMIC-IV, including discharge summaries and radiology reports.

    Note: This is a separate download from the main MIMIC-IV dataset and is available on PhysioNet. Currently, there are no derived concept scripts available for MIMIC-IV-Note.

  4. Overview of MIMIC-IV-ED

    main
    MIMIC-IV-ED is a publicly available database containing emergency department visit data. It is a module within the larger MIMIC-IV dataset ecosystem. For detailed documentation regarding the dataset structure and contents, refer to the PhysioNet project page.

  5. Overview of MIMIC-III Key Tables

    main

    MIMIC-III is a deidentified health dataset. When querying the database, the following tables are central to most analyses:

    • patients: Contains patient demographics. Each patient is identified by a unique subject_id.
    • admissions: Contains hospital admission records. Each admission is identified by a unique hadm_id.
    • icustays: Contains ICU stay records. Each stay is identified by a unique icustay_id.

    For specific column definitions, refer to the official MIMIC documentation.

  6. Understand Clinical Classification Software (CCS) diagnoses

    main
    The Clinical Classification Software (CCS) is used to categorize ICD-9 coded diagnoses into clinically meaningful groups, developed by the Agency for Healthcare Research and Quality (AHRQ). In the MIMIC-III dataset, this information is represented by the ccs_dx concept. The processed data file ccs_multi_dx.csv.gz is intended to be uploaded to the BigQuery table physionet-data.mimiciii_derived.ccs_multi_dx.

  7. Quickstart: Load MIMIC-IV into PostgreSQL

    main

    To quickly set up MIMIC-IV in a PostgreSQL database, clone the repository, download the PhysioNet data, create a database, and run the schema and loading scripts in sequence. This process assumes you have wget, psql, and createdb available in your environment.

    # clone repo
git clone https://github.com/MIT-LCP/mimic-code.git
cd mimic-code

# download data
wget -r -N -c -np --user <USERNAME> --ask-password https://physionet.org/files/mimiciv/3.1/
mv physionet.org/files/mimiciv mimiciv && rmdir physionet.org/files && rm physionet.org/robots.txt && rmdir physionet.org

# setup database and load
createdb mimiciv
psql -d mimiciv -f mimic-iv/buildmimic/postgres/create.sql
psql -d mimiciv -v ON_ERROR_STOP=1 -v mimic_data_dir=mimiciv/3.1 -f mimic-iv/buildmimic/postgres/load_gz.sql
psql -d mimiciv -v ON_ERROR_STOP=1 -v mimic_data_dir=mimiciv/3.1 -f mimic-iv/buildmimic/postgres/constraint.sql
psql -d mimiciv -v ON_ERROR_STOP=1 -v mimic_data_dir=mimiciv/3.1 -f mimic-iv/buildmimic/postgres/index.sql

  8. Install NegBio for MIMIC-CXR

    main

    Follow these steps to set up the environment and the specific NegBio codebase required for MIMIC-CXR:

    1. Create the Conda environment from the provided configuration file:

      conda env create -f environment.yml

    2. Clone the MIMIC-CXR branch of the NegBio repository:

      git clone --single-branch --branch MIMIC-CXR https://github.com/ncbi-nlp/NegBio.git negbio-mimic-cxr

    3. Checkout the specific commit used for label generation to ensure reproducibility:

      cd negbio-mimic-cxr
git checkout 962690b6789920fb0abab4fe05fc8ce6bc1a349d
cd ..
    conda env create -f environment.yml
git clone --single-branch --branch MIMIC-CXR https://github.com/ncbi-nlp/NegBio.git negbio-mimic-cxr
cd negbio-mimic-cxr; git checkout 962690b6789920fb0abab4fe05fc8ce6bc1a349d; cd ..

  9. Requirements for the aline study reproduction

    main

    To run the indwelling arterial catheter study reproduction, ensure the following environment is configured:

    Database

    • An installation of MIMIC-III in a PostgreSQL database.

    Python Environment

    • Python 2.7
    • Required packages: numpy, pandas, matplotlib, and psycopg2.
    • Jupyter with a Python 2 kernel.

    R Environment

    • R with the following libraries installed:
      • Matching
      • pROC
      • MASS

  10. Prepare NOTEEVENTS data for Oracle loading

    main

    Oracle cannot handle standard CSVs where newlines are contained within quoted fields. To load NOTEEVENTS.csv, you must use the add_oracle_rowdelimiter.py script to append a unique row delimiter to each row. This creates a file named NOTEEVENTS_output.csv which can then be loaded using the provided noteevents_output.ctl control file.

    python add_oracle_rowdelimiter.py -d ',' -i 'NOTEEVENTS.csv' -r '><><?~`;;`'

  11. Download MIMIC-III CSV files

    main

    To use the DuckDB import scripts, you must first download the MIMIC-III CSV files (or the demo version). The easiest method is using wget.

    Note: Replace YOURUSERNAME with your PhysioNet username. The script expects the files to be in a directory structure where the .csv or .csv.gz files are directly inside the target folder (e.g., mimic_data_dir/ADMISSIONS.csv.gz).

    wget -r -N -c -np -nH --cut-dirs=1 --user YOURUSERNAME --ask-password https://physionet.org/files/mimiciii/1.4/

  12. Contribute corrections to DuckDB concepts

    main

    The SQL files in this directory are automatically generated from BigQuery SQL dialect using sqlglot.

    Do not make corrections directly in this folder. To contribute a correction, you must make your changes in the /mimic-iv/concepts/ folder using the BigQuery SQL syntax. The DuckDB files will then be updated via the automated generation process.