DOCSEARCHbyAlgolia
Submit Docs

Clinical Trial Parser

repository·main·Indexed 41 days ago

https://github.com/facebookresearch/clinical-trial-parser

A library that converts unstructured clinical trial eligibility criteria into machine-readable nominal, ordinal, and numerical relations using Context-Free Grammar (CFG) and Information Extraction (IE) techniques. It features a pre-trained Caffe2 medical NER model, MeSH vocabulary augmentation, and a pipeline for processing inclusion and exclusion criteria into structured JSON output.

Tokens
13.7K
Snippets
49
Records
67
Agent score
52%

What's inside Clinical Trial Parser

  1. Available scripts for clinical-trial-parser modules

    main

    The script/ directory provides shell scripts to automate various stages of the clinical trial parsing pipeline, including data downloading, ingestion, parsing, and embedding training.

    Available scripts:

    • cfg_parse.sh: Parses eligibility criteria using Context-Free Grammars (CFG).
    • ie_parse.sh: Parses eligibility criteria using Information Extraction (IE).
    • aact.sh: Downloads the AACT (Aggregate Analysis of ClinicalTrials.gov) database.
    • mesh.sh: Downloads MeSH (Medical Subject Headings) descriptors for grounding tasks.
    • ingest.sh: Ingests clinical trial eligibility criteria from the AACT database into a CSV file.
    • train_embeddings.sh: Ingests clinical trial text and trains word embeddings.
    • search.sh: A CLI tool used to search for concepts within a vocabulary.

  2. Understand the three types of clinical trial requirements

    main

    The parser categorizes extracted requirements into three distinct variable types, each handled by different architectural components:

    1. Nominal requirements: Requirements involving a medical variable (e.g., leukemia, allergy) and a condition (e.g., must have or cannot have). These are processed using Information Extraction (IE) techniques (NER and NEL).
    2. Ordinal requirements: Requirements involving an ordinal variable (e.g., ECOG score) and a set of eligible values (e.g., 0, 1, or 2). These are processed via Context-Free Grammar (CFG).
    3. Numerical requirements: Requirements involving a numerical variable (e.g., age, BMI) and an eligible range (e.g., ≥18 and ≤ 35). These are also processed via CFG.

  3. Understand the Clinical Trial Parser architecture and approach

    main

    The Clinical Trial Parser converts clinical trial eligibility criteria (inclusion and exclusion) into machine-readable relations. It uses a combination of Context-Free Grammar (CFG) and Information Extraction (IE) techniques.

    Core Concepts

    • Variable: An abstraction representing a basic unit of clinical or demographic information (e.g., age, BMI, or a specific medical condition) extracted from a criterion.
    • Eligibility Logic: The parser treats eligibility as a Boolean algebra problem. Inclusion (IC) and exclusion (EC) criteria are combined such that a person is 'eligible' only if they satisfy the inclusion logic and do not satisfy the exclusion logic.
    • Parsing Strategy: The library applies both IE classifiers and CFG parsers. Results are typically kept only if their confidence scores exceed a preset threshold to handle composite criteria.

  4. How the IE architecture extracts nominal requirements

    main

    Nominal requirements (e.g., diseases, medications) are extracted using Information Extraction (IE) components:

    1. Named Entity Recognition (NER): Uses a BiLSTMSlotAttention model to extract medical terms into 13 specific labels (e.g., age, allergy, cancer, treatment, pregnancy). It also extracts lower and upper bounds for scalar variables.
    2. Named Entity Linking (NEL): Normalizes extracted terms and links them to a medical ontology. This process:
      • Removes duplicate and synonym entities by grounding them to the same concept.
      • Provides a hierarchy for filtering (e.g., preferring breast cancer over a general cancer requirement).
      • Links entities to industry-standard IDs for interoperability.
      • Uses embedding space clustering to improve recall for terms that vary in phrasing.
    3. Ontology: The system uses the MeSH (Medical Subject Headings) vocabulary, augmented with manually curated concepts for specific domains like technology access, ethnicity, and language fluency to improve precision in clinical trial contexts.

  5. How the CFG architecture extracts ordinal and numerical requirements

    main

    Ordinal and numerical requirements are extracted using a three-stage Context-Free Grammar (CFG) pipeline:

    1. Lexer: Splits the criterion text into a sequence of tokens. Tokens include types such as variable, unit, comparison, negation, and end-of-string. It also handles unknown types for syntactically imperfect text.
    2. Parser: Uses the tokens as terminal nodes to build parse trees based on grammar production rules. It utilizes a modified Cocke–Younger–Kasami (CYK) algorithm that supports binary normal form for efficiency and performs 'best-effort' parsing so the root does not strictly need to be the start symbol.
    3. Interpreter: Analyzes the resulting parse trees by removing duplicates and sub-trees, then evaluates the remaining trees into machine-readable structures called relations.

  6. Understand MeSH Augmentation for Medical Concepts

    main
    To improve the accuracy of eligibility criteria parsing, the project augments the Medical Subject Headings (MeSH) vocabulary. It creates new vocabulary concepts for simple domains and adds synonyms to existing MeSH concepts, allowing the parser to better recognize diverse medical terminology.

  7. Identify Sample Input and Output Data

    main

    For testing the parser or understanding the data format, the data directory contains sample files:

    • Input: input/clinical_trials.csv contains a sample of 20 recent clinical trials (including COVID-19 trials).
    • Output: The output/ directory contains the resulting parsed eligibility criteria for those specific trials.
    # Input file location
input/clinical_trials.csv

# Output directory
output/

  8. The Clinical Trial Parser processing pipeline

    main

    The parser follows a structured pipeline to transform raw eligibility text into structured requirements:

    1. Preprocessing

    • Ingestion: Assumes text is ingested from ClinicalTrials.gov.
    • Labeling: Text lines are labeled as either inclusive or exclusive.
    • Normalization: Requirements are normalized by masking numbers and punctuation to prepare text for the NER model.

    2. Extraction

    • IE Path: For nominal variables (NER/NEL).
    • CFG Path: For ordinal and numerical variables (Lexer/Parser/Interpreter).

    3. Post Processing

    • Filtering: Results are filtered based on CFG and NER confidence levels.
    • Trial-level Logic: Applies rules to increase quality, such as:
      • Resolving contradictions (e.g., a trial requiring birth control consent should not also require pregnancy).
      • Specificity preference (e.g., removing a general cancer requirement if a more specific brain cancer requirement exists).

  9. Install and run the CFG Parser

    main

    The CFG (Clinical Feature Grammar) parser translates clinical trial eligibility criteria. To use it, follow these steps:

    1. Install Go: Download from https://golang.org/dl/.
    2. Setup GOPATH: Ensure the cloned project is located at $GOPATH/src/github.com/facebookresearch/Clinical-Trial-Parser.
    3. Execute Parser: Run the provided shell script from the project root:
      ./script/cfg_parse.sh
    4. View Output: The parsed relations are written to data/output/cfg_parsed_clinical_trials.tsv.

    Configuration: You can modify program parameters in two ways:

    • Edit the command line arguments in script/cfg_parse.sh.
    • Edit the configuration parameters in src/resources/config/cfg.conf.

    Integration: For custom applications, you should write your own driver module based on the example in src/cmd/cfg/main.go.

    ./script/cfg_parse.sh

  10. Use the Caffe2 medical NER model binary

    main
    The bin directory contains the pre-trained Caffe2 binary ner.c2 used for medical Named Entity Recognition (NER). This model is designed to identify various medical entities within clinical text. The model was trained on the processed annotated data available in the repository.
    ner.c2

  11. Improve CFG Parser quality

    main

    To fix parsing errors or add support for new ordinal and numerical criteria in the CFG parser, you can modify the following components:

    • Grammar Rules: Update the production rules in src/ct/parser/production/criterion.go to handle new criteria situations. It is recommended to add corresponding test cases in src/ct/parser/interpreter_test.go.
    • Variables: Add or update variables in src/resources/variables/variables.csv.
    • Units: Add or update units in src/resources/units/units.csv.

  12. Install and meet requirements for Clinical Trial Parser

    main

    The library is designed for Mac OS X or Linux. Requirements vary by module:

    CFG Parser Module

    IE Parser Module

    Word Embedding Module

    Data Acquisition (AACT Database)

    • To use script/aact.sh to download trials, you must have postgreSQL (version 11 or higher) installed.