DOCSEARCHbyAlgolia
Submit Docs

Meditron Documentation

repository·main·Indexed 24 days ago

https://github.com/epfllm/meditron

A suite of open-source medical Large Language Models (7B and 70B) adapted from Llama-2. Specialized for the medical domain through continued pretraining on clinical guidelines, PubMed papers, and abstracts. Includes guides for deployment via FastChat, OpenAI-compatible APIs, and Gradio web interfaces, as well as pipelines for inference, evaluation, supervised fine-tuning (SFT), and GAP-Replay corpus preprocessing.

Tokens
7K
Snippets
18
Records
37
Agent score
77%

What's inside Meditron

  1. Meditron Model Details and Advisory

    main

    Model Overview

    • Type: Causal decoder-only transformer language model.
    • Language: English (primarily).
    • Context Length: 4k tokens.
    • Input/Output: Text only.
    • License: Llama-2 Community License Agreement (Model) / Apache 2.0 (Code).
    • Knowledge Cutoff: August 2023.

    ⚠️ Advisory Notice

    Meditron is designed to encode medical knowledge from high-quality evidence but is not yet adapted to deliver this knowledge appropriately, safely, or within professional actionable constraints.

    Do not use Meditron in medical applications without:

    • Extensive use-case alignment.
    • Additional testing (including randomized controlled trials in real-world settings).

  2. Run the Supervised Fine-tuning (SFT) pipeline

    main

    To start the fine-tuning process, execute the sft.py script. Before running, you must ensure that the CHECKPOINTS constant within sft.py is updated with the correct paths for your base models. You may also edit the N_DOCS dictionary in the script to specify the number of documents in your dataset to optimize runtime.

    Important Notes:

    • Resuming Training: If you run the pipeline twice with the same configuration, the script will automatically resume from the latest saved checkpoint. If the fine-tuning has already completed, the script will crash.
    • Data Loading: The --data and --val arguments are primarily used during the first run with a new dataset. On subsequent runs, the script prioritizes loading pre-tokenized data from the directory specified by --tokenized_data_dir/{run_name}.
    python finetuning/sft.py --checkpoint=meditron --size=7 --run_name=pubmedqa

  4. Download the Guidelines corpus

    main

    There are two ways to access the Guidelines corpus:

    1. Open-access subset: Download 36K open-access articles from the HuggingFace datasets hub using the datasets library.
    2. Full corpus: Scrape and clean all 46K guidelines (including closed-access sources) by running the download script in the guidelines folder.

    To use the HuggingFace subset:

    from datasets import load_dataset

dataset = load_dataset("epfl-llm/guidelines")

    To scrape the full corpus:

    ./download.sh

  5. Convert Megatron-LLM checkpoints to Huggingface format

    main

    If you are using checkpoints from Megatron-LLM, they must be converted to Huggingface format before they can be used in the inference and evaluation pipeline. Use the provided megatron2hf.sh script.

    Required Environment Variables

    • NUM_IN_SHARDS: Number of input model shards.
    • NUM_OUT_SHARDS: Number of output model shards.
    • INPUT_DIR: Path to your Megatron checkpoint.
    • OUTPUT_DIR: Path where you want to save the HF model weights.
    • UNSHARDED_DIR: Temporary path for storing the unsharded Megatron checkpoint.
    # Example execution
NUM_IN_SHARDS=8
NUM_OUT_SHARDS=8
INPUT_DIR=<path to your Megatron checkpoint>
OUTPUT_DIR=<path to save your HF model weights>
UNSHARDED_DIR=<path for stroing unsharded Megatron checkpoint, temporary>

./megatron2hf.sh

  7. Download and downsample Replay and Code data

    main

    Use the scripts in this directory to download data from RedPajama (Replay data) and StarCoder (Code data), estimate their token counts, and downsample them to a specific size using the --keep fraction flag.

    Prerequisites

    1. Run huggingface-cli login and provide your Hugging Face access token.
    2. For StarCoder data, you must accept the Terms of Service at: https://huggingface.co/datasets/bigcode/starcoderdata

    Usage

    Replay Data

    To download and downsample Replay data, use replay.py. Use the --keep flag to specify the fraction of the dataset to retain and --out to specify the output path.

    python replay.py --keep=0.0004 --out=../data/replay.jsonl

    Code Data

    To download and downsample StarCoder data, use starcoder.py. Use the --keep flag to specify the fraction of the dataset to retain and --out to specify the output path.

    python starcoder.py --keep=0.1 --out=../data/starcoder.jsonl

    Arguments

    • --keep: The fraction of the dataset to keep (e.g., 0.0004 for 0.04%).
    • --out: The output file path.
    • --seed: (Optional) Set a seed value for deterministic generation.
    • --streaming: (Optional) Use streaming to avoid downloading the entire dataset. This avoids storing unneeded data but slows down the generation process.
    python replay.py --keep=0.0004 --out=../data/replay.jsonl

  9. Requirements for Medical Benchmark Inference & Evaluation

    main

    To run inference and evaluation against medical benchmarks, ensure the following Python package versions are installed:

    • vllm >= 0.2.1
    • transformers >= 4.34.0
    • datasets >= 2.14.6
    • torch >= 2.0.1

    For detailed instructions on running these evaluations, refer to the ./evaluation/README.md file.

  10. Scrape all 46K Clinical Practice Guidelines

    main

    To scrape and pre-process all 16 sources of clinical guidelines (including those not released on HuggingFace), you must first install the necessary dependencies, the spaCy English pipeline, and the scipdf parser for PDF conversion. After setup, run the guidelines download script.

    # Install dependencies
pip install -r guidelines/requirements.txt

# Use spacy to get the English language pipeline
python -m spacy download en_core_web_sm 

# Install scipdf from GitHub to convert PDFs to text
pip install git+https://github.com/titipata/scipdf_parser

# Run the download and pre-processing
./guidelines/download.sh

  11. Serve Meditron with a Web GUI

    main

    To serve Meditron via a web interface, you must launch three components in order: the controller, the model worker(s), and the UI web server.

    1. Launch the controller: Manages distributed workers.

      python3 -m fastchat.serve.controller

    2. Launch the model worker(s): Hosts the model. For example, to run meditron-70b across 8 GPUs:

      ./fastchat.sh -c meditron-70b -m single -n 8

      Note: Wait until you see "Uvicorn running on ..." before proceeding.

    3. Launch the Gradio web server: Provides the user interface.

      python3 -m fastchat.serve.gradio_web_server

    If models do not appear in the UI, try restarting the Gradio web server.

  12. Use BetterChatGPT as a Third-Party UI

    main

    To use the modified BetterChatGPT UI with your Meditron deployment:

    1. Install and setup the UI:

      git clone https://github.com/eric11eca/BetterChatGPT
cd BetterChatGPT
git checkout meditron-ui
npm install
npm run dev

    2. Configure the API endpoint: In the UI settings, update the API hostname to point to your FastChat OpenAI server: http://localhost:8000/v1/chat/completions

    3. Select the model: Select the appropriate Meditron model from the dropdown to begin interaction.

    git clone https://github.com/eric11eca/BetterChatGPT
cd BetterChatGPT
git checkout meditron-ui
npm install
npm run dev