DOCSEARCHbyAlgolia
Submit Docs

OpenMed Documentation

repository·master·Indexed 26 days ago

https://github.com/maziyarpanahi/openmed

An on-device clinical AI platform for biomedical entity extraction and PII de-identification. OpenMed provides 2,000+ specialized models that run locally on iOS, Android, Python, and the browser. The ecosystem includes OpenMedKit for Android (v2.0.0) with pre-packaged de-identification policy profiles like hipaa_safe_harbor, as well as REST clients for Go and TypeScript to manage clinical text analysis, bulk jobs, and model orchestration.

Tokens
344K
Snippets
758
Records
1.4K
Agent score
40%

What's inside OpenMed

  1. Overview of OpenMed Capabilities

    master

    OpenMed is a platform for clinical natural language processing (NLP) that integrates biomedical models, de-identification, and multimodal inputs. Key features include:

    • Model Registry: Discoverable Hugging Face models with metadata (domain, size, device recommendations).
    • PII Detection & De-identification: HIPAA-compliant entity merging, policy profiles, and audit reports.
    • Multimodal & Structured Input: Support for OCR, Images, PDF, DOCX, DICOM, HL7 v2, CDA/C-CDA, FHIR (Operations and Bulk NDJSON), and more.
    • Cross-Platform Support:
      • Python: Inference via MLX (Apple Silicon) or CPU.
      • Mobile: Swift (OpenMedKit) for iOS and Kotlin/ONNX Runtime for Android.
      • Web: ONNX/WebGPU via Transformers.js exports and React Native bridges.
      • Services: REST (FastAPI) and gRPC endpoints.
    • Deployment & Evidence: Includes leak heatmaps, model scorecards, SBOM, and SLSA provenance for production-grade release evidence.

  2. Understand the `ng_ndpa` de-identification profile

    master

    The ng_ndpa profile is a strict, irreversible de-identification posture designed to align with the technical defaults suggested by the Nigeria Data Protection Act 2023 (NDPA).

    Key characteristics include:

    • Irreversibility: It uses replace (synthetic replacement) or mask (typed placeholder) actions. It does not retain replacement mappings.
    • Safety Sweep: It requires a mandatory safety sweep.
    • Non-keep Policy: It assigns a non-keep action to every canonical label.
    • Local-First Processing: When run entirely on-device without sending data to external recipients, the de-identification operation itself is not considered a cross-border transfer under NDPA Sections 41-42.

  3. Understand OpenMed security scanning and blockers

    master

    OpenMed enforces security standards through several automated processes:

    1. Dependency Scanning: Vulnerable Python dependencies and GitHub Actions are treated as release blockers if a fixed version is available. The CI uses scripts/security/pip_audit_gate.py (wrapping pip-audit) to detect these.
    2. Dependabot: Performs weekly checks on Python packages and GitHub Actions. Updates are grouped into single pull requests for manual review and testing.
    3. Static Analysis: Bandit runs in CI to detect security issues in Python code. The CI job is configured to block on high-severity findings to prevent new critical issues from being introduced.

  4. Use bundled Chinese administrative division data

    master

    The zh_cn_administrative_divisions.json file provides a compilation of factual province, city, and district names for China. This dataset is intended to support shape-correct Chinese surrogates within OpenMed applications.

    Important Limitations:

    • No Geocoding: This is not a national gazetteer and must not be used for geocoding purposes.
    • Data Scope: It contains administrative names only. It does not include patient records, street names, building/room numbers, postcodes, or coordinates.
    • License: Distributed under the Apache-2.0 license.

  5. Understand the `ke_dpa` policy profile behavior

    master

    The ke_dpa policy profile in OpenMed is a conservative profile designed to map sensitive-personal-data classes from Kenya's Data Protection Act No. 24 of 2019 to specific de-identification actions.

    Key characteristics of the ke_dpa profile:

    • Masking Strategy: It masks direct identifiers, county-level and finer location data, sensitive attributes, and health-related concepts.
    • No Reversible Mapping: The profile retains no reversible mapping for masked data.
    • Safety Sweep: It always runs a deterministic safety sweep.
    • High-Recall Arbitration: Uses high-recall arbitration to ensure sensitive data is captured.

    Note: This profile is decision-support guidance and does not constitute legal advice or compliance certification.

  6. Navigate OpenMed Documentation

    master

    The OpenMed documentation is organized into several key sections to help you integrate and use the platform:

    Getting Started & Overview

    • Quick Start: The fastest way to access working environments and copy-paste scripts.
    • Feature Map: A guide showing how specific capabilities map to the underlying code.
    • Release Notes: Details on current installation coordinates, compatibility contracts, and verification requirements (e.g., v2.0.0).

    Core Guides

    • Analyze Text Assistant: For performing single-call inference.
    • REST Service (MVP): For setting up Docker-based HTTP endpoints.
    • PII Identification & Smart Merging: For HIPAA-aware de-identification.
    • Batch Processing: For processing multiple texts or files.
    • ModelLoader & Pipelines: For managing long-running jobs.
    • Model Registry: For selecting the correct model checkpoints.
    • Configuration Profiles: For switching between dev, prod, and test settings.
    • Advanced NER & Output Formatting: For refining spans and output structures.
    • Medical-Aware Tokenizer: For improved clinical token boundaries.
    • Configuration & Validation: For ensuring reproducible deployments.
    • Zero-shot Toolkit: For GLiNER workflows.
    • Performance Profiling: For measuring latency and optimizing performance.
    • Examples & Testing: For everyday usage patterns and quality assurance.

    Project Operations

    • Contributing & Releases: Processes for publishing releases and documentation.
    • Release Streams & Channels: Policies regarding model artifacts and library releases.
    • Generative Model Policy: Guidelines on approved and restricted model-assisted workflows.

  7. Understand the `za_popia` de-identification profile

    master

    The za_popia profile is an irreversible anonymisation posture designed for compliance with South Africa's Protection of Personal Information Act (POPIA).

    Key characteristics:

    • Irreversible: It performs a safety sweep and does not retain replacement mappings.
    • Non-retention: It does not keep any canonical identifier or special-personal-information class by default.
    • Actions: It uses two primary non-keep actions:
      • replace: An irreversible synthetic replacement.
      • mask: A typed placeholder.

    Important Disclaimer: This is a technical de-identification control and does not constitute legal advice. It does not evaluate consent, public-interest grounds, cross-border adequacy, or legal authorisations under POPIA sections 27 or 32.

  8. Handle Multimodal and Structured Inputs

    master

    OpenMed supports various document formats and structured data types:

    • Documents: OCR for scanned documents (Tesseract, PaddleOCR, EasyOCR, docTR), Markdown, AsciiDoc, EPUB, and DOCX.
    • Medical Imaging: DICOM header de-identification and burned-in pixel OCR redaction.
    • Structured Data: JSONL chat logs (with speaker pseudonymization), CSV/TSV PHI column classification, and vCard/iCalendar redaction.
    • Community Health Worker (CHW) Forms: Detection and de-identification for ODK, CommCare, and KoBoToolbox exports.

  9. Use the computing-ecqms skill to supplement eCQM capture

    master

    The computing-ecqms skill helps capture clinical facts from free-text notes that are often missed by structured EHR data. This is used to populate eCQM (electronic Clinical Quality Measure) components like Denominator Exclusions (e.g., "patient declined screening") and Numerators (e.g., "counseling provided").

    Important: OpenMed supplements the data capture process; it does not compute or certify the measure itself. You must feed the resulting QDM (Quality Data Model) data elements into a certified CQL (Clinical Quality Language) engine like cqframework.

  10. Use the querying-terminology-service skill

    master

    The querying-terminology-service skill acts as a thin client for interacting with FHIR terminology servers. Since OpenMed does not bundle restricted vocabularies (SNOMED CT, RxNorm, LOINC, ICD-10), you use this skill to validate, expand, or translate clinical codes using a server you already operate and license (e.g., Ontoserver, HAPI FHIR, or tx.fhir.org).

    When to use:

    • To turn an OpenMed entity span into a validated CodeableConcept.
    • To confirm if a code is valid in a specific system.
    • To expand a ValueSet for picklists or autocomplete.
    • To look up display names or properties for a code.
    • To map codes between different vocabularies (e.g., ICD-10 to SNOMED).

  11. Understand the Open Benchmark Leaderboard structure

    master

    The OpenMed leaderboard aggregates data from the canonical model manifest and joins it with the latest BenchmarkReport for specific model categories. The leaderboard is organized by the following dimensions:

    • System: The provider or system name (e.g., OpenMed, SHIELD).
    • Family: The model family or task type (e.g., General, NER, PII, Vision, ZeroShot).
    • Tier: The model size/scale (e.g., Tiny, Small, Base, Medium, Large, XLarge).
    • Format: The model runtime format (e.g., mlx-fp, mlx-4bit, mlx-8bit, pytorch).
    • Models: The count of models available in that specific configuration.
    • Current Leakage: The measured data leakage percentage.
    • Last Green Release: The date of the last successful benchmark release.
    • Harness Freshness: The timestamp indicating how recent the benchmark harness is.
    • Evidence: The type of evidence used (e.g., golden).

  12. Understand the OpenMed Redactor Threat Model

    master

    The OpenMed 'redactor' is the de-identification path responsible for preventing the exposure of Protected Health Information (PHI) or personal data. A security defect is defined as a redaction bypass, where an identifier survives the de-identification process.

    Scope of the Redactor:

    • Detection, normalization, arbitration, redaction, surrogate/date-shift, and auditing.
    • Out of Scope: Hosted endpoints and third-party models.

    Security Principles:

    • Local-first: De-identification runs in-process on the operator's machine. After model download, no network calls are required by default. Offline mode can be used to hard-block egress.
    • No Raw PHI Logging: The system is designed to ensure no raw PHI, tokens, keys, or redacted-to-original mappings are stored in logs, caches, or audit artifacts.