DataQualityDashboard (DQD)

DataQualityDashboard uses a systematic approach based on the Kahn Framework, which organizes checks into categories and contexts.

Instead of individual checks, the tool uses "data quality check types". These are parameterized templates where OMOP tables, fields, and concepts are substituted.

Checks are categorized into three levels:

1. Table-level checks: Evaluate tables at a high level (e.g., checking if required tables are present).
2. Field-level checks: Evaluate specific fields within a table (e.g., primary key relationships or domain conformity). This is the most common type.
3. Concept-level checks: Related to individual concepts (e.g., checking for gender-specific concepts in the wrong gender).

The R package runs these checks against an OMOP CDM instance, evaluates them against configurable thresholds, and produces a JSON object. This JSON object is then read into an RShiny application for visualization.

When contributing changes to DataQualityDashboard, follow this workflow:

1. Branching: Sync your fork's develop branch with upstream. Create a new branch for your specific issue or feature. For large features, split work into multiple sub-branches.
2. Testing Requirements:
   • New Functionality: You must add unit tests covering the new code.
   • Bug Fixes: You must add a unit test that reproduces the regression.
3. Validation Steps:
   • Run R CMD Check and resolve all errors, warnings, and notes (the package size NOTE can be ignored).
   • Run the specific test file: test_file(path = "tests/testthat/test-executeDqChecks.R") to resolve snapshot test failures.
   • Build and install the package locally, then run DQD against your local Postgres database to verify results.
4. Submission:
   • Push changes to GitHub and open a PR targeting the develop branch of the parent OHDSI/DataQualityDashboard repository.
   • Wait for automated checks. If they fail, address errors by repeating the validation steps.
   • Once approved by a maintainer, you may merge into develop.

# Run specific tests to resolve snapshot failures
test_file(path = "tests/testthat/test-executeDqChecks.R")

Prior to executing data quality checks, the CDM_SOURCE table must be properly populated. This table provides metadata about the CDM instance used for the checks. The following fields are expected:

The <dq-metadata> custom element is a web component used within the Data Quality Dashboard UI to render a structured view of data source metadata. It expects a JSON string passed to the data-results attribute.

When the data-results attribute is updated, the component automatically re-renders using a Handlebars template.

Expected JSON Schema

The data-results attribute must contain a JSON object with the following keys:

• cdmSourceName: Name of the CDM source
• sourceDescription: Description of the source
• cdmHolder: Entity licensed to hold the data
• sourceReleaseDate: Release date of the source
• cdmReleaseDate: Release date of the CDM
• cdmVersion: Version of the CDM
• vocabularyVersion: Version of the vocabulary used
• sourceDocumentationReference: Reference to source documentation
• cdmEtlReference: Reference to the ETL process

<!-- Example usage in HTML -->
<dq-metadata data-results='{"cdmSourceName": "OMOP CDM", "sourceDescription": "Example Source", "cdmHolder": "Example Org", "sourceReleaseDate": "2023-01-01", "cdmReleaseDate": "2023-01-01", "cdmVersion": "5.4", "vocabularyVersion": "2023-01-01", "sourceDocumentationReference": "http://example.com", "cdmEtlReference": "http://etl.example.com"}'></dq-metadata>

The data-results attribute expects a JSON array of objects. Each object represents a single data quality check result. The component uses the following properties to aggregate statistics:

Note on Pass/Fail Logic:

• A check is considered Passed if it has passed == 1 OR notApplicable == 1. If those properties are missing, it is considered passed if failed == 0.
• A check is considered Failed if it has failed == 1 OR isError == 1. If those properties are missing, it is considered failed if failed == 1.

[
  {
    "context": "Verification",
    "category": "Plausibility",
    "passed": 1
  },
  {
    "context": "Validation",
    "category": "Conformance",
    "failed": 1,
    "isError": 1
  }
]