OntoGPT

repository·main·Indexed 21 days ago

https://github.com/monarch-initiative/ontogpt

A Python framework for extracting structured information from unstructured text using Large Language Models (LLMs) and ontology-based grounding. It utilizes LinkML for schema definition and LiteLLM for interfacing with various providers including OpenAI, Anthropic, and local models via Ollama. The package includes tools for custom schema design, OWL annotation integration, and a minimal web application for running extractions.

Tokens
30.6K
Snippets
131
Records
163
Agent score
75%

What's inside ontogpt

  1. What is SPIRES?

    main

    SPIRES (Structured Prompt Interrogation and Recursive Extraction of Semantics) is the primary extraction method in OntoGPT. It is a Zero-shot learning (ZSL) approach designed to extract nested semantic structures from text.

    Inputs:

    1. A LinkML schema
    2. Free text

    Outputs: Knowledge in a structure conformant with the supplied schema in JSON, YAML, RDF, or OWL formats.

    Supported Models:

    • OpenAI GPT models
    • Various LiteLLM-supported hosted providers
    • Local models via Ollama
  2. Schema template for BC5CDR (Chemical to Disease)

    main

    The BC5CDR evaluation uses a specific schema template designed to represent associations between chemicals and diseases.

    Key Components:

    • ChemicalToDiseaseDocument: A document containing ChemicalToDiseaseRelationship triples.
    • ChemicalToDiseaseRelationship: A triple consisting of a subject (Chemical), object (Disease), and predicate (ChemicalToDiseasePredicate, e.g., INDUCES or TREATS).
    • Chemical: A NamedEntity with MESH ID prefixes.
    • Disease: A NamedEntity with MESH ID prefixes.
    • ChemicalToDiseasePredicate: An enum where, for the purposes of BC5CDR evaluation, any predicate other than INDUCES is ignored.

    This template is used to evaluate Semantic Llama against the BC5CDR task.

    id: http://w3id.org/ontogpt/ctd
    name: ctd
    title: Chemical to Disease Template
    description: >-
      A template for Chemical to Disease associations.
    ...
    classes:
      ChemicalToDiseaseDocument:
        is_a: TextWithTriples
        slot_usage:
          triples:
            range: ChemicalToDiseaseRelationship
      ChemicalToDiseaseRelationship:
        is_a: Triple
        slot_usage:
          subject:
            range: Chemical
          object:
            range: Disease
          predicate:
            range: ChemicalToDiseasePredicate
    ...
  3. Incorporate OWL annotations into custom LinkML schemas

    main

    To support complex logic or interoperability with ontology editors like Protege, you can include OWL (Web Ontology Language) annotations within your LinkML schema. This allows you to define how schema components relate to OWL syntax, such as mapping attributes to AnnotationProperty or ObjectProperty.

    Key features for OWL integration include:

    • slot_uri: Defines the URI for the slot in existing vocabularies (e.g., rdfs:label, dcterms:description).
    • annotations: Uses the owl key to specify OWL axiom types (e.g., owl: Class, owl: ObjectProperty).
    • owl.template: Uses Jinja-style templating to define complex OWL axioms, such as EquivalentClasses. You can use the tr() function (e.g., {{tr(step)}}) to translate inputs into valid OWL entities.
    • close_mappings: Indicates that a class is similar to, but not necessarily identical to, an existing ontology term.

    Important: When using a schema with OWL annotations, you must export the results using the -O owl option to preserve this functionality.

    classes:
      Recipe:
        tree_root: true
        close_mappings:
          - FOODON:00004081
        attributes:
          label:
            description: the name of the recipe
            slot_uri: rdfs:label
            annotations:
              owl: AnnotationProperty, AnnotationAssertion
          ingredients:
            description: a semicolon separated list of the ingredients
            multivalued: true
            range: Ingredient
            slot_uri: FOODON:00002420
            annotations:
              owl: ObjectProperty, ObjectSomeValuesFrom
        annotations:
          owl: Class
          owl.template: |
            EquivalentClasses(
              {{url}}
              ObjectIntersectionOf(
                recipe:Recipe
                {% for step in steps %}
                ObjectSomeValuesFrom(recipe:steps {{tr(step)}})
                {% endfor %}
              )
            )
  4. Configure LLM providers and environment variables

    main

    OntoGPT uses LiteLLM to interface with LLM endpoints. While runoak is the preferred method for managing credentials within the OntoGPT ecosystem, you can also use standard LiteLLM environment variables directly.

    Supported environment variables include:

    • OPENAI_API_KEY
    • ANTHROPIC_API_KEY
    • MISTRAL_API_KEY
    • GROQ_API_KEY
    • AZURE_API_KEY
    • AZURE_API_BASE
    • AZURE_API_VERSION

    If the provider is not automatically detected from the model name, use the --model-provider option in your OntoGPT command to specify it explicitly.

  5. Use Enums to restrict identifier sets

    main

    Enums allow schemas to work with specific subsets of identifiers. You can use the reachable_from slot to define dynamic enums based on existing ontologies.

    Example:

    enums:
      GOCellComponentType:
        reachable_from:
          source_ontology: obo:go
          source_nodes:
            - GO:0005575 ## cellular_component
    
      CellType:
        reachable_from:
          source_ontology: obo:cl
          source_nodes:
            - CL:0000000 ## cell

    You can then use these in a class via slot_usage to restrict the id slot:

      GeneLocation:
        is_a: NamedEntity
        slot_usage:
          id:
            values_from:
              - GOCellComponentType
              - CellType
  6. Use ungrounded templates for extraction

    main

    If you do not have a Bioportal API key or do not wish to perform ontology grounding, you can use templates suffixed with _ungrounded. These templates will extract terms and relations from the text but will not attempt to map them to specific ontology IDs.

    For example, instead of using environmental_sample, use environmental_sample_ungrounded.

  7. Install OntoGPT and set up API keys

    main

    To use OntoGPT, install the package with web support and configure your required API keys using the runoak command. You will typically need an OpenAI API key for the LLM and potentially a Bioportal API key if using templates that require Bioportal ontologies.

    Installation

    %pip install ontogpt[web]

    Configure OpenAI API Key

    openai_api_key = "<your openai api key here>"
    !runoak set-apikey -e openai {openai_api_key}

    Configure Bioportal API Key (Required for certain templates)

    bioportal_api_key = "<your bioportal api key here>"
    !runoak set-apikey -e bioportal {bioportal_api_key}
  8. Verify OntoGPT installation with text completion

    main

    To ensure OntoGPT is correctly set up, you can perform a simple text completion test. Create a text file (e.g., example.txt) with a prompt, then use the complete command.

    1. Create example.txt:
    Why did the squid cross the coral reef?
    1. Run the completion command:
    ontogpt complete example.txt
  9. Install OntoGPT via pip

    main

    To install the core OntoGPT package in your workspace, use pip install ontogpt.

    Depending on your use case, you may need to install optional dependency groups using the [extra_name] syntax:

    • dev: Dependencies for testing.
    • docs: Dependencies for building documentation.
    • recipes: Dependencies for recipe scraping and parsing.
    • web: Dependencies for the web application.
    pip install ontogpt
    
    # Example: Install with web application dependencies
    pip install ontogpt[web]
  10. Install OntoGPT from GitHub source

    main

    If you are contributing to development or want to run from the repository, clone the repo and use uv to manage dependencies. Note that all commands must be preceded by uv run.

    1. Clone the repository.
    2. Install using uv pip install ..
    3. To install with extras from source, use uv pip install -e .[extra_name].
    git clone https://github.com/monarch-initiative/ontogpt.git
    cd ontogpt/
    uv pip install .
    
    # To install with extras from source
    uv pip install -e .[dev]
  11. Transform an OWL ontology into an OntoGPT schema

    main

    If you want to use an OWL ontology as a structural schema (to extract concepts and their relationships rather than grounding to individuals), you can transform it into a LinkML YAML schema using the schema-automator tool.

    Workflow

    1. Convert to Functional Syntax (if needed): If schemauto fails, use the robot tool to convert your OWL file to .ofn (functional syntax) format.
    2. Import with schema-automator: Use schemauto import-owl to generate the LinkML YAML.
    3. Refine: The resulting YAML will require manual addition of description and annotation slots to each class to be effective for extraction operations.

    Warning: This process may fail if the input ontology relies heavily on external imports.

    # Step 1: Convert to functional syntax using robot
    robot convert -i fruit.owl -o fruit.ofn
    
    # Step 2: Import using schemauto
    schemauto import-owl fruit.ofn
  12. Extract knowledge using SPIRES

    main

    OntoGPT uses the SPIRES mechanism to extract structured information from text based on a provided data model (schema).

    Working Mechanism

    1. Data Model: You provide a schema (e.g., LinkML) describing the desired output structure.
    2. Grounding: You provide preferred annotations for grounding NamedEntity fields.
    3. Process: OntoGPT generates a prompt, queries a language model, parses the results into a dictionary, and grounds the results using an annotator (like an ontology).

    Extraction Command

    Use the extract command with the following flags:

    • -t / --template: The base name of a predefined LinkML schema available in OntoGPT, or a path to a custom .yaml schema file.
    • -i / --input: Path to the input text file.
    • -o / --output: (Optional) Path to redirect the output to a file.

    To see all available templates, use ontogpt list-templates.

    ontogpt extract -t gocam.GoCamAnnotations -i ~/path/to/abstract.txt