DOCSEARCHbyAlgolia
Submit Docs

PyWhy-LLM

repository·main·Indexed 43 days ago

https://github.com/py-why/pywhyllm

A library that integrates Large Language Models (LLMs) into the causal inference pipeline to augment human expertise. It provides tools for suggesting causal models, identifying confounders, finding identification sets (backdoor, mediators, IVs), and validating causal graphs. PyWhy-LLM integrates with existing PyWhy libraries such as DoWhy and causallearn and includes specialized components like AugmentedModelSuggester for RAG-based modeling via CauseNet.

Tokens
6.2K
Snippets
30
Records
32
Agent score
54%

What's inside PyWhy-LLM

  1. Explore the pywhyllm.suggesters package submodules

    main

    The pywhyllm.suggesters package provides various modules for leveraging LLMs to suggest components for causal analysis. The package is organized into specialized submodules for identification, modeling, and validation tasks, including 'simple' versions of the identification and modeling suggesters.

    Available submodules:

    • identification_suggester: For suggesting variables or relationships related to causal identification.
    • model_suggester: For suggesting causal model structures.
    • simple_identification_suggester: A simplified implementation for identification tasks.
    • simple_model_suggester: A simplified implementation for modeling tasks.
    • validation_suggester: For validating causal hypotheses or models.

  3. Configure ModelType and RelationshipStrategy

    main

    PyWhy-LLM uses ModelType to define the type of LLM used (defaulting to completions models) and RelationshipStrategy to define the nature of the request made to the LLM.

    Available strategies:

    • RelationshipStrategy.Parent
    • RelationshipStrategy.Child
    • RelationshipStrategy.Pairwise
    from pywhyllm import ModelType, RelationshipStrategy

model_type = ModelType.Completion
relationship_strategy = RelationshipStrategy.Parent

  4. How Strategy works in TuebingenModelSuggester

    main

    The Strategy enum defines the prompting approach used by the TuebingenModelSuggester to evaluate causal links. In the provided examples, Strategy.Straight is used to perform direct relationship assessment between variable pairs.

    from pywhyllm.suggesters.tuebingen_model_suggester import Strategy

# Using the Straight strategy
modeler.suggest_relationship(..., strategy=Strategy.Straight)

  6. Set up the environment for PyWhy-LLM

    main

    To use PyWhy-LLM, you need to install python-dotenv to manage environment variables and configure your LLM API key (e.g., OpenAI).

    !pip install python-dotenv
    from dotenv import load_dotenv
import os

load_dotenv()
os.environ["OPENAI_API_KEY"] = 'your-key-here'

  9. Use AugmentedModelSuggester for RAG-based causal suggestions

    main

    The AugmentedModelSuggester class uses Retrieval Augmented Generation (RAG) to improve causal analysis. It searches the CauseNet dataset (a large-scale knowledge base of causal relations) for relevant causal pairs.

    If a relevant pair is found in CauseNet, the LLM is augmented with that specific evidence. If no match is found, the LLM falls back to its internal knowledge.

    To use it, instantiate the class with your desired model name (e.g., 'gpt-4') and call suggest_pairwise_relationship(var1, var2).

    from pywhyllm.suggesters.augmented_model_suggester import AugmentedModelSuggester

# Initialize the model
model = AugmentedModelSuggester('gpt-4')

# Suggest a relationship between two variables
result = model.suggest_pairwise_relationship("smoking", "lung cancer")
print(result)

  10. Use ModelSuggester for causal model construction

    main

    The ModelSuggester class allows you to use an LLM (e.g., 'gpt-4') to automate parts of the causal modeling process. You can use it to identify relevant domain expertise, suggest potential confounders, and propose relationships between variables to build a Directed Acyclic Graph (DAG).

    Key methods:

    • suggest_domain_expertises(all_factors): Returns a list of domain expertises relevant to the provided factors.
    • suggest_confounders(treatment, outcome, all_factors, domain_expertises): Suggests a set of potential confounders.
    • suggest_relationships(treatment, outcome, all_factors, domain_expertises, strategy): Suggests pair-wise relationships between variables using a specified RelationshipStrategy.
    from pywhyllm.suggesters.model_suggester import ModelSuggester
from pywhyllm import RelationshipStrategy

modeler = ModelSuggester('gpt-4')

all_factors = ["smoking", "lung cancer", "exercise habits", "air pollution exposure"]
treatment = "smoking"
outcome = "lung cancer"

# Suggest a list of domain expertises
domain_expertises = modeler.suggest_domain_expertises(all_factors)

# Suggest a set of potential confounders
suggested_confounders = modeler.suggest_confounders(treatment, outcome, all_factors, domain_expertises)

# Suggest pair-wise relationships between variables
suggested_dag = modeler.suggest_relationships(treatment, outcome, all_factors, domain_expertises, RelationshipStrategy.Pairwise)

  11. Use IdentificationSuggester to find causal sets

    main

    The IdentificationSuggester helps identify the sets of variables required for causal identification (e.g., for backdoor or instrumental variable approaches).

    Key methods:

    • suggest_backdoor(treatment, outcome, all_factors, domain_expertises): Suggests a backdoor set.
    • suggest_mediators(treatment, outcome, all_factors, domain_expertises): Suggests a mediator set.
    • suggest_ivs(treatment, outcome, all_factors, domain_expertises): Suggests an instrumental variable (IV) set.
    from pywhyllm.suggesters.identification_suggester import IdentificationSuggester

identifier = IdentificationSuggester('gpt-4')

# Suggest a backdoor set, mediator set, and iv set
suggested_backdoor = identifier.suggest_backdoor(treatment, outcome, all_factors, domain_expertises)
suggested_mediators = identifier.suggest_mediators(treatment, outcome, all_factors, domain_expertises)
suggested_iv = identifier.suggest_ivs(treatment, outcome, all_factors, domain_expertises)

  12. Use AugmentedModelSuggester for RAG-based modeling

    main

    The AugmentedModelSuggester is a specialized modeler that utilizes Retrieval Augmented Generation (RAG) via CauseNet to enhance the LLM's causal reasoning. This is particularly useful for suggesting relationships between variables by grounding the LLM in external knowledge.

    Key method:

    • suggest_relationships(treatment, outcome): Suggests pair-wise relationships between two given variables using RAG.
    from pywhyllm.suggesters.augmented_model_suggester import AugmentedModelSuggester

# Create instance of Modeler
modeler = AugmentedModelSuggester('gpt-4')

treatment = "smoking"
outcome = "lung cancer"

# Suggest pair-wise relationship between two given variables, utilizing CauseNet for RAGing the LLM
suggested_relationship = modeler.suggest_relationships(treatment, outcome)