Ai2 Scholar QA

The Scholar QA pipeline is designed to be modular. You can extend or replace components by implementing the following abstract base classes:

Retrieval Customization

• AbstractRetriever: For custom paper retrieval logic.
• AbstractReranker: For custom reranking logic.
• AbsPaperFinder: For custom paper finding/aggregation logic.

Generation Customization

• MultiStepQAPipeline: Can be extended or modified to change how the multi-step generation (quote extraction, planning, summary) occurs.

Application Customization

To extend the web app functionality, you can use a fastapi.APIRouter to add new endpoints to the existing application. When doing this, ensure you reimplement lazy_load_scholarqa in your new API script to ensure the correct class is initialized.

The ai2-scholar-qa-reranker.py script demonstrates how to structure a Modal application using up to three distinct container types:

1. The Model class: This container is always present and runs on GPUs to perform the actual model inference.
2. A deployed function endpoint: A container that exposes an API endpoint which can be called directly from client code using the modal Python package.

The script handles the full lifecycle, including downloading the model from HuggingFace and creating the basic Modal image.

During the edit process, the pipeline generates an EditClusterPlan which assigns one of the following actions to each section of the report. These actions guide how the generate_iterative_summary_edit step modifies the content:

The mentioned_papers parameter accepts a list of corpus_ids. This allows you to explicitly direct the LLM to include specific research in the edit.

Current Behavior Note: The pipeline currently fetches basic metadata (title, abstract, authors, year, citations) for these papers. When processing mentioned_papers, the system uses the paper's abstract as the text source for quote extraction, whereas searched papers use rich snippets.

Usage Patterns:

• Direct Addition: Provide IDs to add specific papers to the report.
• Contextual Addition: Provide IDs along with an instruction like "add this paper to section 2".
• Similarity Search: Use an instruction like "add papers similar to [ID]" to leverage the search mechanism alongside the mentioned IDs.

You can interact with the Modal scripts using the Modal CLI:

• Testing (Ephemeral): Use modal run <script>.py to run your code on Modal as an ephemeral app. This is useful for testing without permanent deployment.
• Deployment (Permanent): Use modal deploy <script>.py to deploy your API as a persistent service.

# Run as an ephemeral app for testing
modal run <script>.py

# Deploy the API permanently
modal deploy <script>.py

Ai2 Scholar QA requires API keys for Semantic Scholar and LLM providers. Create a .env file in your root directory or set these in your runtime environment:

• S2_API_KEY: Required for retrieving paper passages, keyword search results, and metadata via the Semantic Scholar public API.
• ANTHROPIC_API_KEY: Used for the primary LLM (Claude 3.7 Sonnet). Any model served by litellm can be used.
• OPENAI_API_KEY: Used as the fallback LLM (GPT-4o) and for the OpenAI text moderation API to filter harmful queries.
• MODAL_TOKEN & MODAL_TOKEN_SECRET: Required if using Modal to serve models.
• LANGCHAIN_* variables: Use LANGCHAIN_API_KEY, LANGCHAIN_TRACING_V2, LANGCHAIN_ENDPOINT, and LANGCHAIN_PROJECT to enable Langsmith-based log traces.

export S2_API_KEY=
export ANTHROPIC_API_KEY=
export OPENAI_API_KEY=

To use the provided scripts with Modal, you must first create an account at modal.com and install the modal Python package. The installation process will generate a .modal.toml file in your home directory containing your token_id and token_secret. Ensure the active flag is set to true in the configuration file.

[user]
token_id = "ak-*[...]*"
token_secret = "as-*[...]*"

active = true

You can install the library via pip. To include support for sentence transformer models as a re-ranker, use the [all] extra.

# Standard installation
pip install ai2-scholar-qa

# Installation with sentence transformer support
pip install 'ai2-scholar-qa[all]'

pip install ai2-scholar-qa

#to use sentence transformer models as re-ranker
pip install 'ai2-scholar-qa[all]'

To start the Scholar QA application for local development, use Docker Compose. This launches the FastAPI RESTful API server, the TypeScript/React/Varnish user interface, and an NGINX web server.

Once the processes have finished starting, a program called sonar will display the accessible URL. The application is typically available at http://localhost:8080.

Note: The first startup may take a few minutes. The application supports automatic updates as you make changes, though you may occasionally need to refresh your browser.

docker compose up --build

The SQA Edit Workflow allows you to modify existing research reports using natural language instructions. It operates by mirroring the standard report generation pipeline but includes an additional step (Step 0) to retrieve the current report context and uses extended prompts to handle edits.

To trigger the edit workflow, set the edit_existing flag to true in your request. You can provide a thread_id to identify the report you want to modify, an edit_instruction describing the changes, and an optional list of mentioned_papers (corpus IDs) to specifically include in the edit.

{
    "edit_existing": true,
    "thread_id": "task-id-123",
    "edit_instruction": "Add these papers to section 2",
    "mentioned_papers": [12345, 67890]
}

To add new UI dependencies (React/TypeScript), you must install them within the running ui container using yarn.

Follow these steps:

1. Ensure the environment is running via docker compose up.
2. Access the ui container shell.
3. Install the package using yarn add.
4. Exit the container.
5. Commit and push both yarn.lock and package.json to the repository.

Important: You must commit and push both yarn.lock and package.json for changes to persist in future builds.

# 1. Enter the ui container
docker compose exec ui /bin/sh

# 2. Install the dependency
yarn add <dependency>

# 3. Exit
exit