repository·main·Indexed 18 days ago
https://github.com/noviscl/ai-researcherA research ideation agent developed by Stanford NLP that uses LLMs and retrieval-augmented generation (RAG) to generate, rank, and filter detailed research project proposals. The pipeline includes related paper search via the Semantic Scholar API, grounded idea generation, deduplication using sentence-transformers, and multi-dimensional filtering for novelty, feasibility, significance, and consistency.
Expands each seed idea into a highly detailed project proposal that includes step-by-step instructions for execution.
Usage:
Run scripts/project_proposal_gen.sh from the ai_researcher directory.
cd ai_researcher
bash scripts/project_proposal_gen.shTo use the research ideation agent, set up a Python 3.10 environment using Conda and install the required dependencies. You must also provide API keys for OpenAI and Anthropic in a keys.json file located in the project root.
Required Configuration:
Create a keys.json file with the following structure:
{
"api_key": "Your OpenAI API Key",
"organization_id": "Your OpenAI Organization ID (Optional)",
"s2_key": "Your Semantic Scholar API Key (Optional)",
"anthropic_key": "Your Anthropic API Key"
}git clone https://github.com/NoviScl/AI-Researcher.git
cd AI-Researcher
conda create -n ai-researcher python=3.10
conda activate ai-researcher
pip install -r requirements.txtThe agent provides a single script to run the entire pipeline sequentially: Related Paper Search $\rightarrow$ Grounded Idea Generation $\rightarrow$ Idea Deduplication $\rightarrow$ Project Proposal Generation $\rightarrow$ Project Proposal Ranking $\rightarrow$ Project Proposal Filtering. This generates detailed project proposals from a natural language research topic.
Usage:
Run the end_to_end.sh script from the ai_researcher directory.
cd ai_researcher
bash scripts/end_to_end.shAn optional final step that checks if a proposal is novel and feasible. For novelty, the system retrieves similar papers and uses an LLM to determine if the proposal is essentially the same as existing work. Proposals that fail the novelty check are filtered out.
Usage:
Run scripts/project_proposal_filter.sh from the ai_researcher directory.
cd ai_researcher
bash scripts/project_proposal_filter.shRemoves similar ideas using cosine similarity on sentence embeddings from the sentence-transformers library. This step is local and does not consume API credits.
Key Hyperparameter:
similarity_threshold: Defaults to 0.8.Usage:
Run scripts/idea_dedup.sh from the ai_researcher directory.
cd ai_researcher
bash scripts/idea_dedup.shThis module generates a list of research ideas based on a topic description and optionally uses retrieved papers for retrieval-augmented generation (RAG).
Configuration Options:
ideas_n: Recommended to set to 5 to manage output length constraints; run multiple times with different seeds to collect more ideas.RAG: Set to True or False to enable/disable grounding on retrieved papers.Usage:
Run scripts/grounded_idea_gen.sh from the ai_researcher directory.
cd ai_researcher
bash scripts/grounded_idea_gen.shUses an LLM ranker to score generated project proposals. The output is a JSON file containing the scores for each proposal, which can be used to order them by estimated quality.
Usage:
Run scripts/project_proposal_ranking.sh from the ai_researcher directory.
cd ai_researcher
bash scripts/project_proposal_ranking.shThis module iteratively proposes search queries and searches through the Semantic Scholar API to find relevant literature. It uses an LLM to score and rerank the retrieved papers. The output is a list of relevant papers stored in a cache file.
Key Hyperparameter:
max_paper_bank_size: Controls when the search stops (e.g., max_paper_bank_size=120).Usage:
Run scripts/lit_review.sh from the ai_researcher directory.
cd ai_researcher
bash scripts/lit_review.sh The script expects a JSON configuration file located at ../keys.json relative to the script's execution path. The following keys must be present:
anthropic_key: API key for Anthropic/Claude.api_key: API key for OpenAI.organization_id: OpenAI organization ID.{
"anthropic_key": "your_anthropic_key",
"api_key": "your_openai_key",
"organization_id": "your_org_id"
}This module requires a Semantic Scholar API key. It expects a file named ../keys.json to exist in the parent directory with the following structure:
{
"s2_key": "YOUR_API_KEY"
}Convert a list of paper dictionaries into a formatted string suitable for printing or inclusion in an LLM prompt.
Options:
include_abstract (bool): If true, includes the abstract or the tldr.text if the abstract is missing.include_score (bool): If true, includes the score field if present.include_id (bool): If true, includes the paperId.print_str = format_papers_for_printing(paper_list, include_abstract=True)The call_api function provides a unified interface to interact with Anthropic, Together, and OpenAI models. It handles model-specific logic, including prompt modification for JSON output and cost calculation.
Supported Model Families:
claude in the name.llama, qwen, or qwq.gpt-4o or o1 series.Key Features:
json_output=True, the function appends specific instructions to the prompt to ensure the model returns a raw JSON dictionary without markdown code blocks (e.g., no ```json). For supported models (like meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo on Together or OpenAI's json_object mode), it sets the appropriate response_format.Note: For o1 models, the function uses max_completion_tokens instead of max_tokens to comply with OpenAI's API requirements for that model family.
# Example usage with a generic client
response, cost = call_api(
client=client,
model="gpt-4o",
prompt_messages=[{"role": "user", "content": "Your prompt here"}],
json_output=True
)