DOCSEARCHbyAlgolia
Submit Docs

AutoSci Documentation

repository·main·Indexed 23 days ago

https://github.com/skyllwt/autosci

A memory-centric agentic system designed to automate the full scientific research lifecycle, including literature review, experimental design, execution, and manuscript writing. AutoSci supports multiple runtimes including Claude Code, Codex, and OpenCode Preview, providing over 30 agent skills for knowledge base management, ideation, and experiment execution. It features structured persistent memory, remote GPU server configuration, and integration with arXiv for daily recommendations.

Tokens
10.3K
Snippets
18
Records
38
Agent score
78%

What's inside AutoSci

  1. Understand the AutoSci runtime directory structure

    main

    The AutoSci runtime environment is organized into three main top-level directories: wiki/, raw/, and config/.

    wiki/

    Contains the structured knowledge base and research artifacts.

    • papers/, concepts/, topics/, people/, ideas/, experiments/, methods/, Summary/: Structured research entities.
    • foundations/: Background knowledge (receives links, but does not write them).
    • outputs/: Generated artifacts like Related Work and paper drafts.
    • graph/: Auto-generated data (edges, citations, context). Do not edit manually; maintain via tools/research_wiki.py.
    • CLAUDE.md, index.md, log.md: System schema, content catalog, and chronological logs.

    raw/

    Contains source materials and user-provided inputs.

    • raw/papers/, raw/notes/, raw/web/: User-owned inputs. Place your .tex, .pdf, .md, or HTML files here.
    • raw/discovered/: Managed by the system for externally fetched papers (e.g., via /init or /daily-arxiv). Do not manually drop files here.
    • raw/tmp/: Intermediate state generated during /init or /ingest processes.

    config/

    Contains environment and server settings.

    • server.yaml: Configuration for remote GPU servers (required if using /exp-run --env remote).
    • .env: Environment variables.
    • settings.local.json: Local settings.

  2. How to invoke AutoSci skills in Claude Code and Codex

    main

    AutoSci provides over 30 agent skills categorized by research phase. You can invoke these skills using two different runtimes:

    1. Claude Code: Use slash commands (e.g., /init).
    2. Codex: Use dollar commands (e.g., $init) or select the skill from the /skills menu.

    Each skill has the same name in both runtimes, only the prefix changes.

  3. How ΩmegaWiki configuration works

    main

    All API keys are stored in a .env file located in the project root. This file is created by setup.sh from .env.example.

    Python tools within the project load these variables automatically via tools/_env.py upon startup, so manual export commands are not required. Note that Claude Code itself does not read the .env file; only the underlying Python tools do.

    The /setup skill is designed to manage this configuration by checking currently set keys, explaining them, and writing new values directly into the .env file using the Edit tool.

  4. Set up environment variables for API keys

    main

    API keys and sensitive credentials should be stored in a .env file in the project root. Copy the example file and edit it with your specific keys.

    Refer to the comments within the .env file for detailed instructions on each required key.

    cp config/.env.example .env

  5. Verify configuration settings

    main

    To ensure your .env keys are correctly loaded, run the following command in your terminal. This script checks the status of the primary API keys using the project's internal _env module.

    source .venv/bin/activate && python3 -c "
import _env, os
keys = ['SEMANTIC_SCHOLAR_API_KEY', 'DEEPXIV_TOKEN', 'LLM_API_KEY', 'LLM_BASE_URL', 'LLM_MODEL']
for k in keys:
    v = os.environ.get(k, '')
    status = '✓ set' if v else '✗ not set'
    print(f'{status}  {k}')
"
    source .venv/bin/activate && python3 -c "
import _env, os
keys = ['SEMANTIC_SCHOLAR_API_KEY', 'DEEPXIV_TOKEN', 'LLM_API_KEY', 'LLM_BASE_URL', 'LLM_MODEL']
for k in keys:
    v = os.environ.get(k, '')
    status = '✓ set' if v else '✗ not set'
    print(f'{status}  {k}')
"

  6. Quick Start with Claude Code (Stable)

    main

    For users of Claude Code, use the stable main branch. This requires installing @anthropic-ai/claude-code via npm, logging in, and then using the /init command within the Claude interface.

    Workflow:

    1. Clone the main branch.
    2. Install and login to Claude Code.
    3. Run claude.
    4. Type /init [your-research-topic].
    git clone -b main https://github.com/skyllwt/AutoSci.git
cd AutoSci
npm install -g @anthropic-ai/claude-code
claude login
claude
# Then type: /init [your-research-topic]

  7. Manual Setup for AutoSci (Linux / macOS / Windows)

    main

    If you prefer not to use the provided setup.sh or setup.ps1 scripts, you can manually configure the environment. This involves creating a Python virtual environment, installing dependencies, setting up environment variables, and manually copying skill files from i18n/en/ to the appropriate .agents/skills/ or .claude/skills/ directories.

    Note for Windows users: Remote-GPU experiments via /exp-run --env remote rely on ssh/rsync/screen and are best run from WSL2 or Linux/macOS.

    # Manual setup (Linux / macOS)
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env                 # Edit to add API keys
mkdir -p .agents/skills/shared-references
cp -R i18n/en/skills/. .agents/skills/
cp i18n/en/shared-references/*.md .agents/skills/shared-references/
mkdir -p .claude/skills/shared-references
cp -R i18n/en/skills/. .claude/skills/
cp i18n/en/shared-references/*.md .claude/skills/shared-references/
cp config/settings.local.json.example .claude/settings.local.json  # Claude Code compatibility
    # Manual setup (Windows / PowerShell)
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
Copy-Item .env.example .env          # Edit to add API keys
New-Item -ItemType Directory -Force .agents\skills\shared-references | Out-Null
Copy-Item i18n\en\skills\* .agents\skills -Recurse -Force
Copy-Item i18n\en\shared-references\*.md .agents\skills\shared-references -Force
New-Item -ItemType Directory -Force .claude\skills\shared-references | Out-Null
Copy-Item i18n\en\skills\* .claude\skills -Recurse -Force
Copy-Item i18n\en\shared-references\*.md .claude\skills\shared-references -Force
Copy-Item config\settings.local.json.example .claude\settings.local.json  # Claude Code compatibility

  8. Quick Start with Codex Preview

    main

    To use the Codex Preview version of AutoSci, ensure you have Python 3.9+ and Node.js 18+ installed. This workflow requires installing and signing into Codex, running a one-click setup to create a virtual environment and sync skills, and then invoking the init command within Codex.

    Workflow:

    1. Clone the autosci-codex branch.
    2. Install and verify codex.
    3. Run setup.sh (or setup.ps1 on Windows) to create .venv and sync the Codex .agents/skills tree.
    4. Add papers (.tex or .pdf) to raw/papers/.
    5. Run codex and invoke $init [your-research-topic].
    # 1. Clone the Codex Preview branch
git clone -b autosci-codex https://github.com/skyllwt/AutoSci.git
cd AutoSci

# 2. Install and sign in to Codex
codex --version

# 3. One-click setup
chmod +x setup.sh && ./setup.sh

# 4. Put your own papers in raw/papers/ (.tex or .pdf)

# 5. Build your research memory and start a project
codex
# Then invoke: $init [your-research-topic]

  9. Configure daily arXiv recommendation preferences

    main

    To customize your daily arXiv recommendation preferences (mode, categories, recommendation caps, schedule, and profile hints), copy the template to config/daily-arxiv.yml.

    Note: This file is for non-secret settings only. Do not store API keys or SMTP credentials here; use the .env file instead.

    cp config/daily-arxiv.yml.example config/daily-arxiv.yml

  10. Manage source files and automated ingestion directories

    main

    When working with AutoSci, distinguish between user-provided inputs and system-managed directories to avoid manual errors:

    • User Inputs: Place your own .tex, .pdf, .md notes, or web content in raw/papers/, raw/notes/, and raw/web/ respectively.
    • Automated Scrapes: Do not manually place files in raw/discovered/. This directory is reserved for papers fetched by the /init or /daily-arxiv commands.
    • Intermediate Files: raw/tmp/ is used for intermediate states generated by /init or local /ingest operations.
    • Derived Data: The wiki/graph/ directory is a derived output. It must only be maintained via tools/research_wiki.py and should not be edited manually.

  11. Quick Start with OpenCode Preview

    main

    To use the OpenCode Preview version of AutoSci, ensure you have Python 3.9+, Node.js 18+, and OpenCode installed. This workflow involves cloning a specific branch, running a setup script to generate the local environment and skills, and then using OpenCode to run the init skill for your research topic.

    Workflow:

    1. Clone the autosci-opencode branch.
    2. Verify opencode installation.
    3. Run ./setup.sh --lang en to generate the local environment, skills, AGENTS.md, and opencode.json.
    4. Populate raw/papers/, raw/notes/, or raw/web/ with your research data.
    5. Launch opencode and use the init skill.
    # 1. Clone the OpenCode Preview branch
git clone -b autosci-opencode https://github.com/skyllwt/AutoSci.git
cd AutoSci

# 2. Verify OpenCode
opencode --version

# 3. Generate the local environment, skills, AGENTS.md, and opencode.json
chmod +x setup.sh && ./setup.sh --lang en

# 4. Add your papers and optional notes
#    raw/papers/  raw/notes/  raw/web/

# 5. Start OpenCode and load the init skill
opencode

  12. Setup the `/daily-arxiv` GitHub Actions pipeline

    main

    To deploy the daily arXiv pipeline on GitHub Actions, follow these steps:

    1. Configure Authentication: Choose one of the following secrets and set it using gh secret set <NAME>:
      • ANTHROPIC_API_KEY: Pay-as-you-go API quota.
      • CLAUDE_CODE_OAUTH_TOKEN: Pro/Max subscription quota (generate via claude setup-token).
    2. Install Claude Code GitHub App: You must install the Claude Code GitHub App on your repository. The auth secret alone is insufficient; the app is required to exchange OIDC for a usable token.
    3. Mirror API Keys: The pipeline requires SEMANTIC_SCHOLAR_API_KEY and DEEPXIV_TOKEN to avoid anonymous-tier rate limits. Note that DEEPXIV_TOKEN is typically located in ~/.env.
      gh secret set SEMANTIC_SCHOLAR_API_KEY -b "$(grep ^SEMANTIC_SCHOLAR_API_KEY= .env | cut -d= -f2-)"
gh secret set DEEPXIV_TOKEN            -b "$(grep ^DEEPXIV_TOKEN= ~/.env       | cut -d= -f2-)"
    4. Expose Secrets to Workflow: Run /daily-arxiv setup in your local checkout to auto-patch .github/workflows/daily-arxiv.yml. If the skill is unavailable, manually add the following to the daily-arxiv: job's env: block:
      SEMANTIC_SCHOLAR_API_KEY: ${{ secrets.SEMANTIC_SCHOLAR_API_KEY }}
DEEPXIV_TOKEN:            ${{ secrets.DEEPXIV_TOKEN }}
    5. Configure SMTP (Optional): If email.enabled: true in config/daily-arxiv.yml, set the following secrets: SMTP_HOST, SMTP_PORT, SMTP_USER, SMTP_PASSWORD, SMTP_FROM, DAILY_ARXIV_EMAIL_TO.
    6. Verify: Manually trigger the workflow to ensure success:
      gh workflow run daily-arxiv.yml --ref main
gh run watch
    # Mirroring API keys to repo secrets
gh secret set SEMANTIC_SCHOLAR_API_KEY -b "$(grep ^SEMANTIC_SCHOLAR_API_KEY= .env | cut -d= -f2-)"
gh secret set DEEPXIV_TOKEN            -b "$(grep ^DEEPXIV_TOKEN= ~/.env       | cut -d= -f2-)"

# Manual dispatch and watch
gh workflow run daily-arxiv.yml --ref main
gh run watch