OpenReview Python Client

repository·master·Indexed 42 days ago

https://github.com/openreview/openreview-py

A programmatic interface for accessing and modifying data within the OpenReview system. The library provides clients for both standard API interactions via openreview.Client and API 2 interactions via openreview.api.OpenReviewClient, allowing researchers and developers to manage conference data, reviews, user profiles, groups, and invitations.

Tokens
17.5K
Snippets
51
Records
73
Agent score
54%

What's inside OpenReview

  1. Manage Venue Workflows with the Venue class

    master

    The Venue class provides a high-level abstraction for managing conference-style workflows (submissions, reviews, decisions). You can use it to programmatically bootstrap a venue, manage committee matching, and handle bulk decisions.

    Key capabilities include:

    • Setup: Bootstrapping groups, invitations, and stages (submission, review, decision).
    • Committee Management: Recruiting reviewers and setting up matching (affinity scores and conflicts).
    • Assignments: Deploying and reverting reviewer assignments.
    • Decisions: Bulk-posting decisions from a CSV and sending automated email notifications to authors.
    • Lifecycle: Expiring submission invitations and retrieving accepted submissions.
    import openreview
    from openreview.venue import Venue
    
    client = openreview.api.OpenReviewClient(
        baseurl='https://api2.openreview.net',
        username='pc@example.com',
        password='...'
    )
    
    # Initialize the Venue helper
    venue = Venue(client, 'Conf25.cc/2025/Conference', 'openreview.net/Support')
    
    # Bootstrap venue groups and invitations
    venue.setup(
        program_chair_ids=['pc1@example.com', 'pc2@example.com']
    )
    
    # Create stages
    venue.create_submission_stage()
    venue.create_review_stage()
    venue.create_decision_stage()
  2. Wait for asynchronous processing in OpenReview

    master

    When performing write operations like post_note_edit, the server may process the request asynchronously. You can handle this in two ways:

    1. await_process=True (Recommended): Pass this parameter to the method to block execution until the server-side process completes.
    2. Manual Polling: In test environments, you may need to manually poll process logs until the status reaches 'ok'.
    # Option 1: await_process parameter (recommended)
    result = client.post_note_edit(
        invitation='VenueID/-/Submission',
        signatures=['~Author1'],
        note=Note(content={...}),
        await_process=True  # blocks until server process completes
    )
    
    # Option 2: manual polling (for test environments)
    result = client.post_note_edit(
        invitation='VenueID/-/Submission',
        signatures=['~Author1'],
        note=Note(content={...})
    )
    # Then poll process logs until status is 'ok'
  3. Access Note content fields correctly

    master

    When accessing data within a Note's content dictionary, you must always access the ['value'] key. Accessing the field directly will return the dictionary wrapper rather than the actual data value.

    note = client.get_note(note_id)
    
    # Correct: always access through ['value']
    title = note.content['title']['value']           # str
    authors = note.content['authors']['value']        # list[str]
    authorids = note.content['authorids']['value']    # list[str]
    
    # WRONG: do not access content fields directly
    # title = note.content['title']  # Returns {'value': '...'}, not the string!
  4. Manage Journal Workflows with the Journal class

    master

    The Journal class is designed for managing journal-style workflows (e.g., TMLR), which differ from conference workflows by focusing on Action Editors (AEs) and continuous submission cycles.

    Key capabilities include:

    • Setup: Bootstrapping journal groups, invitations, and editor roles.
    • Recruitment: Inviting Action Editors and Reviewers.
    • AE Management: Configuring AE matching systems and deploying assignments.
    • Reporting: Generating CSV statistics for reviewer and AE performance.
    • Archiving: Moving completed submissions to archived slots and cleaning up scoring edges.
    from openreview.journal import Journal
    
    journal = Journal(
        client,
        venue_id='TMLR',
        secret_key='your-secret-key',
        contact_info='tmlr@jmlr.org',
        full_name='Transactions on Machine Learning Research',
        short_name='TMLR'
    )
    
    # Bootstrap all journal groups, invitations, and group variables
    journal.setup(
        support_role='OpenReview.net/Support',
        editors=['~EIC_One1', '~EIC_Two1'],
        assignment_delay=5  # minutes before assignment processes run
    )
  5. Run integration tests with Docker (Recommended)

    master

    The easiest way to run integration tests is via Docker Compose. This requires Docker and requires the following sibling repositories to be cloned next to openreview-py:

    • openreview-api
    • openreview-api-v1
    • openreview-web
    • openreview-py (this repo)

    Setup

    1. Navigate to the docker directory.
    2. Copy the example config: cp config.example.json config.json.
    3. Required: Edit config.json with your local repository paths.

    Test Commands

    • Run a specific test file: ./run.py tests/test_client.py
    • Run a specific test with verbose output: ./run.py tests/test_client.py::TestClient::test_get_groups -v
    • Run all tests: ./run.py

    Each run starts the infrastructure (MongoDB, Redis, Elasticsearch, web), runs tests, and tears down services automatically.

    cd docker
    cp config.example.json config.json
    # Edit config.json with your repo paths
    ./run.py tests/test_client.py
  6. Install openreview-py

    master

    You can install the OpenReview Python library using pip or by cloning the repository for development.

    Prerequisites

    • Python 3.9 or newer

    Standard Installation

    pip install openreview-py

    Development Installation (Editable)

    git clone https://github.com/openreview/openreview-py.git
    cd openreview-py
    pip install -e .
  7. Run tests locally without Docker

    master

    To run tests without Docker, you must manually manage the required services and the Selenium driver.

    1. Prerequisites

    • Clone and configure openreview-api-v1, openreview-api-v2, and openreview-web to run on ports 3000, 3001, and 3030 respectively.
    • Install the package with test dependencies: pip install -e ".[test]"
    • Download the appropriate Firefox Selenium driver (geckodriver) and place it in openreview-py/tests/drivers/.

    2. Start Services

    • In openreview-api-v1 directory: npm run cleanStart
    • In openreview-api-v2 directory: npm run cleanStart
    • In openreview-web directory: SUPER_USER=openreview.net npm run dev

    3. Run Tests

    • Ensure any existing OpenReview credentials are cleared: unset OPENREVIEW_USERNAME && unset OPENREVIEW_PASSWORD
    • Run all tests: pytest
    • Run a specific file: pytest tests/test_client.py
    pip install -e ".[test]"
    # ... start services ...
    pytest
  8. Use Serve Mode for manual browser testing

    master

    You can start all services with ports exposed to your host machine for manual testing via the ./run.py script.

    • Start services only (browse http://localhost:3030): ./run.py --serve
    • Populate DB by running a test, then keep services running: ./run.py --serve tests/test_icml_conference.py
    • Preserve existing data across restarts (skip cleanStart): ./run.py --serve --no-clean
    • Serve with interactive shell access: ./run.py --serve --shell
    • Populate DB, then drop into shell: ./run.py --serve --shell tests/test_icml_conference.py

    Press Ctrl+C to stop. If keep_infra is enabled, only API servers are stopped. Use docker compose down to stop everything.

    ./run.py --serve
  9. Install openreview-py for Production or CI

    master

    For reproducible deployments in production or CI environments, use the pinned lock file as constraints to ensure dependency versions are exact.

    Production Installation

    pip install -c requirements.txt .

    Development Installation (Editable with pinned dependencies)

    pip install -c requirements.txt -e .
  10. Update dependencies using pip-tools

    master

    This project uses pip-tools to manage dependency versions. pyproject.toml defines compatible ranges, while requirements.txt (production) and requirements-dev.txt (test + docs) contain pinned versions.

    Update all dependencies

    pip-compile pyproject.toml -o requirements.txt --strip-extras
    pip-compile pyproject.toml --extra test --extra docs -o requirements-dev.txt --strip-extras

    Update a single package

    pip-compile --upgrade-package requests pyproject.toml -o requirements.txt --strip-extras

    Note on NumPy pinning: Because pip-compile resolves for the current Python version, you must manually add environment-marked entries for NumPy in the generated lock files to support multiple Python versions:

    numpy==2.0.2 ; python_version < '3.10'
    numpy==2.2.6 ; python_version == '3.10'
    numpy==2.4.3 ; python_version >= '3.11'
    pip-compile pyproject.toml -o requirements.txt --strip-extras
  11. Authenticate with the OpenReview v2 API

    master

    To use the recommended v2 API, instantiate openreview.api.OpenReviewClient. You can authenticate using a username and password, or via a bearer token. If using a token, it takes precedence over username/password credentials.

    Authentication Methods:

    • Username/Password: Pass username and password directly to the constructor.
    • Token-based: Pass a token. You can also specify tokenExpiresIn (in seconds, max 1 week) when using username/password to manage session duration.
    • Environment Variables: The client can be initialized without arguments if OPENREVIEW_USERNAME, OPENREVIEW_PASSWORD, and OPENREVIEW_API_BASEURL_V2 are set in your environment.
    • Guest Access: Initializing the client without credentials provides read-only access to public notes, groups, and profiles.
    import openreview
    
    # v2 API (recommended)
    client = openreview.api.OpenReviewClient(
        baseurl='https://api2.openreview.net',
        username='user@example.com',
        password='your_password'
    )
    
    # Token-based auth
    client = openreview.api.OpenReviewClient(
        baseurl='https://api2.openreview.net',
        token='your_bearer_token'
    )
    
    # With custom expiration (seconds, max 1 week)
    client = openreview.api.OpenReviewClient(
        baseurl='https://api2.openreview.net',
        username='user@example.com',
        password='your_password',
        tokenExpiresIn=86400  # 1 day
    )
    
    # Guest access (read-only)
    guest = openreview.api.OpenReviewClient(baseurl='https://api2.openreview.net')