Sotopia: Social Interaction Simulation Platform

repository·main·Indexed 43 days ago

https://github.com/sotopia-lab/sotopia

An open-ended social learning environment for evaluating and facilitating social intelligence in language agents. Version 0.1.5 provides a scalable framework for multi-agent interactions, featuring TOML-based dataflow configurations, a variety of agent actions (speak, thought, browse, run, etc.), and specialized simulation environments such as the Negotiation Arena and the Duskmire Werewolves social deduction game.

Tokens
42.7K
Snippets
139
Records
193
Agent score
55%

What's inside Sotopia

  1. What is an Agent in Sotopia?

    main
    An Agent is a decision-making entity that interacts within a social environment. Agents can represent human participants, AI models, or other entities. Regardless of their underlying type, all agents share a common interface for environmental interaction: their inputs and outputs are derived types of aact.messages.DataModel.
  2. What is an Environment in Sotopia?

    main
    An Environment in Sotopia is the orchestration layer for agent interactions. It provides the social context required for agents to communicate, manages the passing of messages between them, and evaluates the quality of those social interactions. Environments are highly customizable, allowing developers to test different social scenarios, turn-ordering logic, and evaluation metrics.
  3. Overview of experimental API states

    main

    Experimental APIs in Sotopia are categorized by their development lifecycle status. When choosing an API for your project, note its state:

    • scheduled: The API is slated to be merged into the next minor releases.
    • implemented: The API is functional and ready for use, though it may be merged into stable APIs in upcoming minor releases.
    • planned: The API is designed and scheduled for future implementation.
    • idealized: The API represents a target design that might be implemented in the future.
  4. Create custom evaluation dimensions and lists

    main

    To use customized metrics, you must first initialize the database (refer to examples/use_custom_dimensions.py). Once initialized, you can use the following classes to define your evaluation structure:

    • CustomEvaluationDimension: Defines a single metric. Requires name (str), description (str), range_low (int), and range_high (int).
    • CustomEvaluationDimensionList: Groups multiple dimensions together for specific use cases. Requires name (str) and dimension_pks (list of primary keys of the dimensions).
    • EvaluationDimensionBuilder: A utility class used to generate or select dimension models.

    Note: SotopiaDimensions is the default, but CustomEvaluationDimension objects are not in the database by default and require manual initialization.

  5. Configure simulation tags and duplicate prevention

    main

    Sotopia uses tags to identify simulations in the database and prevent redundant runs.

    • TAG: A string used to identify the specific simulation in the database.
    • TAG_TO_CHECK_EXISTING_EPISODES: A parameter used by scripts like examples/experiment_eval.py. If this is enabled and the script finds existing episodes in the database with the same TAG, the simulation will not run.

    To re-run a simulation that has already been tagged, you must either:

    1. Change the TAG value.
    2. Set TAG_TO_CHECK_EXISTING_EPISODES to None.
  6. How XMLRenderers work

    main

    XMLRenderers take an XML string and a RenderContext as input to determine how content is displayed. By default, the renderer processes two tags: root and p (paragraph).

    Visibility of the text within these tags is controlled by the viewer attribute in the RenderContext. The viewer value determines what content is exposed:

    • "environment": Renders all text.
    • "human": Renders the raw XML string.
    • "agent_i": Renders only the text that is viewable by agent_i.

    Supported tags include:

    • root
    • p
    • <extra_info>
    • <clarification_hint>
    • <strategy_hint>
  7. How ParallelSotopiaEnv works

    main

    The ParallelSotopiaEnv is a general-purpose environment designed to run multiple agents. While agents can run in parallel, the environment supports three specific turn-ordering modes:

    1. Random ordering: In each turn, an agent is chosen at random to act.
    2. Round-robin ordering: Agents act in a fixed sequence (agent i % n acts in turn i, where n is the number of agents).
    3. Simultaneous ordering: All agents take actions at the same time.

    In both Random and Round-robin modes, only one agent's aact method is called per turn, while the others remain idle.

  8. Use SocialGame and SocialDeductionGame classes

    main

    The engine provides two primary class hierarchies for building social simulations:

    SocialGame (Abstract Base Class)

    Use this for any multiplayer social game. It provides core logic for:

    • Turn Management: Supports round-robin (sequential) or simultaneous (parallel) action execution.
    • State Transitions: Manages the flow of the game through defined states.

    SocialDeductionGame (Specialized Subclass)

    Use this for deduction-style games (e.g., Werewolves, Undercover, Spyfall). It adds:

    • Action Masking: Enforces constraints on which agents can act during specific phases.
    • Visibility System: Controls message visibility levels: Public, Team-only, or Private.
    • Environment Notifications: Automatically broadcasts state changes to all agents to maintain valid context, even during private phases.
  9. Extend Sotopia by subclassing core components

    main

    Sotopia is designed with a modular architecture that allows developers to create custom social agents, environments, and evaluators by subclassing the following base classes:

    • BaseAgent: For creating custom agents.
    • ParallelSotopiaEnv: For creating custom social environments.
    • Evaluator: For creating custom evaluation logic.
  10. Understand the Social Game Engine core concepts

    main

    The Social Game Engine is an experimental module for multi-agent social simulations. Unlike standard dyadic (two-party) interactions, it is designed to handle:

    • Multiple Agents: Simultaneous interaction between more than two agents.
    • Roles & Teams: Agents with distinct roles (e.g., Villager, Werewolf) and conflicting objectives.
    • Dynamic Eras/Phases: Structured game progression through distinct states (e.g., Day Discussion, Night Action).
    • Private Information: Mechanisms for agents to hold secrets and maintain limited visibility of others' actions.
  11. Understand the Sotopia AACT experimental engine model

    main

    Sotopia is transitioning its experimental features to the AACT engine, an actor model library featuring strong typing and validation.

    Unlike traditional multi-agent frameworks (like Autogen or CrewAI) that rely on asyncio for turn-based, cooperative interactions, the AACT engine treats each agent as an independent process. This enables a "real-time" asynchronous interaction flow where agents can act independently of one another, which is essential for realistic social simulations where agents do not necessarily wait for a turn-based response from others.

  12. How Sotopia uses Redis

    main

    Sotopia utilizes Redis for two primary purposes:

    1. Data Storage: Persisting agent profiles, environment profiles, relationship profiles, and interaction logs. This enables reproducibility and analysis of simulated interactions.
    2. Message Brokering: Serving as a broker between the built-in Redis agent and frontends (such as human user interfaces or other agents).

    Sotopia primarily interacts with Redis using Redis-OM (Object Mapping) combined with Pydantic for structured data, though it may use redis-py directly for performance-critical operations.