VTube Studio Public API Documentation

repository·master·Indexed 22 days ago

https://github.com/denchisoft/vtubestudio

Documentation for the VTube Studio public API, enabling developers to build plugins and scripts to control avatars, feed tracking data, and interact with the environment. Includes detailed guides on event subscriptions (such as ModelLoadedEvent, ModelMovedEvent, and HotkeyTriggeredEvent), handling model interactions, and integrating with the Live2D Cubism Animation Editor.

Tokens
21.5K
Snippets
59
Records
74
Agent score
79%

What's inside VTube Studio API

  1. Overview of the VTube Studio Public API

    master

    VTube Studio provides a public API that allows developers to write plugins or scripts to interact with the software. Capabilities include:

    • Triggering hotkeys
    • Feeding in face tracking data
    • Loading items and models
    • Receiving event notifications
    • Tinting model ArtMeshes
    • And more.
  2. Understand the structure of post-processing effects and configs

    master

    The postProcessingEffects array contains all available visual effects. Each effect is composed of multiple configEntries that define its parameters (e.g., strength, color, or texture).

    Config Types

    Each configuration entry has a type field. Only the fields corresponding to that type will be populated; all other fields should be ignored:

    • Float: Uses floatValue, floatMin, floatMax, and floatDefault.
    • Int: Uses intValue, intMin, intMax, and intDefault.
    • Bool: Uses boolValue and boolDefault.
    • String: Uses stringValue and stringDefault.
    • Color: Uses colorValue (RGBA hex string, e.g., "77CCAAFF") and colorDefault. Use colorHasAlpha to check if the alpha channel is relevant.
    • SceneItem: A string value representing a filename (ending in .jpg or .png) located in the VTube Studio "Items" folder.

    Identifying Effects

    • enumID: The primary ID used to identify the effect and its configs.
    • internalID: The ID used by VTube Studio internally for preset JSON files (not recommended for most use cases, but usable via API).
    {
    	"apiName": "VTubeStudioPublicAPI",
    	"apiVersion": "1.0",
    	"requestID": "SomeID",
    	"messageType": "PostProcessingListResponse",
    	"data": {
    		"postProcessingEffects": [
    			{
    				"internalID": "color_grading",
    				"enumID": "ColorGrading",
    				"configEntries": [
    					{
    						"enumID": "ColorGrading_Strength",
    							"type": "Float",
    						"activationConfig": true,
    						"floatValue": 0.0
    						
    					},
    						{
    						"enumID": "ColorGrading_ColorFilter",
    							"type": "color",
    						"colorValue": "FFFFFFFF"
    						}
    					]
    				}
    			]
    		}
    	}
    }
  3. Structure a VTube Studio API Request

    master

    Every request sent to the VTube Studio API must include the apiName and apiVersion.

    • apiName: Must be exactly "VTubeStudioPublicAPI".
    • apiVersion: Currently "1.0". This version remains constant even when new fields are added; your parser should be resilient to unknown fields. The version only increments for incompatible changes (e.g., renaming or removing fields).
    • requestID (Optional): A string of ASCII characters (1-64 characters). It is highly recommended to use this to map responses to requests and to help debug errors via VTube Studio logs.
    • messageType: Defines the type of action being performed.
    {
    	"apiName": "VTubeStudioPublicAPI",
    	"apiVersion": "1.0",
    	"requestID": "MyIDWithLessThan64Characters",
    	"messageType": "APIStateRequest"
    }
  4. Subscribe to VTube Studio Events

    master
    In addition to the request/response model, VTube Studio provides an Event API. This allows plugins to subscribe to specific occurrences within the application, such as when a model is loaded or when a hotkey is executed.
  5. How permissions work in VTube Studio

    master

    Certain VTube Studio API functionalities are protected by a permission system. To use these features, a plugin must request permission after authenticating. When a permission is requested, VTube Studio displays a popup to the user explaining the permission and the plugin's identity, allowing the user to grant or deny access.

    Key behaviors:

    • Granting: Once a permission is granted, it cannot be revoked via the API. However, users can manually revoke permissions at any time via the VTube Studio API configuration settings for your specific plugin.
    • Re-requesting: If a plugin has already been granted a permission, subsequent requests for that same permission return instantly without showing a popup to the user.
    • Error Handling: Errors are returned if you request an unknown permission, if the user has the API config window open, or if there are file I/O issues with the plugin config.
  6. Handle API Permissions

    master

    Certain high-risk or sensitive functionalities (such as loading arbitrary images as items) are protected by a permission system.

    After authenticating, a plugin must explicitly request the necessary permissions. When a request is made, VTube Studio will display a popup to the user explaining the purpose of the permission, allowing them to either grant or deny access.

  7. Discover VTube Studio API state via UDP

    master

    VTube Studio broadcasts its API state on the local network via UDP on port 47779 every two seconds. This broadcast occurs even if the API is turned off in the user settings. This is useful for discovering the instanceID (a unique random ID for the running instance) and the windowTitle (which varies by OS and instance count).

    {
    	"apiName": "VTubeStudioPublicAPI",
    	"apiVersion": "1.0",
    	"timestamp": 1630159656406,
    	"messageType": "VTubeStudioAPIStateBroadcast",
    	"requestID": "VTubeStudioAPIStateBroadcast",
    	"data": {
    		"active": false,
    		"port": 8001,
    		"instanceID": "93aa0d0494304fddb057ae8a295c4e59",
    		"windowTitle": "VTube Studio"
    	}
    }
  8. Configure restricted or experimental effects

    master

    VTube Studio includes restricted or experimental visual effects. To use these when setting individual config values, you must explicitly set usingRestrictedEffects to true in your PostProcessingUpdateRequest payload.

    Requirements:

    • If usingRestrictedEffects is false, attempting to use a restricted effect returns PostProcessingUpdateRequestTriedToLoadRestrictedEffect.
    • Even if requested via API, the user must have manually enabled the usage of these effects in the VTube Studio VFX settings. If they haven't, the error PostProcessingUpdateRequestTriedToLoadRestrictedEffect will be returned.
    • Presets: If a loaded preset contains restricted effects, the preset will load without error, but the restricted effects will not be activated.
  9. Connect to the VTube Studio WebSocket API

    master

    The VTube Studio WebSocket server runs on ws://localhost:8001 by default, though users can change this port in the application settings.

    Connection Requirements

    • Message Types: While the API supports both binary and text messages, VTube Studio always responds with text messages. It is recommended to send text messages. If sending binary messages, ensure they are encoded in UTF-8.
    • Troubleshooting: If a connection fails, ensure the port is correct and that no firewall/antivirus is blocking it.
    • User Authorization: Users must manually select "Allow Plugin API access" in the main configuration page within VTube Studio to permit connections.
  10. Subscribe to and unsubscribe from VTube Studio events

    master

    To receive notifications when specific actions occur in VTube Studio (like hotkey activation or model loading), you must explicitly subscribe to those events using an EventSubscriptionRequest.

    Subscription Logic

    • Subscribe: Set subscribe: true in the request. If you subscribe to an event multiple times, the new config will overwrite the previous one.
    • Unsubscribe: Set subscribe: false for a specific eventName, or leave eventName empty to unsubscribe the entire plugin session from all events.
    • Automatic Cleanup: When your plugin disconnects from the VTube Studio API, all active subscriptions are automatically removed.

    Request Format

    Use the EventSubscriptionRequest message type. The config object allows for event-specific filtering (e.g., filtering by modelID).

    Response Format

    A successful request returns an EventSubscriptionResponse containing the subscribedEventCount and a list of subscribedEvents.

    {
        "apiName": "VTubeStudioPublicAPI",
        "apiVersion": "1.0",
        "requestID": "SomeID",
        "messageType": "EventSubscriptionRequest",
        "data": {
            "eventName": "ModelLoadedEvent",
            "subscribe": true,
            "config": {
                "modelID": ["1234567890abcdef1234567890abcdef"]
            }
        }
    }