DOCSEARCHbyAlgolia
Submit Docs

Label Studio

repository·develop·Indexed 12 days ago

https://github.com/humansignal/label-studio

An open-source data labeling tool for audio, text, images, video, and time series. It features a UI for manual annotation and integrates with ML models for automated pre-labeling and active learning. Version 1.24.0.dev0 includes a Django-based FSM framework for state transitions, support for synchronous and asynchronous data imports, and a flexible feature flag system.

Tokens
521.3K
Snippets
1.2K
Records
1.9K
Agent score
93%

What's inside Label Studio

  1. Overview of Prompts in Label Studio

    develop

    The Prompts feature allows you to evaluate and refine LLM (Large Language Model) prompts to generate automated predictions for your labeling tasks. This feature is designed to increase annotation throughput and accuracy by leveraging AI to handle bulk labeling, allowing subject matter experts to focus on higher-level tasks.

    Key Capabilities

    • Auto-labeling: Generate predictions for thousands of tasks using an LLM.
    • Bootstrapping: Kickstart new projects by generating predictions without requiring an initial ground truth dataset.
    • Prompt Evaluation: Use accuracy scores against ground truth data to iteratively fine-tune prompts for better performance.

    Requirements and Constraints

    • Supported Data Types: Text and Image (Note: Images must be uploaded via cloud storage).
    • Supported Object Tags: Text, HyperText, Image.
    • Supported Control Tags: Choices (Text/Image), Labels (Text), TextArea (Text/Image), Pairwise (Text/Image), Number (Text/Image), Rating (Text/Image).
    • Task Size Limit: Maximum 1MB per task (approx. 200-500 pages of text).
    • Text Encoding: Must be UTF-8 compatible.
    • Permissions:
      • Owners, Administrators, Managers: Can create Prompt models and apply auto-annotations.
      • Reviewers and Annotators: Cannot access the Prompts tool but can view generated predictions.
    • ML Backend: Do not use Prompts with projects connected to an ML backend, as it may interfere with evaluation metrics.
    • Availability: Available in Label Studio Enterprise and Starter Cloud.

  2. Overview of Label Studio Interfaces

    develop

    Interfaces are a powerful UI builder in Label Studio that allow you to create highly customized, React-based labeling interfaces. Unlike the classic XML tag system, Interfaces allow you to describe the desired experience and build fully interactive UIs that support complex data types (like 3D files, GEOTiff, or DICOM) and custom business logic written directly into the component.

    Availability

    Interfaces are available in Label Studio Enterprise (including self-hosted) and Starter Cloud.

    Development Paths

    There are two primary ways to build an Interface:

    1. Create with Agent: An in-product builder available on Label Studio Cloud only.
    2. Develop Locally: Build using your own coding tools and agents (e.g., Claude Code, Cursor) and sync the result to Label Studio. This is the required method for self-hosted, on-prem, or air-gapped deployments.

    To work with external coding agents, you may need to install the create-interface-skill skill.

  3. Overview of Label Studio components

    develop

    Label Studio is an NX-managed project composed of three primary parts:

    1. Main App (apps/labelstudio): The central application that integrates all frontend framework elements and manages the various libraries.
    2. Label Studio Frontend (libs/editor): A React and mobx-state-tree library designed for data annotation. It is highly customizable and intended for seamless integration into other applications.
    3. Datamanager (libs/datamanager): An advanced tool specifically for data exploration within the Label Studio ecosystem.

  5. Overview of Identity and Access Management in Label Studio Enterprise

    develop

    Label Studio Enterprise offers three primary methods for managing identity and access, depending on your deployment and automation needs:

    • SSO (SAML 2.0): Handles authentication. It allows users to log in via an external Identity Provider (IdP) like Okta, Google SAML, Azure AD/Microsoft Entra, Ping Identity, or OneLogin. This provides centralized login and enforces enterprise security policies.
    • SCIM 2.0: Handles provisioning. It automates the user lifecycle (Create, Update, Deactivate) and synchronizes user profile changes. It also supports 'Push Groups' to manage workspace and project membership by mapping groups to organization and project-level roles.
    • LDAP: Designed for on-premise deployments. It allows authentication against an existing directory (e.g., Active Directory) while maintaining Role-Based Access Control (RBAC) and project/workspace permissions within Label Studio.

    Recommended Workflow: Most enterprises enable SSO first for authentication, followed by SCIM for ongoing user and group synchronization.

  6. Compare Label Studio Editions

    develop

    Label Studio is available in three distinct editions to suit different project needs:

    1. Label Studio Community Edition: The open-source version available to everyone.
    2. Starter Cloud: A paid cloud-hosted version.
    3. Enterprise: A paid version designed for advanced organizational needs.

    Choose your edition based on whether you require self-hosted open-source capabilities or managed cloud services with advanced enterprise features.

  7. Manage projects in Label Studio Community Edition

    develop

    In the Label Studio Community Edition, project management is primarily handled through project settings and the Data Manager.

    Key management capabilities include:

    • Configuring project settings: Adjusting specific configurations for an individual project.
    • Using the Data Manager: Navigating and managing labeling tasks through the built-in Data Manager interface.

    Note: Label Studio Enterprise provides advanced features not available in the Community Edition, such as workspace organization, member definitions, and workflow automation.

  8. Compare Label Studio Community, Starter Cloud, and Enterprise editions

    develop

    Label Studio is available in three editions, each catering to different needs regarding user management, data handling, and advanced workflows:

    Community Edition

    Best for individual developers or small teams starting with open-source labeling.

    • Includes: Data management view, multiple data formats, import/export (JSON, COCO, etc.), sync data, custom ML backends, and predictions from connected models.
    • Lacks: Role-based access control, advanced cloud storage (S3/Azure/GCP IAM), automated task assignment, and enterprise security (SSO/SOC2).

    Starter Cloud

    A managed cloud version with additional features for growing teams.

    • Includes everything in Community, plus: Role-based access control (Admin, Manager, Reviewer, Annotator), project membership controls, automated task assignment, simplified annotator interfaces, and a support portal.
    • Lacks: Advanced cloud storage integrations, bulk labeling, and full enterprise-grade security/SLA.

    Enterprise Edition

    Designed for large-scale, secure, and highly customized organizational labeling operations.

    • Includes everything in Starter Cloud, plus: Customizable permissions, advanced cloud storage (Databricks Unity Catalog, AWS S3 IAM, Azure Service Principal, GCP WIF), workspace organization, white labeling, plugins, programmable/embeddable React interfaces, active learning loops, and comprehensive analytics (dashboards, activity logs, annotator performance).
    • Security: Provides SSO (SAML/LDAP), SOC2 compliance, and 99.9% uptime SLA.

  10. Contribute updates to Label Studio documentation

    develop

    Documentation contributions can take several forms:

    • Small fix: Correcting typos or broken links.
    • Add a section: Expanding an existing topic (e.g., adding troubleshooting steps for an ML backend).
    • Refactor: Rewriting existing topics for better clarity (e.g., improving webhook event guidance).
    • Add new: Creating entirely new topics, such as new templates or ML tutorials. It is recommended to open an issue to discuss new topics with the team before implementation.

  11. Benefits of using the Label Studio Python SDK

    develop

    The Python SDK is designed to automate and streamline interactions with the Label Studio API. Key capabilities include:

    • Streamlined API Interactions: Uses user-friendly Python methods instead of raw HTTP requests.
    • Workflow Integration: Easily embeds Label Studio actions into existing data science workflows.
    • Automation: Automates repetitive tasks like project creation, task imports, and data exports.
    • Enhanced Data Preparation: Provides filters and custom configurations for efficient data management.
    • Asynchronous Operations: Supports asynchronous data operations to improve performance when handling large datasets.

  12. Integrate external storage with Label Studio

    develop

    Label Studio supports connecting to various cloud and external storage systems to sync data into projects (Source Storage) or export annotations (Target Storage).

    Supported Storage Providers:

    • Amazon S3: Community & Enterprise (includes IAM role support in Enterprise)
    • Google Cloud Storage: Community & Enterprise (includes WIF and GKE impersonation in Enterprise)
    • Microsoft Azure Blob Storage: Community & Enterprise (includes Service Principal in Enterprise)
    • Databricks Files (UC Volumes): Enterprise only
    • Redis database: Community & Enterprise
    • Local storage: Community & Enterprise

    Each storage connection is project-specific, and you can connect multiple buckets, containers, or directories to a single project.