Argilla Documentation

Argilla is an open-source data curation platform designed for Large Language Models (LLMs) and general NLP tasks. It supports the full MLOps cycle, including data labeling, model monitoring, and iterative data collection using both human and machine feedback.

Key capabilities include:

• Human-in-the-loop: Combining hand-labeling with active learning, zero-shot models, and weak supervision.
• Iterative Development: Enabling continuous data collection and model monitoring once models are in production.
• Library Compatibility: Seamless integration with major NLP libraries like Hugging Face transformers, spaCy, Stanford Stanza, and Flair.

To deploy Argilla on AWS, first configure an AWS profile:

aws configure --profile argilla
export AWS_PROFILE=argilla

Then, create a Docker machine on an EC2 instance:

docker-machine create --driver amazonec2 \
--amazonec2-root-size 60 \
--amazonec2-instance-type t2.large \
--amazonec2-open-port 80 \
--amazonec2-ami ami-0b541372 \
--amazonec2-region eu-west-1 \
argilla-aws

Note: The AMI ami-0b541372 is for eu-west regions (Ubuntu 16.04). You may need to select a different AMI based on your region.

To connect to the remote machine and launch Argilla:

eval $(docker-machine env argilla-aws)
docker-compose pull
docker-compose up -d

Access Argilla via http://<ASSIGNED_MACHINE_IP>:6900.

Argilla provides helper functions within the rg.markdown module to facilitate the use of Markdown in datasets. Specifically, these helpers assist with:

1. DataURL conversions: Converting media or other data into formats suitable for Markdown display.
2. Chat message visualizations: Formatting chat messages for proper rendering within the Argilla interface.

Refer to the src.argilla.markdown.media and src.argilla.markdown.chat modules for specific implementation details.

Install the Argilla Python SDK using pip.

pip install argilla

Deploy Argilla as a managed, scaling container on Google Cloud Run.

Note: Using the argilla/argilla-quickstart:latest image means using a pre-packaged storage layer, which prevents the use of Cloud Run's horizontal scaling features.

1. Authenticate: gcloud auth login
2. Deploy:

gcloud run deploy <deployment-name> \
--region <region> \
--image argilla/argilla-quickstart:latest \
--allow-unauthenticated \
--port 6900 \
--cpu 2 \
--memory 4Gi \
--max-instances 1 \
--min-instances 1

Retrieve the service URL using: gcloud run services describe <deployment-name> --region <region> --format 'value(status.url)'.

For developers working on the argilla-server backend:

• Environment Variables: By default, commands run with pdm run use variables from .env.dev. The pdm test command uses .env.test.
• Run Dev Server: Use pdm server-dev to migrate the database, create default users, and launch the server in one step.
• Run Tests: Use pdm test to run the full test suite (uses a SQLite database at ~/.argilla/argilla-test.db).
• Run Server: Use pdm server to run the uvicorn FastAPI server.
• Run Workers: Use pdm worker to start RQ background workers.

# Quick start for development
pdm server-dev

# Run tests
pdm test

To interact with the Argilla Server and UI programmatically, install the Python SDK using pip.

pip install argilla

To start the Argilla frontend in a local development environment, use the dev script:

npm run dev

The easiest way to deploy on Azure is using Azure Container Instances (ACI).

1. Authenticate: az login
2. Create a resource group: az group create --name <resource-group> --location <location>
3. Create the container:

az container create --resource-group <resource-group> --name <deployment-name> --image argilla/argilla-quickstart:latest --dns-name-label <dns-name> --ports 6900

Access Argilla at the FQDN provided by the az container show command on port 6900.

To generate a local build of the Argilla frontend, use the generate script:

npm run generate

To create a new user profile, instantiate the rg.User object with a username and password, then call the .create() method. This profile can be used to track feedback activity and manage server access.

user = rg.User(username="my_username", password="my_password")
user.create()