DOCSEARCHbyAlgolia
Submit Docs

Self-Hosted Sentry

repository·master·Indexed 27 days ago

https://github.com/getsentry/self-hosted

A feature-complete version of Sentry packaged for low-volume deployments and proofs-of-concept. This repository provides nightly builds, installation scripts, and configuration for services including Snuba, Postgres, Pgbouncer, Redis, Memcached, and Clickhouse. It includes a patch system for modifying configuration files and support for remote development via Google Cloud Workstations and Artifact Registry, manageable through the sentry CLI.

Tokens
2.8K
Snippets
9
Records
18
Agent score
94%

What's inside sentry-self-hosted

  1. Overview of Self-Hosted Sentry

    master
    Self-Hosted Sentry provides a feature-complete version of Sentry packaged for low-volume deployments and proofs-of-concept. For detailed installation instructions, configuration, and management, refer to the official documentation.

  2. Create a new patch for optional modifications

    master

    To create a patch for a configuration file, follow these steps:

    1. Copy the original file to a temporary file (e.g., docker-compose.yml becomes docker-compose.my-feature.yml).
    2. Apply your desired changes to the temporary file.
    3. Generate a unified diff patch using the diff -Naru command.
    4. Organize the patch by creating a new directory under optional-modifications/patches/[patch-name] and moving the .patch files there.

    Template command: diff -Naru [original file] [patched file] > [destination file].patch

  3. Configure Google Cloud for Remote Self-Hosted Development

    master

    To enable remote development, you must set up two Google Cloud services: Google Cloud Workstations (to run virtual machines) and Artifact Registry (to store base images).

    1. Create an Artifact Registry

    1. Navigate to the Google Cloud Artifact Registry console.
    2. Create a new repository.
    3. Recommended name: sentry-workstation-us.
    4. Recommended region: us-west1.

    2. Set up Cloud Workstations

    1. Navigate to the Cloud Workstations control panel.
    2. Create a Cluster: Create one cluster per region (e.g., us-west in us-west1).
    3. Create a Configuration: Within the cluster, create a configuration with the following settings:
      • Name: Use the convention [INSTALL_KIND]-[SIZE]-[CLUSTER_NAME] (e.g., postinstall-standard-us-west).
      • Install Mode: Choose preinstall (if ./install.sh has not run) or postinstall (if it has).
      • Resource Allocation: Choose small, standard, or large (e.g., E2 instances are sufficient for most work).
      • Image: Select Custom container image and choose your uploaded Docker image.
      • Disk: Select Create a new empty persistent disk (10GB is recommended).
      • IAM: Set appropriate policies to allow user access.

  4. Apply optional modifications using patches

    master

    Optional modifications are implemented via a patch system that allows you to modify existing Sentry configuration files (like .env, config.example.yml, sentry.conf.example.py, or docker-compose.yml) to suit your infrastructure.

    To apply patches:

    1. Run the patch commands from the root directory of the repository.
    2. Use the -p0 flag with the patch command to ensure files are applied to the correct absolute paths.
    3. Crucial: After applying patches, review the modified files for any required manual steps (such as providing credentials or TLS certificates) before running the install.sh script.
    4. Run ./install.sh to finalize the installation with the new configurations.
    WARNING

    This system is experimental. Use it at your own risk.

  5. Build and upload workstation Docker images

    master

    Workstation configurations require Docker images located in the Artifact Registry. You can build either preinstall or postinstall images using the Dockerfiles provided in the repository.

    First, authenticate your local environment with Google Cloud and Docker:

    export GCP_PROJECT_ID=my-gcp-project
gcloud auth application-default login
gcloud config set project $GCP_PROJECT_ID
gcloud auth print-access-token | docker login -u oauth2accesstoken --password-stdin https://us-docker.pkg.dev

    Then, define your environment variables and push the image:

    export GROUP=sentry-workstation
export REGION=us
export PHASE=pre # Use 'pre' for preinstall, 'post' for postinstall
export REPO=${GROUP}-${REGION}
export IMAGE_TAG=${GROUP}/${PHASE}install:latest
export IMAGE_URL=us-docker.pkg.dev/${GCP_PROJECT_ID}/${REPO}/${GROUP}/${PHASE}install:latest

# Build the image
docker build -t ${IMAGE_TAG} -f ./${PHASE}install/Dockerfile .

# Tag and push to Artifact Registry
docker tag ${IMAGE_TAG} ${IMAGE_URL}
docker push ${IMAGE_URL}
    export GCP_PROJECT_ID=my-gcp-project
gcloud auth application-default login
gcloud config set project $GCP_PROJECT_ID
gcloud auth print-access-token | docker login -u oauth2accesstoken --password-stdin https://us-docker.pkg.dev

export GROUP=sentry-workstation
export REGION=us
export PHASE=pre
export REPO=${GROUP}-${REGION}
export IMAGE_TAG=${GROUP}/${PHASE}install:latest
export IMAGE_URL=us-docker.pkg.dev/${GCP_PROJECT_ID}/${REPO}/${GROUP}/${PHASE}install:latest

docker build -t ${IMAGE_TAG} -f ./${PHASE}install/Dockerfile .
docker tag ${IMAGE_TAG} ${IMAGE_URL}
docker push ${IMAGE_URL}

  7. Configure Redis and Memcached

    master

    The setup uses redis:6.2.20-alpine and memcached:1.6.45-alpine.

    Redis Configuration:

    • Uses a bind mount for ./redis.conf at /usr/local/etc/redis/redis.conf.
    • ulimits: nofile soft/hard limits are set to 10032.

    Memcached Configuration:

    • The command uses -I followed by ${SENTRY_MAX_EXTERNAL_SOURCEMAP_SIZE:-1M} to set the maximum external sourcemap size.

  8. Configure Sentry event retention via cleanup

    master

    The sentry-cleanup service runs a cron job that executes sentry cleanup --days $SENTRY_EVENT_RETENTION_DAYS. The number of days to retain events is determined by the SENTRY_EVENT_RETENTION_DAYS environment variable.

    command: '"0 0 * * * gosu sentry sentry cleanup --days $SENTRY_EVENT_RETENTION_DAYS"'

  9. Configure Sentry service environment variables

    master

    The sentry service uses several environment variables for configuration. Many of these can be passed through from the host system or a .env file by leaving them empty in the docker-compose.yml.

    Key variables include:

    • DOCKER_PLATFORM: Sets the platform for the build (e.g., for Apple silicon Macs).
    • SENTRY_STATSD_ADDR: Address for StatsD monitoring.
    • SENTRY_EVENT_RETENTION_DAYS: Number of days to retain events.
    • SENTRY_KAFKA_MAX_POLL_INTERVAL_MS: Kafka polling interval.
    • SENTRY_MAIL_HOST: The host for the SMTP service.
    • SENTRY_MAX_EXTERNAL_SOURCEMAP_SIZE: Maximum size for external sourcemaps.
    • SENTRY_SYSTEM_SECRET_KEY: The system secret key.
    • LAUNCHPAD_RPC_SHARED_SECRET: Shared secret for Launchpad RPC.

    To support custom CA certificates, the following variables are hardcoded to use the system CA bundle:

    • DEFAULT_CA_BUNDLE (used by botocore)
    • REQUESTS_CA_BUNDLE (used by requests)
    • GRPC_DEFAULT_SSL_ROOTS_FILE_PATH_ENV_VAR (used by grpc/google modules)

  10. Configure Postgres and Pgbouncer

    master

    The setup uses postgres:14.23-bookworm and edoburu/pgbouncer:v1.25.2-p0.

    Postgres Configuration:

    • POSTGRES_USER: The database user (defaults to postgres).
    • POSTGRES_HOST_AUTH_METHOD: Set to trust.

    Pgbouncer Configuration:

    • DB_USER: Set to ${POSTGRES_USER:-postgres}.
    • DB_HOST: Set to postgres.
    • DB_NAME: Set to postgres.
    • AUTH_TYPE: Set to trust.
    • POOL_MODE: Set to transaction.
    • ADMIN_USERS: Set to postgres,sentry.
    • MAX_CLIENT_CONN: Set to 10000.

  11. Configure Uptime Checker settings

    master

    The uptime-checker service can be customized via the following environment variables:

    • UPTIME_CHECKER_RESULTS_KAFKA_CLUSTER: The Kafka cluster to report results to (default: kafka:9092).
    • UPTIME_CHECKER_REDIS_HOST: The Redis host (default: redis://redis:6379).
    • UPTIME_CHECKER_ALLOW_INTERNAL_IPS: Set to true to allow uptime checks against private IP addresses (default: false).
    • UPTIME_CHECKER_FAILURE_RETRIES: Number of retries before reporting a failure (default: 1).
    • UPTIME_CHECKER_STATSD_ADDR: StatsD address (default: ${STATSD_ADDR:-127.0.0.1:8125}).
    environment:
  UPTIME_CHECKER_RESULTS_KAFKA_CLUSTER: kafka:9092
  UPTIME_CHECKER_REDIS_HOST: redis://redis:6379
  UPTIME_CHECKER_ALLOW_INTERNAL_IPS: "false"
  UPTIME_CHECKER_FAILURE_RETRIES: "1"
  UPTIME_CHECKER_STATSD_ADDR: ${STATSD_ADDR:-127.0.0.1:8125}