DOCSEARCHbyAlgolia
Submit Docs

Snowplow Documentation

website·Indexed 40 days ago

https://docs.snowplow.io/docs

Technical documentation for the Snowplow event collection platform, including Analytics SDKs for Go, .NET, Python, and Scala; configuration references for Enrich (Kinesis, Kafka, PubSub, NSQ); Iglu schema registry and server management; and data loaders for BigQuery, Databricks, and Elasticsearch.

Tokens
631.7K
Snippets
2.5K
Records
4K
Agent score
71%

What's inside Snowplow

  1. Overview of the Flink live shopper features architecture

    The live shopper features stack is designed for sub-second real-time personalization by converting raw behavioral events into metrics available for downstream services (chat bots, ML models, pricing engines).

    Component Stack:

    • Snowplow tracker: Captures clean, strictly defined events.
    • Kafka: Buffers and scales traffic.
    • Flink: Performs in-memory windowing and aggregation for sub-second output.
    • Redis: Serves the resulting metrics with microsecond read latency.

  2. Overview of Iglu Server

    Iglu Server is a RESTful schema registry used by self-hosted Snowplow pipelines to store, validate, and serve self-describing schemas. It allows users to publish and update schemas with validation at upload time, serve schemas via the Iglu Resolver, issue scoped API keys for vendor prefix access control, and receive webhook notifications via webhooks.schemaPublished when schemas change. Most users manage schemas through Igluctl rather than the REST API directly.

  3. Overview of the SQL Runner Mobile Data Model

    The Snowplow mobile data model aggregates events from iOS and Android trackers to create derived tables for screen views, sessions, and users. Unlike the web model, its aggregation is non-linear, allowing the sessions module to consume various event types (e.g., app installs, crashes) in addition to screen views. The model runs incrementally, processing only new or updated events. Note: This model is legacy and no longer under active development; for new projects, the unified digital package for dbt is recommended.

  4. Overview of Snowplow Client for GTM Server Side

    The Snowplow Client allows GTM Server Side containers to receive Snowplow events (either directly from a tracker or enriched from the pipeline). It populates common event data to ensure compatibility with standard GTM SS tags and provides additional rich Snowplow properties for specialized tags, such as Snowplow Authored Tags.

  5. Overview of the legacy Snowplow Mobile dbt package

    The legacy Snowplow Mobile dbt package transforms raw mobile event data from iOS and Android trackers into derived tables for screen views, sessions, and users. It is now superseded by the Unified Digital package. The package provides the following standard modules:

    • Base: De-duplicates data into snowplow_mobile_base_events_this_run.
    • Screen Views: Aggregates data to snowplow_mobile_screen_views.
    • Sessions: Aggregates data to snowplow_mobile_sessions.
    • Users: Aggregates data to snowplow_mobile_users.
    • User Mapping: Maps device_user_id and user_id in snowplow_mobile_user_mapping for session stitching.

  7. Overview of Snowplow Signals for real-time personalization

    Snowplow Signals is a real-time personalization engine built on the Snowplow behavioral data pipeline. It enables developers to compute, access, and act on both in-session stream data and historical user data in real time. This allows for the creation of dynamic user segments, content customization, and proactive interventions based on real-time behavioral attributes.

  8. Overview of manual failed event recovery

    Manual event recovery allows developers to fix failed events and replay them through the Snowplow pipeline. This is useful for correcting data that failed validation or processing due to specific, fixable errors. Supported recovery actions include:

    • Replacing values (e.g., correcting a typo in a schema name).
    • Removing values (e.g., stripping improperly encoded characters from a URL).
    • Casting JSON types (e.g., converting a string to an integer).

    Because storage may contain a mix of recoverable and unrecoverable data, recovery jobs use a configuration file to target only the specific subset of failed events that can be fixed.

  9. Overview of Snowplow Mini

    Snowplow Mini is a single-instance version of the Snowplow real-time stack packaged into a single image. It is designed for three primary use cases: allowing consumers (analysts/marketing teams) to understand Snowplow's data flow, providing developers an easy way to start with Snowplow, and enabling existing users to debug tracker updates.

    Version 0.26.0 includes the following components:

    • Snowplow Collector NSQ 3.7.0
    • Snowplow Enrich NSQ 6.11.0
    • Snowplow Elasticsearch Loader 2.1.3
    • Snowplow Iglu Server 0.14.0
    • Opensearch 3.6.0
    • Opensearch Dashboards 3.6.0
    • Postgresql 16.10
    • NSQ v1.3.0

  10. Overview of legacy SQL Runner components

    SQL Runner is a legacy tool used to execute SQL scripts against Snowplow data in warehouses (Snowflake, Redshift, BigQuery, and PostgreSQL). It is no longer under active development; for new projects, dbt packages are recommended. A SQL Runner data model consists of two primary components: SQL files (containing one or more SQL statements) and Playbooks (YAML files that organize these SQL files into execution steps).

  11. Overview of Snowplow Event Forwarding

    Event forwarders are managed apps deployed within a Snowplow cloud account that filter, transform, and send enriched events to third-party platforms in real-time. They use Snowbridge under the hood to read events from streams (Kinesis, Pub/Sub, or EventHub), apply JavaScript filters, transform data via field mapping expressions, and deliver them via HTTP API calls. End-to-end latency is typically on the order of seconds.

  12. Overview of Snowplow Signals

    Snowplow Signals is a real-time personalization engine that computes user attributes from behavioral data (via event streams or warehouse syncs) and serves them to applications via an API. It is used for personalizing content/pricing, providing context to AI agents, triggering automatic actions based on user criteria, and serving behavioral features to ML models for real-time scoring.