Retentioneering Documentation

repository·master·Indexed 21 days ago

https://github.com/retentioneering/retentioneering-tools

An open-source Python toolkit, MCP server, and agent skills for reproducible product and quantitative UX analytics on clickstream and event logs. Version 5.1.0 features a DuckDB-backed Eventstream for high-performance computations, immutable data processors for chaining operations, and interactive visualizations including transition graphs, funnels, and step matrices. It provides tools for behavioral segmentation, A/B tests, process mining, and Markov models, with 'headless twin' methods for extracting raw data for ML pipelines.

Tokens
57.9K
Snippets
164
Records
235
Agent score
74%

What's inside retentioneering

  1. What is an Eventstream?

    master

    An Eventstream is the central object in retentioneering. It wraps your event data and serves as the primary interface for all analysis tools, including widgets and data processors.

    Key concepts include:

    • Path: The unit of analysis (e.g., a user journey or a single session). The grain of a path is determined by the path_col you choose.
    • Step: A specific position within a path (e.g., the 1st event, 2nd event).
    • Segment: A way to split paths into groups (e.g., by country or device type) using segment columns. Segments can be static or dynamic (changing along a path).

    Unlike tools like Amplitude or Mixpanel which are user-centric, retentioneering is path-centric. This allows you to switch between user-grain and session-grain analysis easily by changing the path_col without reloading data.

  2. Understand Segments in Retentioneering

    master

    In Retentioneering, segmentation is the mechanism used to slice an eventstream into comparable groups. Unlike tools that define a single group of users, a segment in Retentioneering is a segment column that maps each event to a group (e.g., a country column splits the stream into US, DE, FR, etc.).

    Segments are categorized into two types:

    • Static Segments: The value is constant for the entire path (e.g., acquisition_channel, cluster_label, or the deepest funnel step reached). These answer "which paths behave differently?".
    • Dynamic Segments: The value can change along a single path (e.g., weekend vs weekday, or inside vs outside an incident window). These answer "which parts of a path behave differently?".

    Segment-aware tools like Segment Overview can split paths into within-segment fragments, allowing you to compare behavior within a specific time window against behavior outside of it for the same user.

    stream = Eventstream(df, schema={
        "segment_cols": ["country", "plan"],
    })
  3. Core Concept: Immutability in Retentioneering 5.x

    master

    In 5.x, every data processor is immutable. A processor never modifies the existing Eventstream in-place; instead, it returns a new Eventstream instance. You must capture the returned object to continue using the processed data.

    Incorrect (3.x style):

    stream.filter_events(drop={"event": ["checkout_bug"]})

    Correct (5.x style):

    stream = stream.filter_events(drop={"event": ["checkout_bug"]})
    stream = stream.filter_events(drop={"event": ["checkout_bug"]})
  4. How Retentioneering widgets are bundled and distributed

    master

    Starting with version 5.0, Retentioneering widgets have moved away from CDN-hosted JS bundles to a wheel-embedded model using anywidget.

    Key characteristics of this architecture:

    • No CDN dependency: There is no runtime download or CDN fallback. This ensures offline/air-gapped installs work and prevents version skew between Python and JS.
    • Embedded JS: The JS bundles (widget.js for ESM/live use and widget-static.js for static exports) are built from the js/ npm workspace and included directly in the Python wheel.
    • State Synchronization: Python ↔ JS state sync is handled via anywidget traitlets. The model keys used in JS (model.get/set) follow the same naming conventions as the Python API.
    • Error Handling: If the required JS bundle is missing from the installation, widgets/_esm.py will raise a FileNotFoundError with a hint to run make build.
  5. Differentiate between 'path' and 'segment' for analysis

    master

    When analyzing data in Retentioneering, it is critical to distinguish between the unit of analysis (path) and cross-cutting breakdowns (segments).

    The Path

    • A path is the unit of analysis (e.g., a user journey, a session, or a specific fragment).
    • The path is defined by the path_cols hierarchy.
    • While tools allow a path_col override to analyze at different grains (e.g., switching from user-level to session-level), the chosen column must be one of the columns defined in the path_cols hierarchy.
    • Using a column outside of path_cols as a path will raise an error.

    Segments

    • A segment is defined by a segment column (e.g., device_type, campaign_id).
    • Segments can be static (one value per path) or dynamic (the value changes along the path).
    • Best Practice: If you want to perform a cross-cutting breakdown (e.g., comparing how 'Mobile' users behave vs 'Desktop' users), do not try to use device_type as a path_col. Instead, use segment_cols or the diff functionality.
  6. Use Data Processors to transform Eventstreams

    master

    Data processors are methods available on the Eventstream object used to transform an Eventstream and return a new one. Processors can be chained together in a pipeline. Crucially, each processor returns a new Eventstream instance, meaning the original Eventstream is never modified (immutability).

    stream = (
        rete.datasets.load_ecom()
        .add_start_end_events()
        .filter_events(drop={"event": ["checkout_bug"]})
        .rename_events({"wishlist_add": "add_to_wishlist"})
    )
  7. How documentation is generated and maintained

    master

    Retentioneering uses a docstring-driven documentation pipeline where code docstrings serve as the single source of truth for reference material. This ensures consistency across the documentation website, Python's help() function, and LLM agents (via the MCP server).

    Key Components:

    • Reference Docs: Rendered from Eventstream docstrings using Jinja templates located in docs/templates/. The docstring provides the summary, parameters, and runnable examples, while templates handle the layout.
    • Conceptual Guides: Hand-written markdown files in docs/guide/*.md (e.g., Quick Start, Eventstream guide) are copied directly to the build.
    • Widget Demos: Live widget demos are generated by docs/scripts/generate_widget_demos.py. These use the <DemoWidget cmd={...}> tag in templates, which executes against a bundled dataset and exports via export_html() to ensure examples are always functional code.
    • Figures/Images: Hand-written SVGs are stored in docs/img/ and referenced in markdown using the path /docs-demos/img/<name>.svg. The system automatically renders markdown images as centered <figure> elements with the image title as the <figcaption>.
  8. Analyze specific sequences using Funnels

    master

    A Funnel is used when you only care about a specific sequence of milestones. It ignores all events that occur between the named steps.

    Usage

    Pass a list of event names to the steps argument. The funnel will calculate the conversion rate (the share of paths) that reaches each subsequent milestone in the specified order.

    Limitations

    • Blind to Intermediate Behavior: Funnels tell you that users dropped off between milestones, but they cannot tell you why or what they did instead. To understand the behavior between levels, you should use other path analysis representations like Step Matrices or Transition Graphs.
    # Analyze conversion through a specific sequence
    stream.funnel(steps=["home", "cart", "purchase"])
  9. Predict derived column names for metrics

    master

    When using path metrics, the resulting column names in your data follow a specific naming convention: <metric>_<args>. This is critical because widgets and report anchor links reference these columns by their derived names.

    Examples of naming patterns:

    • has_event_purchase (for has_event with argument purchase)
    • in_segment_<segment>_<value>_<mode> (for in_segment metrics)
    • time_from_<a>_to_<b> (for time_between metrics)
  10. Compare groups of paths using diff mode

    master

    Most path analysis questions are comparative (e.g., comparing an A/B test arm, a platform, or a time window). You can define these groups as a segment. When a widget is used with two segments, it enters diff mode, rendering the difference (group1 − group2) instead of the absolute numbers for a single group.

    Common use cases for diffing include:

    • Anomalous periods: Using a dynamic segment to split an eventstream into "inside the window" and "outside" to see which transitions degraded.
    • Funnel progression: Using add_segment(..., funnel_events=[...]) to label paths by the deepest funnel level completed, allowing you to compare users who "stalled" against those who "converted".
    • User lifecycle stages: Using split_sessions to number sessions and creating segments based on that number (e.g., "first session" vs "experienced user") to compare the same people at different stages of their lifetime.

    If you have many segment levels (e.g., dozens of countries), use the Segment Overview widget to view metrics in a heatmap across all levels before choosing a specific pair to diff.

    # Example concept: labeling paths by funnel depth to enable comparison
    # (Refer to recipes for specific implementation details)
    stream = stream.add_segment(..., funnel_events=[event1, event2, event3])