Perspective

repository·master·Indexed 27 days ago

https://github.com/perspective-dev/perspective

A high-performance data manipulation and visualization engine designed for large and streaming datasets. It features a query engine compiled for WebAssembly, Python, and Rust, and provides a CLI for format conversion and local server hosting. The ecosystem includes the perspective-python bindings, a JupyterLab extension, a perspective-viewer Web Component, and an esbuild plugin for ESM builds.

Tokens
49.4K
Snippets
156
Records
290
Agent score
91%

What's inside Perspective

  1. Overview of Perspective

    master
    Perspective is an interactive analytics and data visualization component designed for large and streaming datasets. It allows developers to build user-configurable reports, dashboards, notebooks, and applications using a high-performance query engine. The engine is compiled for WebAssembly, Python, and Rust, supporting various data sources and visualization types.
  2. Overview of perspective-python modules

    master

    The perspective module provides several key components for data engine management, visualization, and web integration:

    • Server: Constructor for a new instance of the Perspective data engine.
    • perspective.widget: Exports PerspectiveWidget, a JupyterLab widget for interactive visualization in notebook cells.
    • perspective.handlers: Exports web framework handlers to interface with a perspective-client in JavaScript:
      • perspective.handlers.tornado.PerspectiveTornadoHandler for Tornado
      • perspective.handlers.starlette.PerspectiveStarletteHandler for Starlette and FastAPI
      • perspective.handlers.aiohttp.PerspectiveAIOHTTPHandler for AIOHTTP
  3. Explore <perspective-viewer> UI features

    master

    The viewer includes an interactive side panel with the following capabilities:

    • Column list: Drag and drop columns to configure group_by, split_by, sort, and filter fields.
    • New Column button: Opens an expression editor for creating computed columns.
    • Plugin selector: Switch between visualization plugins (e.g., Datagrid, X/Y Line, X/Y Scatter, Treemap, Sunburst, and Heatmap).
    • Theme selector: Toggle between available themes.
    • Export: Download the current view as CSV or Arrow.
    • Copy: Copy the current view to the clipboard.
    • Reset: Restore the viewer to its default configuration.
  4. Understand Perspective deployment modes

    master

    Perspective supports three primary architectural modes:

    • Client-only: The engine runs entirely in the browser via WASM. Ideal for small to medium datasets.
    • Client/Server (replicated): Data is hosted on a server and replicated to the client. The client holds a full copy and performs queries locally.
    • Server-only: All queries execute on the server; the client only renders the results. Best for very large datasets.
  5. Understand Perspective Virtual Servers

    master

    A Virtual Server allows Perspective to query external data sources (such as DuckDB or ClickHouse) without loading the entire dataset into Perspective's built-in data engine. Instead of duplicating data, Perspective translates its operations (group by, sort, filter, etc.) into queries the external source can execute natively, transferring only the data needed for the current view.

    This is particularly useful when:

    • The dataset exceeds browser memory or a single process capacity.
    • Data already resides in a database and you want to avoid duplication.
    • You want to leverage a database's native query optimizations.
    • You are using a WASM-based data source in the browser (e.g., @duckdb/duckdb-wasm).
  6. Understand the Perspective Table concept

    master

    A Table is Perspective's columnar data frame, similar to a Pandas DataFrame or Apache Arrow. It supports appending data, in-place updates, and removal by index, while providing update notifications.

    Key characteristics include:

    • Strong Typing: Each column has a unique name and a consistent data type.
    • Uniformity: All columns in a Table must have the same number of rows. Missing values are represented by nulls.
    • Immutable Schema: Once a Table is created, its schema is immutable. You cannot change column names, data types, or add/delete columns. To work with a subset of columns, you must use a View.
  7. Understand Perspective JavaScript module structure

    master

    Perspective is modular. You can choose specific packages based on your requirements:

    • @perspective-dev/client: The core data engine library (ES6 and Node.js). Provides WebAssembly, WebWorker (browser), and Process (Node.js) runtimes.
    • @perspective-dev/viewer: A Web Component visualization widget. It includes @perspective-dev/client as a dependency. By default, it only implements a trivial debug renderer that prints the view() as CSV.
    • @perspective-dev/viewer-datagrid: A high-performance HTML <table> based data-grid component.
    • @perspective-dev/viewer-charts: A set of WebGL-based charting components.

    Plugin Registration: If you import plugin modules (like viewer-charts or viewer-datagrid) after @perspective-dev/viewer, they will automatically register themselves. Their renderers will then appear in the plugin dropdown within the <perspective-viewer> UI.

  8. Choose a Perspective data architecture design

    master

    Perspective supports three primary architectural patterns for binding and synchronizing data. Developers can choose one or a mix of these designs depending on their application requirements:

    1. Client (WebAssembly): Data is managed entirely within the client runtime using WebAssembly.
    2. Server (Python/Node): Data is managed on the server side (e.g., using Python or Node.js).
    3. Client/Server Replicated: Data is synchronized between the client and server. This design uses Apache Arrow serialization to efficiently duplicate and synchronize tables across different runtimes.

    While examples often use Python and JavaScript, these principles apply to any supported Client/Server combination.

  9. Use `perspective-python` for data processing

    master
    The perspective-python library provides a Python interface to the Perspective C++ data engine. It supports loading data from NumPy, Pandas, and Apache Arrow. It is designed for high-performance data manipulation and can be used in local environments or integrated into web servers to serve data to web-based <perspective-viewer> components.