Perspective
repository·master·Indexed 27 days ago
https://github.com/perspective-dev/perspectiveA high-performance data manipulation and visualization engine designed for large and streaming datasets. It features a query engine compiled for WebAssembly, Python, and Rust, and provides a CLI for format conversion and local server hosting. The ecosystem includes the perspective-python bindings, a JupyterLab extension, a perspective-viewer Web Component, and an esbuild plugin for ESM builds.
What's inside Perspective
- Perspective is an interactive analytics and data visualization component designed for large and streaming datasets. It allows developers to build user-configurable reports, dashboards, notebooks, and applications using a high-performance query engine. The engine is compiled for WebAssembly, Python, and Rust, supporting various data sources and visualization types.
Overview of perspective-python modules
masterThe
perspectivemodule provides several key components for data engine management, visualization, and web integration:Server: Constructor for a new instance of the Perspective data engine.perspective.widget: ExportsPerspectiveWidget, a JupyterLab widget for interactive visualization in notebook cells.perspective.handlers: Exports web framework handlers to interface with aperspective-clientin JavaScript:
Use perspective-python for Python bindings
masterTheperspective-pythoncrate provides the official Python bindings for the Perspective engine, allowing you to use Perspective's high-performance data manipulation capabilities within Python environments.Explore <perspective-viewer> UI features
masterThe viewer includes an interactive side panel with the following capabilities:
- Column list: Drag and drop columns to configure
group_by,split_by,sort, andfilterfields. - New Column button: Opens an expression editor for creating computed columns.
- Plugin selector: Switch between visualization plugins (e.g., Datagrid, X/Y Line, X/Y Scatter, Treemap, Sunburst, and Heatmap).
- Theme selector: Toggle between available themes.
- Export: Download the current view as CSV or Arrow.
- Copy: Copy the current view to the clipboard.
- Reset: Restore the viewer to its default configuration.
- Column list: Drag and drop columns to configure
Understand Perspective deployment modes
masterPerspective supports three primary architectural modes:
- Client-only: The engine runs entirely in the browser via WASM. Ideal for small to medium datasets.
- Client/Server (replicated): Data is hosted on a server and replicated to the client. The client holds a full copy and performs queries locally.
- Server-only: All queries execute on the server; the client only renders the results. Best for very large datasets.
Understand Perspective Virtual Servers
masterA Virtual Server allows Perspective to query external data sources (such as DuckDB or ClickHouse) without loading the entire dataset into Perspective's built-in data engine. Instead of duplicating data, Perspective translates its operations (group by, sort, filter, etc.) into queries the external source can execute natively, transferring only the data needed for the current view.
This is particularly useful when:
- The dataset exceeds browser memory or a single process capacity.
- Data already resides in a database and you want to avoid duplication.
- You want to leverage a database's native query optimizations.
- You are using a WASM-based data source in the browser (e.g.,
@duckdb/duckdb-wasm).
Understand the Perspective Table concept
masterA
Tableis Perspective's columnar data frame, similar to a PandasDataFrameor Apache Arrow. It supports appending data, in-place updates, and removal by index, while providing update notifications.Key characteristics include:
- Strong Typing: Each column has a unique name and a consistent data type.
- Uniformity: All columns in a
Tablemust have the same number of rows. Missing values are represented by nulls. - Immutable Schema: Once a
Tableis created, its schema is immutable. You cannot change column names, data types, or add/delete columns. To work with a subset of columns, you must use aView.
Use the perspective-viewer Web Component
masterTheperspective-viewercrate provides the<perspective-viewer>Web Component. It is accessible via JavaScript bindings that are generated fromwasm_bindgen.Use Perspective Rust bindings
masterTheperspectivecrate provides Rust bindings to the JavaScript Perspective API, allowing you to interface with Perspective's core functionality from a Rust environment.Understand Perspective JavaScript module structure
masterPerspective is modular. You can choose specific packages based on your requirements:
@perspective-dev/client: The core data engine library (ES6 and Node.js). Provides WebAssembly, WebWorker (browser), and Process (Node.js) runtimes.@perspective-dev/viewer: A Web Component visualization widget. It includes@perspective-dev/clientas a dependency. By default, it only implements a trivial debug renderer that prints theview()as CSV.@perspective-dev/viewer-datagrid: A high-performance HTML<table>based data-grid component.@perspective-dev/viewer-charts: A set of WebGL-based charting components.
Plugin Registration: If you import plugin modules (like
viewer-chartsorviewer-datagrid) after@perspective-dev/viewer, they will automatically register themselves. Their renderers will then appear in theplugindropdown within the<perspective-viewer>UI.Choose a Perspective data architecture design
masterPerspective supports three primary architectural patterns for binding and synchronizing data. Developers can choose one or a mix of these designs depending on their application requirements:
- Client (WebAssembly): Data is managed entirely within the client runtime using WebAssembly.
- Server (Python/Node): Data is managed on the server side (e.g., using Python or Node.js).
- Client/Server Replicated: Data is synchronized between the client and server. This design uses Apache Arrow serialization to efficiently duplicate and synchronize tables across different runtimes.
While examples often use Python and JavaScript, these principles apply to any supported
Client/Servercombination.Use `perspective-python` for data processing
masterTheperspective-pythonlibrary provides a Python interface to the Perspective C++ data engine. It supports loading data from NumPy, Pandas, and Apache Arrow. It is designed for high-performance data manipulation and can be used in local environments or integrated into web servers to serve data to web-based<perspective-viewer>components.