VisiData Documentation
repository·develop·Indexed 27 days ago
https://github.com/saulpw/visidataA terminal-based interface for exploring, arranging, and manipulating tabular data in formats including CSV, JSON, Excel, and SQLite. Documentation covers installation via pip, pipx, and uv; user guides for data loading, navigation, and manipulation; and a developer API reference for building plugins, loaders, and custom sheets. Includes specialized guides for vdsql database interaction and the Galactic Conquest game server.
What's inside VisiData
- VisiData (vd) is a curses-based spreadsheet application designed for efficient exploration of large datasets (10GB+). It allows users to source data from various formats, browse, join, filter, and sort tables using minimal keystrokes. It is highly extensible via Python3 adapters and custom sheets.
Overview of vgit features
developvgit provides a terminal user interface for several git operations, including:
- Viewing, stashing, staging, unstaging, and committing diff hunks.
- Viewing the git log (history).
- Branch and remotes management.
- Branch merges and commit cherry-picks.
- Popping, applying, and dropping stashed diffs.
- Setting local and global git config options.
Explore VisiData 'How to' recipes
developThe documentation provides specific recipes for common data tasks, including:
- Data Loading: Specifying source files, loading pandas-supported sources, opening R data frames, and converting filetypes.
- Navigation: Scrolling, searching, and moving between sheets.
- Row Operations: Filtering, selecting (null/non-null), moving, copying, removing, and sorting rows.
- Column Operations: Hiding/unhiding, specifying types, splitting, expanding nested data, and creating derivative columns.
- Data Analysis: Grouping, pivot tables, frequency charts, and descriptive statistics.
- Data Manipulation: Joining datasets, appending datasets, and editing cells.
- Visualization: Drawing and interacting with graphs.
- Session Management: Saving and replaying sessions.
- Extensibility: Customizing configurations and developing plugins.
Understand module loading and conflicts
developVisiData loads all installed plugins and internal modules by default. Note the following behaviors:
- Conflict Resolution: Some modules are incompatible (e.g., the traditional
visidatasqlite loader vs. thevdsqlsqlite loader). The last module imported will have its functions registered for the specific filetype or command. - Module Precedence: If you need to use an older or specific module that is being overridden by a more advanced one, the overriding module must be prevented from loading.
- Conflict Resolution: Some modules are incompatible (e.g., the traditional
Understand VisiData Terminal Graphics Architecture
developVisiData uses braille Unicode characters to display low-resolution terminal graphics. The graphics system is built on a hierarchy of classes that transition from raw terminal pixels to high-level, interactive graphs:
Plotter: A pixel-addressable surface covering the terminal. It uses exact integer (x, y) coordinates.Canvas: A zoomable/scrollable virtual surface on top of aPlotterthat uses arbitrary units (floats) for coordinates.InvertedCanvas: ACanvaswith an inverted Y-axis (minimum coordinates in the lower-left).Graph: AnInvertedCanvasthat includes axis labels, legends, and gridlines.
Data flows from a
Sheet(the source) through aGraph.reload()call, which usesCanvas.polyline()to define what to render, eventually being drawn by thePlotterusing braille characters.Overview of VisiData User Guides
developVisiData provides extensive documentation for various data manipulation tasks:
Data Loading & Input
- Loading files: Specifying source files and piping data in (
loading.md). - Pipes: Using VisiData in
stdin/stdoutpipelines (pipes.md). - Formats: Supported file formats, loaders, savers, and dependencies (
formats.md). - Internal Formats: Understanding
.vd,.vdj,.vdx(command logs) and.vds(sheet state) (internal_formats.md).
Navigation & Interface
- Navigation: Movement commands, vim-style keys, scrolling, and searching (
navigate.md,move.md). - Menu & Mouse: Toplevel menubar navigation (
menu.md) and mouse interaction (mouse.md). - Split Screen: Viewing two sheets simultaneously (
split.md). - Directory Sheet: Browsing, editing, and previewing files via
DirSheet(dirsheet.md).
Data Manipulation
- Columns: Pinning, moving, hiding/unhiding, and resizing (
columns.md). - Rows: Selecting, filtering, and toggling rows (
rows.md). - Editing: Cell editing, bulk updates, regex substitution, and expressions (
edit.md). - CRUD: Creating blank sheets, rows, and columns (
crud.md). - Grouping & Stats: Grouping data, aggregators, and descriptive statistics (
group.md). - Frequency & Pivoting: Frequency tables, binning (discrete/numeric), and pivot sheets (
freq.md). - Joining: Combining datasets via the Sheets Sheet (
join.md).
Customization & Automation
- Configuration: Using
.visidatarc, command bindings, and the Options Sheet (customize.md). - Macros: Recording and replaying command sequences (
macros.md). - Sessions: Saving and replaying sessions (
save-restore.md). - Themes: Color themes and interface customization (
colors.md).
- Loading files: Specifying source files and piping data in (
Perform a multi-column sort
developTo sort by multiple columns in a specific priority order, start with the most important column and then add secondary columns using the following commands:
sort_asc_add: Add current column to sort order (ascending)sort_desc_add: Add current column to sort order (descending)sort_keys_asc_add: Add key columns to sort order (ascending)sort_keys_desc_add: Add key columns to sort order (descending)
Perform git diffs with vgit
developThe
vgit difffunctionality provides several ways to inspect changes:- View git diff hunks.
- Show diffs between committed, staged, and unstaged changes.
- Add or remove hunks in staged/unstaged areas.
- Compare the diff between two specific commits.
Define and use custom options in VisiData
developYou can make commands or behaviors configurable by defining options using
vd.option. This allows users to modify behavior via the command line or a.visidatarcfile.Defining an option
Use
vd.option(name, default, description)to register an option.Overriding options
- Command Line: Convert underscores to hyphens. For an option
disp_hello, use--disp-hello="value". - Persistent Configuration: Add the option to your
.visidatarcfile using theoptions.prefix.
Naming Conventions
- Plugins: Start option names with a short module abbreviation (e.g.,
mod_). - Theme/Display: Use
disp_for displayed strings andcolor_for color options. - Format: Use underscores to separate words, keep names under 20 characters, and use a maximum of 3 words.
- Command Line: Convert underscores to hyphens. For an option
Create Frequency Tables to group data
developFrequency Tables allow you to group rows into bins based on column values. This is the VisiData equivalent of a
GROUP BYoperation. You can usefreq_colto create a frequency table for a single column orfreq_keysto group by multiple keys simultaneously.To bin numeric rows into ranges rather than discrete values, use the
--numeric-binningoption.Explore underlying data from a Frequency Table
developOnce you have a Frequency Table, you can inspect the specific rows that make up a group:
- Use
open_rowto view the data. - Use
dive_selectedto drill down into a group. - Selecting a group in the Frequency Table will select all of its underlying rows in the original source sheet.
- Use
Modify the split pane view
developTo navigate or rearrange your split panes:
- Swap the focus/position of the panes:
{help.commands.splitwin_swap} - Swap the actual pane content/view:
{help.commands.splitwin_swap_pane} - Mouse Navigation: You can click on an inactive pane with your mouse to immediately jump focus to it.
- Swap the focus/position of the panes: