VisiData Documentation

repository·develop·Indexed 27 days ago

https://github.com/saulpw/visidata

A terminal-based interface for exploring, arranging, and manipulating tabular data in formats including CSV, JSON, Excel, and SQLite. Documentation covers installation via pip, pipx, and uv; user guides for data loading, navigation, and manipulation; and a developer API reference for building plugins, loaders, and custom sheets. Includes specialized guides for vdsql database interaction and the Galactic Conquest game server.

Tokens
58.4K
Snippets
110
Records
440
Agent score
93%

What's inside VisiData

  1. Overview of VisiData (vd)

    develop
    VisiData (vd) is a curses-based spreadsheet application designed for efficient exploration of large datasets (10GB+). It allows users to source data from various formats, browse, join, filter, and sort tables using minimal keystrokes. It is highly extensible via Python3 adapters and custom sheets.
  2. Overview of vgit features

    develop

    vgit provides a terminal user interface for several git operations, including:

    • Viewing, stashing, staging, unstaging, and committing diff hunks.
    • Viewing the git log (history).
    • Branch and remotes management.
    • Branch merges and commit cherry-picks.
    • Popping, applying, and dropping stashed diffs.
    • Setting local and global git config options.
  3. Explore VisiData 'How to' recipes

    develop

    The documentation provides specific recipes for common data tasks, including:

    • Data Loading: Specifying source files, loading pandas-supported sources, opening R data frames, and converting filetypes.
    • Navigation: Scrolling, searching, and moving between sheets.
    • Row Operations: Filtering, selecting (null/non-null), moving, copying, removing, and sorting rows.
    • Column Operations: Hiding/unhiding, specifying types, splitting, expanding nested data, and creating derivative columns.
    • Data Analysis: Grouping, pivot tables, frequency charts, and descriptive statistics.
    • Data Manipulation: Joining datasets, appending datasets, and editing cells.
    • Visualization: Drawing and interacting with graphs.
    • Session Management: Saving and replaying sessions.
    • Extensibility: Customizing configurations and developing plugins.
  4. Understand module loading and conflicts

    develop

    VisiData loads all installed plugins and internal modules by default. Note the following behaviors:

    • Conflict Resolution: Some modules are incompatible (e.g., the traditional visidata sqlite loader vs. the vdsql sqlite loader). The last module imported will have its functions registered for the specific filetype or command.
    • Module Precedence: If you need to use an older or specific module that is being overridden by a more advanced one, the overriding module must be prevented from loading.
  5. Understand VisiData Terminal Graphics Architecture

    develop

    VisiData uses braille Unicode characters to display low-resolution terminal graphics. The graphics system is built on a hierarchy of classes that transition from raw terminal pixels to high-level, interactive graphs:

    • Plotter: A pixel-addressable surface covering the terminal. It uses exact integer (x, y) coordinates.
    • Canvas: A zoomable/scrollable virtual surface on top of a Plotter that uses arbitrary units (floats) for coordinates.
    • InvertedCanvas: A Canvas with an inverted Y-axis (minimum coordinates in the lower-left).
    • Graph: An InvertedCanvas that includes axis labels, legends, and gridlines.

    Data flows from a Sheet (the source) through a Graph.reload() call, which uses Canvas.polyline() to define what to render, eventually being drawn by the Plotter using braille characters.

  6. Overview of VisiData User Guides

    develop

    VisiData provides extensive documentation for various data manipulation tasks:

    Data Loading & Input

    • Loading files: Specifying source files and piping data in (loading.md).
    • Pipes: Using VisiData in stdin/stdout pipelines (pipes.md).
    • Formats: Supported file formats, loaders, savers, and dependencies (formats.md).
    • Internal Formats: Understanding .vd, .vdj, .vdx (command logs) and .vds (sheet state) (internal_formats.md).
    • Navigation: Movement commands, vim-style keys, scrolling, and searching (navigate.md, move.md).
    • Menu & Mouse: Toplevel menubar navigation (menu.md) and mouse interaction (mouse.md).
    • Split Screen: Viewing two sheets simultaneously (split.md).
    • Directory Sheet: Browsing, editing, and previewing files via DirSheet (dirsheet.md).

    Data Manipulation

    • Columns: Pinning, moving, hiding/unhiding, and resizing (columns.md).
    • Rows: Selecting, filtering, and toggling rows (rows.md).
    • Editing: Cell editing, bulk updates, regex substitution, and expressions (edit.md).
    • CRUD: Creating blank sheets, rows, and columns (crud.md).
    • Grouping & Stats: Grouping data, aggregators, and descriptive statistics (group.md).
    • Frequency & Pivoting: Frequency tables, binning (discrete/numeric), and pivot sheets (freq.md).
    • Joining: Combining datasets via the Sheets Sheet (join.md).

    Customization & Automation

    • Configuration: Using .visidatarc, command bindings, and the Options Sheet (customize.md).
    • Macros: Recording and replaying command sequences (macros.md).
    • Sessions: Saving and replaying sessions (save-restore.md).
    • Themes: Color themes and interface customization (colors.md).
  7. Perform a multi-column sort

    develop

    To sort by multiple columns in a specific priority order, start with the most important column and then add secondary columns using the following commands:

    • sort_asc_add: Add current column to sort order (ascending)
    • sort_desc_add: Add current column to sort order (descending)
    • sort_keys_asc_add: Add key columns to sort order (ascending)
    • sort_keys_desc_add: Add key columns to sort order (descending)
  8. Define and use custom options in VisiData

    develop

    You can make commands or behaviors configurable by defining options using vd.option. This allows users to modify behavior via the command line or a .visidatarc file.

    Defining an option

    Use vd.option(name, default, description) to register an option.

    Overriding options

    • Command Line: Convert underscores to hyphens. For an option disp_hello, use --disp-hello="value".
    • Persistent Configuration: Add the option to your .visidatarc file using the options. prefix.

    Naming Conventions

    • Plugins: Start option names with a short module abbreviation (e.g., mod_).
    • Theme/Display: Use disp_ for displayed strings and color_ for color options.
    • Format: Use underscores to separate words, keep names under 20 characters, and use a maximum of 3 words.
  9. Create Frequency Tables to group data

    develop

    Frequency Tables allow you to group rows into bins based on column values. This is the VisiData equivalent of a GROUP BY operation. You can use freq_col to create a frequency table for a single column or freq_keys to group by multiple keys simultaneously.

    To bin numeric rows into ranges rather than discrete values, use the --numeric-binning option.

  10. Explore underlying data from a Frequency Table

    develop

    Once you have a Frequency Table, you can inspect the specific rows that make up a group:

    • Use open_row to view the data.
    • Use dive_selected to drill down into a group.
    • Selecting a group in the Frequency Table will select all of its underlying rows in the original source sheet.