data-table-filters

The project's architectural goal is to enable the deployment of data tables using configuration files. This approach relies on a system that includes:

• A simple API endpoint to receive configurations.
• A CLI command to deploy configurations via the API.
• A TypeScript SDK for configuration validation (or direct support for yaml).
• A versioning strategy to manage breaking changes (e.g., using paths like logs.run/v1, logs.run/eab5f3, or logs.run/2025-03).

Configuration files are expected to include definitions for filter fields and components.

The data layer uses a three-pass filtering strategy to prevent UI issues like slider bounds collapsing when a user adjusts them. This strategy ensures that facet counts and slider ranges remain stable.

1. Pass 1 (Date only): Apply only the date range filter. This establishes the base time window for all facet computations.
2. Pass 2 (Date + non-slider filters): Add checkbox, input, and other non-slider filters. Use these conditions to compute the min/max bounds for sliders. Because the slider values themselves are excluded from this pass, moving a slider won't shrink its own available range.
3. Pass 3 (All filters): Apply all filters, including the current slider values. This pass is used for the final data query, row counts, and checkbox facets.

When using generateSheetFields(), the type of each field is automatically derived from its filter configuration. This type determines if the sheet row displays a filter dropdown:

To get the dropdown in a manual sheetFields definition, ensure the type matches the corresponding filterFields entry.

Do not use OFFSET for pagination in datasets where new items are frequently prepended (live updates). When new items are added to the top of the list, existing offsets shift, which causes duplicate items to appear in subsequent 'load more' queries. Use cursor-based pagination (e.g., using timestamps) instead.

// AVOID THIS PATTERN for live data
const data = await sql`
  SELECT * FROM table 
  ORDER BY timestamp DESC 
  LIMIT ${limit} 
  OFFSET ${offset}
`;

Fields configured with .commandDisabled() are hidden from the manual command palette's auto-suggestions, but they are still included in the AI prompt.

This is useful for fields that are difficult to type manually (like complex date ranges) but easy to express in natural language (e.g., "last 24 hours").

date: col
  .timestamp()
  .label("Date")
  .commandDisabled()  // Hidden from manual palette, available to AI
  .sortable(),

The data-table-filters system is organized into three distinct layers that work together to provide a complete filtering and data display experience:

1. Table Schema: A declarative builder used to define columns, filters, display types, sorting, and row details in a single location. It uses col.* builders and presets to define the data structure.
2. State Management: A pluggable adapter system that manages filter state. You can choose between URL-based state (using nuqs) or client-side memory state (using Zustand).
3. UI Components: Pre-built React components that consume the schema and state. This includes DataTableInfinite for tables, DataTableFilterControls for UI inputs, DataTableFilterCommand for command palettes, and DataTableSheetDetails for row details.

Data Flow: Table Schema $\rightarrow$ Generators $\rightarrow$ Components

Components located within the _components folder are non-standard and specific to the /infinite example. They are not intended for general use across different projects because they are highly specialized for specific implementation details within the /infinite route.

These components are typically utilized in the following locations:

• filterFields[number].component (defined in constants.tsx)
• columns[0].cell (defined in columns.tsx)
• DataTableSheetDetails (defined in popover-percentile.tsx)

If you are looking for reusable or generic components, do not use the _components folder. Instead, look in:

• @/components/custom
• @/components/data-table/{data-table-column}

The data layer uses cursor-based pagination instead of offset pagination. This provides stable performance on large tables and prevents data shifting when new rows are inserted.

Implementation Logic

• Cursor Type: Use a timestamp column (e.g., date) as the cursor.
• Next Page (Older rows): WHERE date < cursor ORDER BY date DESC LIMIT size
• Previous Page (Newer rows): WHERE date > cursor ORDER BY date ASC LIMIT size (Note: you must reverse the results on the client or server to maintain order).
• Communication: The client sends the cursor and direction. The server responds with nextCursor and prevCursor.

The project uses a pluggable adapter pattern for managing filter state. This allows you to choose the storage mechanism that best fits your application requirements. You can use built-in adapters or implement the StoreAdapter interface for a custom solution.

Built-in Adapters:

• nuqs: URL-based state (enables shareable URLs and browser history support).
• zustand: Client-side state (ideal for integrating with an existing Zustand store).
• memory: Ephemeral in-memory state (suitable for embedded tables or builders).

The auto-inference engine uses specific patterns in your data to determine the column type (ColKind) and the appropriate filter UI (FilterType).