AVA Framework

AVA is designed to run in both environments with automatic detection and adaptation:

Note for Browsers: For extremely large datasets, it is recommended to use the server-side (Node.js) version to avoid browser memory pressure.

AVA follows a modular pipeline that adapts to data size and environment:

1. Data Loading: Data is ingested via CSV, JSON, URL, or Text.
2. Metadata Extraction: The system performs type inference and statistics.
3. Size-based Routing:
   • Small datasets (< 10KB): Uses JavaScript helpers for in-memory analysis.
   • Large datasets (≥ 10KB): Switches to a database-backed approach.
4. Environment-specific Storage:
   • Browser: Uses IndexedDB for large datasets.
   • Node.js: Uses SQLite for large datasets.
5. Analysis & Summary: Generates code/SQL, executes it, and uses an LLM to summarize the result into natural language.
6. Visualization (Optional): Detects intent and generates chart syntax and HTML.

You can install the AVA package using npm, pnpm, or yarn.

npm install @antv/ava

pnpm install @antv/ava

yarn add @antv/ava

AVA uses different storage strategies depending on the environment and the size of the dataset to optimize performance and memory usage:

1. Small Datasets: Kept directly in memory as a JavaScript array.
2. Large Datasets (Node.js): If the data size exceeds sqlThreshold, it is loaded into a SQLiteDataStore. Analysis is performed using generated SQL queries.
3. Large Datasets (Browser): If the data size exceeds sqlThreshold and IndexedDB is available, it is loaded into an IndexedDBDataStore. Analysis is performed by generating and executing JavaScript code against the data.
4. Browser Fallback: If IndexedDB is unavailable, large datasets remain in memory, which may cause performance issues.

AVA uses DatasetInfo to describe the structure and scale of the data being analyzed. This includes row and column counts, total size in bytes, and an array of FieldMetadata. Each field describes its name, type (number, string, date, or boolean), and statistical properties like uniqueCount, nullCount, and samples.

export interface DatasetInfo {
  rowCount: number;
  columnCount: number;
  fields: FieldMetadata[];
  sizeInBytes: number;
}

export interface FieldMetadata {
  name: string;
  type: 'number' | 'string' | 'date' | 'boolean';
  samples?: any[];
  uniqueCount?: number;
  nullCount?: number;
}

To use AVA, create a new instance using the AVA class. You must provide an llm configuration object containing the model, apiKey, and baseURL.

You can optionally specify a sqlThreshold (in bytes) to control when the system switches from in-memory processing to SQLite. The default threshold is 10KB.

import { AVA } from '@antv/ava';

const ava = new AVA({
  llm: {
    model: 'ling-1t',
    apiKey: 'YOUR_API_KEY',
    baseURL: 'LLM_BASE_URL',
  },
  sqlThreshold: 1024 * 1024 * 2, // Threshold for switching to SQLite
});

To prevent memory leaks and release resources such as SQLite, IndexedDB, or in-memory data, call dispose() when the AVA instance is no longer needed.

ava.dispose();

AVA supports loading data from various formats and sources. The behavior of loadCSV varies depending on the environment:

• Node.js: Accepts a file path.
• Browser: Accepts a CSV content string.

Available loading methods:

• loadCSV(filePathOrContent): Load CSV data.
• loadObject(data): Load a JSON object.
• loadURL(url, transform?): Load data from a URL with an optional transformation function.
• loadText(text): Load unstructured text (processed via LLM).

AVA provides AI-driven analysis capabilities:

1. suggest(count?): Generates a list of recommended analysis queries based on your data. Returns an array of objects containing query, score, and reason.
2. analysis(query): Executes a natural language query. It returns an object containing:
   • text: A natural language summary of the result.
   • data: The structured analysis result.
   • code: (Optional) JavaScript code used for in-memory analysis (small datasets).
   • sql: (Optional) SQL query used for SQLite analysis (large datasets).

// Get top 5 suggested queries
const queries = await ava.suggest(5);

// Run analysis using a natural language query
const result = await ava.analysis('What is the average revenue by region?');
console.log(result.text);

// Or use a suggested query
const suggestedResult = await ava.analysis(queries[0].query);