Papa Parse

repository·master·Indexed 12 days ago

https://github.com/mholt/papaparse

A high-performance, dependency-free JavaScript library for parsing and unparsing CSV or delimited text files. Version 5.5.3 supports browser and Node.js environments, featuring web workers, streaming for large files, and RFC 4180 compatibility. It provides `Papa.parse` for converting CSV to JSON and `Papa.unparse` for converting JSON back to CSV.

Tokens
2.7K
Snippets
7
Records
13
Agent score
46%

What's inside Papa Parse

  1. Use Papa Parse in Node.js with Readable Streams

    master

    In Node.js environments, Papa Parse can parse Readable Streams in addition to strings.

    Important Constraints:

    • If using a stream, the encoding option must be a Node-supported character encoding.
    • The following options are unavailable when parsing a stream: Papa.LocalChunkSize, Papa.RemoteChunkSize, download, withCredentials, and worker.

    Node.js Streaming Style (.pipe): To use Node's .pipe() method, pipe your Readable Stream to the stream returned by Papa.parse(Papa.NODE_STREAM_INPUT, options).

    When using this streaming mode:

    • The following options are unavailable: Papa.LocalChunkSize, Papa.RemoteChunkSize, download, withCredentials, worker, step, and complete.
    • Use the 'data' event to process chunks: stream.on('data', callback).
    • Use the 'end' event to signal the end of the stream: stream.on('end', callback).
  2. Use Papa Parse for CSV parsing and unparsing

    master

    Papa Parse provides two primary methods: Papa.parse for converting CSV/delimited text into JavaScript objects, and Papa.unparse for converting JavaScript objects back into CSV strings.

    import Papa from 'papaparse';
    
    // Parse CSV data or a file
    Papa.parse(file, config);
        
    // Convert JSON/objects to CSV
    const csv = Papa.unparse(data[, config]);
  3. Handle duplicate headers in CSV files

    master

    When header: true is enabled, Papa Parse automatically handles duplicate header names to ensure data can be accessed reliably. If two columns have the same name, the second one is renamed using the format {original_name}_{suffix} (e.g., id and id_1).

    You can find the mapping of original names to new names in the meta.renamedHeaders object of the result.

  4. Configure Papa Parse unparsing options

    master

    The config object passed to Papa.unparse() controls how JSON is converted to CSV. Common options include:

    OptionTypeDescription
    headerbooleanIf true, the first row of the CSV will be the keys from the objects.
    columnsArray<string>An explicit list of columns (keys) to include in the output.
    delimiterstringThe character used to separate values.
    newlinestringThe character used for newlines.
    quoteCharstringThe character used for quoting values.
    quotesboolean, function, or Array<boolean>Whether to surround every datum with quotes.
    skipEmptyLinesboolean or 'greedy'Whether to skip empty lines in the output.
    escapeCharstringThe character used to escape quotes.
    escapeFormulaeboolean or RegExpIf true, prevents outputting cells that could be parsed as formulae by spreadsheet software.
  5. Configure the Parser with options

    master

    When creating a new Parser instance, you can provide a configuration object to control the parsing behavior.

    Key configuration options include:

    • delimiter: The character used to separate fields (defaults to ,).
    • newline: The character used for line breaks (\n, \r, or \r\n).
    • comments: A character that indicates a comment line (e.g., #). If set to true, it defaults to #.
    • quoteChar: The character used to wrap fields containing special characters (defaults to ").
    • escapeChar: The character used to escape the quoteChar.
    • step: A callback function executed for every row parsed. This is useful for processing large files row-by-row to save memory.
    • preview: The number of rows to parse before stopping.
    • fastMode: A boolean to enable a faster, simplified parsing logic (skips some complex quote handling).
    • header: If true, the first row is treated as headers. If duplicate headers are found, they are automatically renamed with a suffix (e.g., header_1).
    • transformHeader: A function called for each header to transform its value.
    var parser = new Parser({
        delimiter: ",",
        newline: "\n",
        comments: "#",
        header: true,
        step: function(results) {
            console.log("Row parsed:", results.data);
        }
    });
  6. Configure Papa Parse parsing options

    master

    The config object passed to Papa.parse() controls how the CSV is processed. Common options include:

    OptionTypeDescription
    headerbooleanIf true, the first row is treated as headers and results are returned as objects.
    dynamicTypingboolean or functionIf true, numbers and booleans are automatically converted. If a function is provided, it is used to determine typing.
    skipEmptyLinesboolean or 'greedy'If true, empty lines are skipped. If 'greedy', lines containing only whitespace are also skipped.
    delimiterstring or functionThe character used to separate values. If a function is provided, it is called with the input to determine the delimiter.
    newlinestringThe character used for newlines.
    quoteCharstringThe character used for quoting values.
    transformfunctionA function called for every cell: transform(value, field).
    transformHeaderfunctionA function called for every header: transformHeader(header, index).
    workerbooleanIf true, parsing is performed in a Web Worker.
    stepfunctionA callback called for every row parsed (useful for streaming).
    chunkfunctionA callback called for every chunk of data parsed.
    completefunctionA callback called when parsing is finished.
    errorfunctionA callback called when an error occurs.
    encodingstringThe character encoding (e.g., 'UTF-8').
  7. Control the Parser lifecycle with abort() and getCharIndex()

    master

    The Parser instance provides methods to manage the parsing state:

    • abort(): Sets an internal flag to stop the parser during the next iteration (useful when using the step callback).
    • getCharIndex(): Returns the current character position (cursor) in the input string, which can be useful for debugging or resuming parsing.
    var parser = new Parser({
        step: function(results) {
            if (someCondition) {
                parser.abort();
            }
        }
    });
    parser.parse(csvString);
  8. Convert JSON to CSV with Papa.unparse()

    master

    Use Papa.unparse() to convert JSON data (an array of objects or an array of arrays) into a CSV string.

    Input Formats:

    • Array of Objects: Each object represents a row; keys are used as headers.
    • Array of Arrays: Each inner array represents a row.
    • Single Object: Wrapped into an array automatically.
    • Stringified JSON: If the input is a JSON string, it will be parsed before unparsing.
    const data = [
        { name: "John", age: 30 },
        { name: "Jane", age: 25 }
    ];
    
    const csv = Papa.unparse(data);
    console.log(csv);
    // "name,age\nJohn,30\nJane,25"
  9. Parse CSV with Papa.parse()

    master

    Use Papa.parse() to convert CSV data into JSON. The function accepts a string, a File object, or a URL. It supports various input types including local files, remote files (via streaming), and Node.js streams.

    Key features:

    • Automatic Delimiter Detection: If no delimiter is provided, Papa Parse attempts to guess it.
    • Dynamic Typing: Automatically converts values to numbers, booleans, or dates.
    • Workers: Can run in a Web Worker to prevent UI blocking.
    • Streaming: Supports chunked parsing for large files via step or chunk callbacks.
    // Parsing a string
    Papa.parse("name,age\nJohn,30\nJane,25", {
        header: true,
        complete: function(results) {
            console.log(results.data);
        }
    });
    
    // Parsing a file (e.g., from an <input type="file">)
    const file = document.getElementById('file-input').files[0];
    Papa.parse(file, {
        header: true,
        complete: function(results) {
            console.log(results.data);
        }
    });
  10. Parse CSV data using the Parser class

    master

    To parse a CSV string, instantiate a Parser with your desired configuration and call its .parse(input) method.

    The method returns an object containing:

    • data: An array of arrays representing the parsed rows.
    • errors: An array of error objects encountered during parsing.
    • meta: Metadata about the parsing process, including the delimiter, linebreak, aborted status, truncated status, and renamedHeaders (if headers were duplicated and renamed).

    Note: The input must be a string.

    var parser = new Parser({ header: true });
    var results = parser.parse("name,age\nAlice,30\nBob,25");
    
    console.log(results.data); // [{name: 'Alice', age: '30'}, {name: 'Bob', age: '25'}]
    console.log(results.errors); // []
  11. Reference: Parser Error Object structure

    master

    When parsing errors occur, the errors array contains objects with the following structure:

    KeyTypeDescription
    typestringThe category of error (e.g., Quotes).
    codestringA specific error code (e.g., MissingQuotes, InvalidQuotes).
    messagestringA human-readable description of the error.
    rownumberThe row index where the error occurred (optional).
    indexnumberThe character index in the input where the error was detected.