rapidcsv

repository·master·Indexed 21 days ago

https://github.com/d99kris/rapidcsv

A lightweight, header-only C++11 CSV parser library. It provides the rapidcsv::Document class for interacting with CSV files via rows, columns, and cells, with support for custom separators, row/column headers, and custom data type conversions through the rapidcsv::Converter<T> template. The library supports integration via CMake (add_subdirectory, FetchContent, find_package), vcpkg, and Conan.

Tokens
9.8K
Snippets
42
Records
49
Agent score
73%

What's inside rapidcsv

  1. Overview of the rapidcsv API

    master

    Rapidcsv provides a set of classes for parsing and interacting with CSV files in C++. The core API is organized around the following components:

    • rapidcsv::Document: The primary interface for interacting with a loaded CSV file, allowing access to data via rows, columns, or specific cells.
    • rapidcsv::Converter<T>: A template class used to define how data is converted from the raw CSV format into specific C++ types.
    • rapidcsv::ConverterParams: Configuration parameters for the conversion process.
    • rapidcsv::LabelParams: Configuration for handling column and row headers (labels).
    • rapidcsv::LineReaderParams: Parameters for the line-by-line reading process.
    • rapidcsv::SeparatorParams: Configuration for defining custom delimiters (e.g., commas, tabs, semicolons).
    • rapidcsv::no_converter: A specialized converter type for cases where no conversion is required.
  2. Configure Column and Row headers with LabelParams

    master

    By default, Rapidcsv treats the first row as column headers and the first column as data. To enable row headers (allowing you to access rows by label), use rapidcsv::LabelParams in the Document constructor.

    • Column and Row Headers: Set pRowNameIdx to 0 (e.g., rapidcsv::LabelParams(0, 0)).
    • Row Headers Only: Set pColNameIdx to -1 (e.g., rapidcsv::LabelParams(-1, 0)).
    • No Headers: Set both to -1 (e.g., rapidcsv::LabelParams(-1, -1)). When no headers exist, access data using integer indices instead of strings.
    // Column and Row Headers
    rapidcsv::Document doc("examples/colrowhdr.csv", rapidcsv::LabelParams(0, 0));
    std::vector<float> close = doc.GetRow<float>("2017-02-22");
    
    // Row Headers Only
    rapidcsv::Document doc("examples/rowhdr.csv", rapidcsv::LabelParams(-1, 0));
    
    // No Headers
    rapidcsv::Document doc("examples/nohdr.csv", rapidcsv::LabelParams(-1, -1));
    std::vector<float> col = doc.GetColumn<float>(5); // Access by index
  3. Specialize rapidcsv::Converter<T> for custom datatypes

    master

    While rapidcsv::Converter<T> is primarily used internally, it is exposed to allow developers to specialize it for custom datatype conversions. By providing a template specialization for ToStr and ToVal, you can define how your custom types are converted to and from strings during CSV parsing and writing.

    To implement a custom converter, you must specialize the following methods for your type T:

    1. ToStr(const T & pVal, std::string & pStr): Converts your type to a string representation.
    2. ToVal(const std::string & pStr, T & pVal): Converts a string representation back into your type.
    /* Example of how one might specialize the converter for a custom type */
    
    template<> 
    void rapidcsv::Converter<MyCustomType>::ToStr(const MyCustomType & pVal, std::string & pStr) {
        pStr = pVal.to_string();
    }
    
    template<> 
    void rapidcsv::Converter<MyCustomType>::ToVal(const std::string & pStr, MyCustomType & pVal) {
        pVal = MyCustomType::from_string(pStr);
    }
  4. Build the CMake Find Package Example Project

    master

    To build the example project that demonstrates how to use find_package with rapidcsv, follow these steps. First, install the library using the provided shell script, then create a build directory and use CMake to configure and build the project.

    # Install the library
    pushd ../../ ; ./make.sh install ; popd
    
    # Build the example project
    mkdir -p build && cd build && cmake .. && make
  5. Install Rapidcsv

    master

    Rapidcsv is a header-only C++ library. You can install it using one of the following methods:

    1. Manual: Copy rapidcsv.h to your project's include directory.
    2. vcpkg: Available via the vcpkg package manager.
    3. Conan: Available via the Conan package manager.
  6. Integrate rapidcsv using CMake add_subdirectory

    master

    To use rapidcsv as a subdirectory in your own CMake project, you can symlink the repository into your project structure and use the add_subdirectory command. This allows you to build rapidcsv as part of your main build process.

    Setup and Build Steps

    1. Create a symbolic link to the rapidcsv directory within your project.
    2. Create a build directory.
    3. Run cmake and make to compile the project.
    # Link the rapidcsv repository to your current directory
    ln -s ../.. rapidcsv
    
    # Create build directory and compile
    mkdir -p build && cd build && cmake .. && make
  7. Limit maximum document size

    master
    To protect against memory exhaustion from malformed or untrusted input, define the macro RAPIDCSV_MAX_COUNT before including rapidcsv.h. This sets an upper limit on the number of rows or columns read; exceeding this limit will throw a std::out_of_range exception.
  8. Basic usage: Read a column as a vector

    master

    To read a CSV file and extract a specific column into a std::vector, instantiate a rapidcsv::Document with the file path and use the GetColumn<T> method. By default, the first row is treated as column headers.

    #include <iostream>
    #include <vector>
    #include "rapidcsv.h"
    
    int main()
    {
      rapidcsv::Document doc("examples/colhdr.csv");
    
      std::vector<float> col = doc.GetColumn<float>("Close");
      std::cout << "Read " << col.size() << " values." << std::endl;
    }
  9. Retrieve columns and rows in rapidcsv::Document

    master

    You can extract entire rows or columns as vectors of a specified type T.

    Get Columns:

    • GetColumn<T>(size_t pColumnIdx): By zero-based index.
    • GetColumn<T>(const std::string & pColumnName): By label name.

    Get Rows:

    • GetRow<T>(size_t pRowIdx): By zero-based index.
    • GetRow<T>(const std::string & pRowName): By label name.

    Both methods support an optional ConvFunc<T> pToVal for custom conversion.

    // Get a whole column as a vector of ints
    std::vector<int> ages = doc.GetColumn<int>("Age");
    
    // Get a whole row as a vector of strings
    std::vector<std::string> row = doc.GetRow<std::string>("RowName");
  10. Configure quoting and line reading with SeparatorParams and LineReaderParams

    master

    Use SeparatorParams to control how quoted cells are handled and LineReaderParams to manage comment and empty lines.

    • Disable Auto-dequoting: Set pAutoQuote = false in SeparatorParams.
    • Skip Comment Lines: Use LineReaderParams with pSkipCommentLines = true and specify the pCommentPrefix.
    • Skip Empty Lines: Use LineReaderParams with pSkipEmptyLines = true.
    // Disable auto-quoting
    rapidcsv::SeparatorParams sep_params(',' /* pSeparator */, false /* pTrim */, rapidcsv::sPlatformHasCR, false /* pQuotedLinebreaks */, false /* pAutoQuote */);
    
    // Skip comments starting with '#'
    rapidcsv::LineReaderParams line_params(true /* pSkipCommentLines */, '#' /* pCommentPrefix */, false /* pSkipEmptyLines */);
    
    rapidcsv::Document doc("file.csv", rapidcsv::LabelParams(), sep_params, rapidcsv::ConverterParams(), line_params);
  11. Access cell data in rapidcsv::Document

    master

    You can retrieve individual cell values using several combinations of indices and names. Most methods are templated, allowing you to specify the desired return type T.

    Access Methods:

    • By Index: GetCell<T>(size_t pColumnIdx, size_t pRowIdx)
    • By Name: GetCell<T>(const std::string & pColumnName, const std::string & pRowName)
    • Mixed (Column Name, Row Index): GetCell<T>(const std::string & pColumnName, size_t pRowIdx)
    • Mixed (Column Index, Row Name): GetCell<T>(size_t pColumnIdx, const std::string & pRowName)

    All methods support an optional ConvFunc<T> pToVal parameter to provide a custom conversion function.

    // Get an integer from a specific cell by name
    int val = doc.GetCell<int>("Age", "John Doe");
    
    // Get a double from a cell by index
    double price = doc.GetCell<double>(0, 5);