csv

repository·master·Indexed 23 days ago

https://github.com/burntsushi/rust-csv

A fast and flexible CSV reader and writer for Rust with native support for Serde deserialization. The project includes the main csv crate (v1.4.0), csv-core for high-performance parsing and writing in no_std contexts, and csv-index for creating serializable indices to enable efficient random access to CSV records.

Tokens
17.9K
Snippets
39
Records
78
Agent score
81%

What's inside csv

  1. Use csv-core for no_std CSV parsing and writing

    master

    The csv-core crate provides high-performance CSV reader and writer APIs specifically designed for no_std environments. It does not use the Rust standard library.

    Note: If you require more ergonomic or high-level CSV parsing routines, use the csv crate instead. csv-core is intended for low-level, performance-critical, or resource-constrained applications.

  2. Supported and unsupported Serde types for CSV serialization

    master

    When using serialize with Serde, be aware of the following constraints:

    Supported

    • Scalars: bool, i8 through i128, u8 through u128, f32, f64, char, &str, &[u8].
    • Sequences/Tuples: Vec<T>, (T, U), and other tuple-like structures are flattened into a comma-separated list of values.
    • Structs: Fields are serialized as a sequence of values. If using serialize_header, field names are used as headers.
    • Newtype Structs: The inner value is serialized directly.
    • Unit/None: Serialized as an empty field (e.g., "").
    • Enums: Unit variants and Newtype variants are supported (e.g., Enum::Variant or Enum::Variant(value)).

    Unsupported

    • Maps: Serializing maps is not supported and will return an error.
    • Enum Tuple Variants: e.g., Enum::Variant(T, U) is not supported.
    • Enum Struct Variants: e.g., Enum::Variant { field: T } is not supported.
    • Nested Containers inside Structs (for headers): While serialize handles nested structures by flattening them, serialize_header will return an error if it encounters a container (like a sequence or map) inside a struct field while trying to determine headers.
  3. Use ByteRecord for raw byte-based CSV handling

    master

    A ByteRecord stores a single CSV record as raw bytes. This is useful when you need to handle CSV rows that are not valid UTF-8. While StringRecord is generally more ergonomic, ByteRecord is necessary for:

    1. Deserializing fields that contain invalid UTF-8.
    2. Using Serde to read into borrowed data like &'a str or &'a [u8].

    Two ByteRecords are compared based on their field data; position information is ignored during equality checks.

    use csv::ByteRecord;
    
    let record = ByteRecord::new();
  4. Use StringRecord for UTF-8 based CSV records

    master

    A StringRecord is a single CSV record stored as valid UTF-8 bytes. It is used when you need to work with text data and want to ensure all fields are valid UTF-8.

    Important Considerations:

    • Invalid UTF-8: If you attempt to read CSV data that contains invalid UTF-8 into a StringRecord, the reader will return a FromUtf8Error. If your data contains invalid UTF-8, use ByteRecord instead, as it makes no assumptions about UTF-8 encoding.
    • Serde Integration: While StringRecord can be used with Serde, if you need to deserialize fields that might contain invalid UTF-8, you should first read the row into a ByteRecord and then use ByteRecord::deserialize.
    use csv::StringRecord;
    
    let record = StringRecord::new();
  5. Understand CSV parsing error behavior

    master

    The csv crate prioritizes finding a parse over rejecting data. Because CSV data is highly variable, the parser attempts to find a way to interpret the data even if it seems malformed.

    Common error scenarios include:

    • Unequal Lengths: Occurs when a record has a different number of fields than previous records (unless .flexible(true) is used).
    • IO Errors: Occurs when reading from the underlying resource (e.g., a file) fails. Subsequent read attempts after an IO error will behave as if EOF was reached to prevent infinite loops.
    • UTF-8 Errors: Occurs when reading StringRecords if the data contains invalid UTF-8. To handle invalid UTF-8, use the byte-oriented ByteRecord API instead.
    • Serde Errors: Occurs when deserializing CSV data into specific Rust types (e.g., trying to parse a non-numeric string into an i32).
  6. Handle unequal field lengths in CSV records

    master

    By default, the csv reader expects every record to have the same number of fields as the first record (or the header). If a record has a different length, an ErrorKind::UnequalLengths error is returned.

    To allow records with varying numbers of fields, enable the flexible option in the ReaderBuilder.

    Error Details: When an UnequalLengths error occurs, it contains:

    • pos: The Position of the error.
    • expected_len: The number of fields expected.
    • len: The actual number of fields found.
  7. Handle unequal record lengths with flexible parsing

    master

    By default, the csv crate enforces that all records have the same number of fields. If a record is encountered with a different number of fields than a previous record, an error of type ErrorKind::UnequalLengths is returned.

    To allow records to have varying numbers of fields, enable flexible parsing using .flexible(true) on the ReaderBuilder.