FastCSV Documentation
repository·main·Indexed 20 days ago
https://github.com/osiegmar/fastcsvA high-performance, lightweight, and RFC 4180 compliant CSV library for Java. FastCSV provides high-speed reading and writing with zero runtime dependencies and is compatible with Java 17+, Android 34+, GraalVM Native Image, and OSGi. It features flexible configuration via CsvReaderBuilder and CsvWriterBuilder for handling field separators, quote characters, encoding, null values, and comment strategies.
What's inside FastCSV
- FastCSV is a high-performance, dependency-free CSV parser and writer for Java. It is designed to be lightweight, compliant with the RFC 4180 specification, and suitable for both big data applications (massive scale) and small data applications (minimal footprint).
Understand FastCSV's CSV interpretation grammar
mainFastCSV supports two modes of CSV interpretation based on whether comment handling is enabled.
Default Mode (Comment handling disabled): FastCSV adheres to the ABNF grammar of RFC 4180-bis. In this mode, fields are either escaped (enclosed in
DQUOTE) or non-escaped. Line breaks are recognized asCR,LF, orCRLF.Comment Mode (Comment handling enabled): FastCSV adheres to an extended grammar that allows for comments starting with a
HASH(#) character. In this mode, a line starting with aHASHis treated as a comment. Fields within a record can contain data that includes aHASHcharacter, provided they are handled according to the specific grammar rules fordata-with-hash.
; Default Mode (RFC 4180-bis) file = [header] *(record) header = [field] *(COMMA field) linebreak record = [field] *(COMMA field) linebreak field = (escaped / non-escaped) escaped = DQUOTE *(textdata / COMMA / CR / LF / 2DQUOTE) DQUOTE non-escaped = *(textdata) ; Comment Mode file = *((comment / record) linebreak) comment = HASH *comment-data record = first-field *(COMMA field)Encoding support in FastCSV
mainFastCSV supports reading and writing CSV files in any encoding supported by Java.
Limitation: FastCSV does not support mixed encodings within a single CSV file; the encoding must be consistent throughout the file.
Handle null values and empty fields
mainSince CSV does not natively distinguish between empty fields and null values, FastCSV provides the following mechanisms:
On Writing
Depending on your
QuoteStrategy, null values are written as:- Empty fields (default).
- Quoted empty fields (using
QuoteStrategies.EMPTY). This is useful for PostgreSQL compatibility. - To write a specific string for a null value, pass that string directly to
CsvWriter.writeRecord().
On Reading
To distinguish between an empty field and a null value, you can implement a custom
CsvRecordthat utilizes thequotedparameter provided toCsvCallbackHandler.addField()to check if the field was enclosed in quotes.1,,fooCRLF 2,"",barCRLFRead CSV files with comments using CommentStrategy
mainWhen reading CSV files that contain comments, you can control how FastCSV treats those lines using the
CommentStrategyenum. FastCSV provides three main behaviors:- Treat as data (Default): Comments are treated as part of the data to ensure maximum compatibility with the RFC 4180 standard and prevent data loss.
- Skip comments: The comment lines are ignored during the reading process.
- Read as comments: The lines are explicitly interpreted as comments.
To use a specific strategy, configure the
CsvReaderBuilder.Field separators and encapsulation constraints
mainFastCSV allows you to configure the field separator (e.g., semicolon or tab) and the encapsulation character (e.g., single quotes) to support various real-world CSV formats.
However, the following are not supported:
- Multiple characters: Field separators and encapsulation characters must be a single character.
- Mixed separators/encapsulation: You cannot use different separators or different encapsulation characters within the same file. The configured character is used for the entire file.
Configure encoding for reading and writing CSV
mainFastCSV supports any encoding, defaulting to UTF-8.
Note for
IndexedCsvReader: Because it builds its index at the byte level, it is restricted to UTF-8 and single-byte encodings.Detect if a field is quoted when reading CSV
mainFastCSV does not provide a built-in way to check if a field was enclosed in quotes during the reading process. To distinguish between quoted and unquoted fields (which can be useful for differentiating between empty strings and null values), you must implement a custom callback handler using
QuotableFieldHandlerandQuotableField.// Implementation pattern for detecting quoted fields // Requires implementing a custom handler to intercept field metadata CsvReader reader = CsvReader.builder() .fieldHandler(new MyQuotableFieldHandler()) .build(inputStream);FastCSV CSV specification compliance
mainFastCSV is compliant with RFC 4180, which includes support for:
- Newline and field separator characters within fields.
- Quote escaping.
- Configurable field separators.
- Support for
CRLF(Windows),LF(Unix), andCR(old macOS) line endings. - Full Unicode character support.
Manage internal buffer flushing in CsvWriter
mainIn FastCSV 4.x,
CsvWriter(for bothOutputStreamandWriter) no longer flushes the internal buffer after every record. It only flushes when the buffer is full, or whenflush()orclose()is called.Best Practices:
- Always call
close()on theCsvWriterat the end of the process to ensure all data is written. - Call
flush()manually if you need to write to the underlyingWriterorOutputStreamdirectly before closing. - You no longer need to wrap your
Writerin aBufferedWriterfor performance unless you have explicitly disabled FastCSV's internal buffer usingCsvWriterBuilder.bufferSize(0).
- Always call
How CsvReader works
mainThe
CsvReaderis the central entry point for reading CSV data. It uses a builder pattern (CsvReaderBuilder) to configure format specifications (like separators and quote characters) and parsing behavior.Internally, the
CsvReaderorchestrates aCsvParserto handle low-level data parsing and aCsvCallbackHandlerto materialize fields into records. TheCsvReaderimplements theIterableinterface, allowing you to iterate over parsed records (such asCsvRecordobjects) using standard loops orforEach.try (CsvReader<CsvRecord> csv = CsvReader.builder().ofCsvRecord(file)) { csv.forEach(System.out::println); }How indexed reading works in FastCSV
mainFastCSV provides
IndexedCsvReaderfor random access and paginated reading of large CSV files. Since CSV files lack a native index, FastCSV builds one in memory while reading the file. This index allows you to navigate to specific rows or pages without reading the entire file sequentially.Key Characteristics
- Non-blocking Indexing: The indexing process runs in the background. You can start reading the file while the index is still being generated. A status monitor is available to track progress.
- Memory Usage: The index is stored in the heap and scales with the record count, not the file size. It holds one page entry (approximately 40 bytes) per
pageSizerecords.- Formula:
index heap ≈ 40 bytes × ceil(recordCount / pageSize) - A smaller
pageSizeprovides finer-grained random access but increases heap consumption.
- Formula:
- Persistence: The index can optionally be stored in a file to avoid re-indexing the CSV on subsequent reads.
// Index the CSV file with up to 5 records per page IndexedCsvReader<CsvRecord> csv = IndexedCsvReader.builder() .pageSize(5) .ofCsvRecord(file);