crc-32

repository·master·Indexed 43 days ago

https://github.com/sheetjs/js-crc32

A high-performance, pure-JS implementation of the CRC-32 and CRC32C (Castagnoli) algorithms for browser and Node.js environments, supporting versions back to IE6. Version 1.2.3 provides methods for calculating checksums from buffers, binary strings, and UTF-8 strings, returning signed 32-bit integers. Includes a command-line tool and a Python script for computing checksums.

Tokens
5.2K
Snippets
28
Records
30
Agent score
55%

What's inside crc-32

  1. Convert signed CRC32 to unsigned or hex

    master

    Because the library returns signed 32-bit integers for performance, you may need to convert them to unsigned integers or hex strings using standard JS operators.

    • To Unsigned 32-bit: Use the unsigned right shift operator >>> 0.
    • To Hex String: Use .toString(16) on the unsigned result.
    • To Signed 32-bit: Use the bitwise OR operator | 0.
    // Convert signed to unsigned
    CRC32.str("SheetJS") >>> 0                      // 2647669026
    
    // Convert unsigned to hex
    (CRC32.str("SheetJS") >>> 0).toString(16)         // "9dd03922"
    
    // Convert unsigned back to signed
    (2647669026 | 0)                                // -1647298270
  2. Integrate crc32 in the Browser

    master

    Load the script directly via a <script> tag. The script exposes a global CRC32 variable.

    Note: The script may attempt to manipulate module.exports if it detects a module environment. To prevent this behavior, define the environment variable DO_NOT_EXPORT_CRC before loading the script.

    <script src="crc32.js"></script>
  3. Best practices for CRC32 performance

    master
    1. Explicitly pass seed 0: While the seed is optional, passing 0 explicitly is recommended for performance.
    2. Node.js Buffer optimization: In NodeJS, it is often faster to convert binary strings to Buffers using Buffer.from(bstr, "binary") before calling .buf() rather than using .bstr() directly.
    /* Frequently slower in NodeJS */
    crc32 = CRC32.bstr(bstr, 0);
    
    /* Frequently faster in NodeJS */
    crc32 = CRC32.buf(Buffer.from(bstr, "binary"), 0);
  4. Use the crc32 CLI

    master

    The crc32 command-line tool calculates CRC32 checksums for files or data piped via stdin. By default, it uses the standard CRC32 algorithm and outputs a signed integer. You can specify different algorithms (like CRC32C), custom seeds for rolling CRCs, and various output formats (hex, unsigned, etc.).

    To read from standard input (stdin), use - as the filename or pipe data directly into the command.

    # Calculate checksum for a file
    crc32 myfile.txt
    
    # Calculate checksum for piped data
    echo "hello" | crc32
    
    # Use CRC32C (Castagnoli) and output in uppercase hex
    echo "hello" | crc32 -c -X
  5. Use the crc32 CLI to compute checksums

    master

    The crc32 command-line tool allows you to compute CRC32 or CRC32C checksums for files or data piped from stdin. By default, it outputs the result as a signed integer.

    To read from standard input (stdin), use - as the filename or pipe data directly into the command.

    Usage:

    # Compute checksum for a file
    crc32 [options] [filename]
    
    # Compute checksum for piped data
    cat file.txt | crc32
    crc32 [options] [filename]
  6. Use CRC32C (Castagnoli) implementation

    master

    For CRC32C calculations, use the crc-32/crc32c entry point.

    NodeJS/Bundler (CommonJS):

    var CRC32C = require("crc-32/crc32c");

    NodeJS/Bundler (ESM):

    import { bstr, buf, str } from "crc-32/crc32c";

    Browser: Load crc32c.js directly; it exposes a global CRC32C variable. Use DO_NOT_EXPORT_CRC to prevent module.exports manipulation.

  7. Use CRC32 calculation methods

    master

    All CRC32 methods take a data argument and an optional seed (for rolling CRC).

    Important: The return value is a signed 32-bit integer.

    • CRC32.buf(byte array or buffer[, seed]): Assumes a sequence of 8-bit unsigned integers (Node.js Buffer, Uint8Array, or an array of bytes).
    • CRC32.bstr(binary string[, seed]): Assumes a binary string where byte i is the low byte of the UCS-2 char: str.charCodeAt(i) & 0xFF.
    • CRC32.str(string[, seed]): Assumes a standard JS string and calculates the hash of the UTF-8 encoding.
    // Example usage
    CRC32.str("SheetJS")                            // -1647298270
    CRC32.bstr("SheetJS")                           // -1647298270
    CRC32.buf([ 83, 104, 101, 101, 116, 74, 83 ])   // -1647298270
    
    // Rolling CRC
    crc32 = CRC32.buf([83, 104])                    // -1826163454  "Sh"
    crc32 = CRC32.str("eet", crc32)                 //  1191034598  "Sheet"
    CRC32.bstr("JS", crc32)                         // -1647298270  "SheetJS"
  8. Calculate CRC-32C checksum for a buffer with CRC32C.buf()

    master

    Use CRC32C.buf(B, [seed]) to calculate the CRC-32C (Castagnoli) checksum of a buffer (such as a Uint8Array). This method is optimized for performance using a slice-by-16 algorithm.

    • B: The input buffer (e.g., Uint8Array).
    • seed (optional): An initial seed value for the calculation.
    const CRC32C = require('./crc32c.js');
    
    const data = new Uint8Array([0x01, 0x02, 0x03, 0x04]);
    const checksum = CRC32C.buf(data);
    console.log(checksum);
  9. Calculate CRC-32C checksum for a binary string with CRC32C.bstr()

    master

    Use CRC32C.bstr(bstr, [seed]) to calculate the CRC-32C checksum of a binary string (where each character represents a single byte).

    • bstr: The input binary string.
    • seed (optional): An initial seed value for the calculation.
    const CRC32C = require('./crc32c.js');
    
    const bstr = "\x01\x02\x03\x04";
    const checksum = CRC32C.bstr(bstr);
    console.log(checksum);
  10. Calculate CRC32C (Castagnoli) checksums

    master

    The js-crc32 module provides high-performance CRC32C (Castagnoli) checksum calculations for different data types. You can calculate checksums for strings, binary strings, or buffers/typed arrays using the following exported functions:

    • str(str, [seed]): Calculates the checksum for a UTF-16 string. It handles multi-byte characters (including surrogate pairs) correctly.
    • bstr(bstr, [seed]): Calculates the checksum for a binary string (where each character represents a byte).
    • buf(B, [seed]): Calculates the checksum for a buffer, Uint8Array, or an array of numbers. This is the most performant method for raw binary data.

    All functions accept an optional seed of type number to allow for incremental checksum calculation. The returned value is a signed 32-bit integer.

    import { str, buf, bstr } from './crc32c.mjs';
    
    // For standard strings
    const checksumStr = str('hello world');
    
    // For binary data (Uint8Array, Buffer, etc.)
    const data = new Uint8Array([0x01, 0x02, 0x03]);
    const checksumBuf = buf(data);
    
    // For binary strings
    const checksumBstr = bstr('abc');
    
    // Using a seed for incremental calculation
    const seed = 0x12345678;
    const incremental = buf(data, seed);