compact_str

repository·main·Indexed 21 days ago

https://github.com/parkmycar/compact_str

A Rust crate providing CompactString, a memory-efficient string type that implements Small String Optimization (SSO). It stores short strings (up to 24 bytes on 64-bit or 12 bytes on 32-bit architectures) directly on the stack to avoid heap allocations. The library includes the ToCompactString trait for efficient type conversion, the CompactStringExt trait for joining collections, and the format_compact! macro for string formatting.

Tokens
6K
Snippets
20
Records
26
Agent score
75%

What's inside compact_str

  1. What is CompactString and when to use it

    main

    A CompactString is a memory-efficient string type that implements Small String Optimization (SSO). It can store up to 24 bytes (on 64-bit architectures) or 12 bytes (on 32-bit architectures) directly on the stack. If a string exceeds this size, it is transparently stored on the heap.

    Key use cases:

    • As a drop-in replacement for String to reduce heap allocations.
    • In parsing, deserializing, or applications where many small strings are present.

    Key Properties:

    • size_of::<CompactString>() == size_of::<String>() (it won't increase your stack frame size).
    • Clone is O(n).
    • From<String> or From<Box<str>> re-uses the underlying buffer where possible.
    • O(1) creation from &'static str using CompactString::const_new.
    • Space optimized for Option<CompactString>: size_of::<CompactString>() == size_of::<Option<CompactString>>().
  2. What is CompactString and how to use it

    main

    A CompactString is a string type designed to be memory-efficient by inlining short strings directly on the stack, avoiding heap allocation whenever possible. It can be used almost anywhere a String or &str is expected because it implements AsRef<str>, Borrow<str>, and dereferences to str.

    Key Characteristics

    • Inlining: Short strings (up to 24 chars on 64-bit, 12 chars on 32-bit) are stored on the stack.
    • Interoperability: Easily converts to/from String and &str.
    • Memory Reuse: When converting from a String, it reuses the existing buffer if the string is long, or eagerly inlines it if it is short.

    Basic Usage

    use compact_str::CompactString;
    use std::collections::HashMap;
    
    // Creation
    let s = CompactString::new("hello");
    let s_from_string = CompactString::from(String::from("world"));
    let s_from_ref = CompactString::from("rust");
    
    // Usage in collections
    let mut map: HashMap<CompactString, CompactString> = HashMap::new();
    map.insert(CompactString::new("key"), CompactString::new("value"));
    
    // Deref to str
    let slice: &str = &s;
    assert_eq!(slice, "hello");
    
    // Comparison
    assert_eq!(CompactString::new("abc"), "abc");
    use compact_str::CompactString;
    use std::collections::HashMap;
    
    let s = CompactString::new("hello");
    let mut map: HashMap<CompactString, CompactString> = HashMap::new();
    map.insert(CompactString::new("key"), CompactString::new("value"));
    
    assert_eq!(s, "hello");
  3. Convert from String and manage memory

    main

    When converting from a String to a CompactString using From<String> or CompactString::new(string), the library performs the following:

    1. Eager Inlining: If the string is short, it is copied to the stack and the original String buffer is dropped.
    2. Buffer Reuse: If the string is long, the CompactString reuses the existing heap allocation from the String to avoid a new allocation.

    If you need to convert a large number of Strings and want to avoid the overhead of eager inlining/de-allocation, use the CompactString::from_string_buffer API (not shown in this segment).

    use compact_str::CompactString;
    
    // Re-uses the same buffer for long strings
    let long = String::from("this is a longer string that will be heap allocated");
    let long_ptr = long.as_ptr();
    
    let mut long_c = CompactString::from(long);
    assert_eq!(long_c.as_ptr(), long_ptr); // Buffer reuse confirmed
    
    // Eagerly inlines short strings
    let short = String::from("hi");
    let short_c = CompactString::from(short);
    assert!(!short_c.is_heap_allocated());
  4. Use the Drain iterator to remove ranges

    main

    The drain(range) method returns a Drain iterator that yields the characters in the specified range.

    Important: Dropping the Drain iterator will remove the selected range from the source CompactString, even if you haven't consumed all elements from the iterator. This mirrors the behavior of std::string::Drain.

    use compact_str::CompactString;
    
    let mut cs = CompactString::from("hello world");
    {
        let mut drain = cs.drain(0..5);
        while let Some(c) = drain.next() {
            println!("{}", c);
        }
    }
    // cs is now " world"
  5. Convert types to CompactString using ToCompactString

    main

    The ToCompactString trait provides the to_compact_string(&self) method. This trait is automatically implemented for all types that implement std::fmt::Display, with high-performance specialized implementations for:

    • Integers: u8, u16, u32, u64, usize, u128, i8, i16, i32, i64, isize, i128
    • Floats: f32, f64
    • Others: bool, char, NonZeroU*, NonZeroI*, String, CompactString
  6. Join and concatenate collections into CompactString using CompactStringExt

    main

    The CompactStringExt trait allows you to efficiently join collections into a CompactString. It is automatically implemented for any type that can be converted into an iterator yielding types that implement AsRef<str> (e.g., Vec<String>, slices, etc.).

    It provides two methods:

    • join_compact(separator: impl AsRef<str>)
    • concat_compact()
  7. Reference of compact_str optional features

    main

    The following features can be enabled to add functionality to CompactString:

    FeatureDescription
    serdeImplements Serialize and Deserialize
    bytesProvides from_utf8_buf and from_utf8_buf_unchecked for bytes::Buf
    markupImplements Render for HTML escaping
    dieselSupport for diesel text columns
    sqlx-mysql / sqlx-postgres / sqlx-sqliteSupport for sqlx text columns
    arbitrary / proptest / quickcheckFuzzing trait implementations
    rkyvZero-copy serialization support
    smallvecProvides into_bytes() via smallvec::SmallVec
    valuableImplements valuable::Valuable
    pyo3Python conversion via FromPyObject and IntoPyObject
    serde/schemarsImplements JsonSchema
    gardeImplements validation rule traits
    borshImplements BorshSerialize and BorshDeserialize
    zeroizeImplements Zeroize for secure memory wiping
    defmtImplements defmt::Format for embedded logging
    bevy-reflectImplements bevy_reflect traits
    utoipaImplements ToSchema and PartialSchema for OpenAPI
  8. Concatenate items in a collection into a `CompactString`

    main

    Use concat_compact() to merge all elements of an iterator or collection into one CompactString without a separator.

    use compact_str::CompactStringExt;
    
    let items = ["hello", " ", "world", "!"];
    let compact = items.concat_compact();
    
    assert_eq!(compact, "hello world!");
  9. Join items in a collection with a separator into a `CompactString`

    main

    Use join_compact(separator) to merge elements of an iterator or collection into a single CompactString, inserting the separator between each element.

    use compact_str::CompactStringExt;
    
    let fruits = vec!["apples", "oranges", "bananas"];
    let compact = fruits.join_compact(", ");
    
    assert_eq!(compact, "apples, oranges, bananas");
  10. Convert CompactString to and from String

    main

    To String

    • into_string(self): Consumes the CompactString and returns a standard String.

    From String (Memory Efficient)

    • from_string_buffer(s): Converts a String into a CompactString without inlining. This method reuses the existing heap allocation from the String instead of deallocating it and allocating a new one. This is useful for performance-sensitive code where you want to avoid allocator trips.

    Note: For standard usage where you want short strings to be inlined onto the stack, use From<String> instead of from_string_buffer.

    # use compact_str::CompactString;
    let s = CompactString::new("Hello world");
    let standard_string = s.into_string();
    
    // Efficiently reuse buffer
    let og = "hello world".to_string();
    let og_addr = og.as_ptr();
    let mut c = CompactString::from_string_buffer(og);
    assert_eq!(og_addr, c.as_ptr());
  11. Convert CompactString to an error

    main

    If the std feature is enabled, CompactString can be converted into a Box<dyn std::error::Error> or Box<dyn std::error::Error + Send + Sync>. This is useful for using CompactString as an error message in applications using the ? operator.

    use compact_str::CompactString;
    
    fn do_something() -> Result<(), Box<dyn std::error::Error>> {
        let err_msg = CompactString::from("something went wrong");
        Err(err_msg.into())
    }
  12. Use CompactString as an iterator and extend it

    main

    CompactString implements FromIterator and Extend, allowing you to build strings from iterators of char, &char, &str, Box<str>, Cow<'a, str>, String, or other CompactString instances.

    use compact_str::CompactString;
    
    // Building from an iterator
    let cs: CompactString = "hello".chars().collect();
    
    // Extending an existing CompactString
    let mut cs = CompactString::new("hello");
    cs.extend(" world");