tomlc17 TOML Parser

repository·main·Indexed 41 days ago

https://github.com/cktan/tomlc17

A lightweight, strictly compliant TOML v1.1 parser for C and C++. It parses TOML documents into an in-memory tree structure and provides a public API for navigation via toml_get and toml_seek, merging documents with toml_merge, and structural comparison using toml_equiv. The library is designed for zero-friction integration with no dependencies beyond the C standard library and supports custom allocators via toml_option_t.

Tokens
6.1K
Snippets
12
Records
24
Agent score
51%

What's inside tomlc17

  1. Use C++20 accessors in tomlc17

    main

    The toml namespace provides C++20 constructs for safe and efficient TOML parsing and data extraction. Key features include:

    • Safe resource management: The toml::Result class follows RAII rules, automatically handling memory cleanup via its destructor.
    • Direct subtable access: You can retrieve values from nested subtables using path-based access.
    • Safe value access: Accessing values uses std::optional. If an access is invalid or a value is missing, it throws std::bad_optional_access.
    • Time and date support: Timestamps are represented as std::chrono objects.
    • Array conversion: Datum arrays can be converted into std::vector<int64_t> or std::vector<std::string> for easy manipulation.
  2. Public API Overview for tomlc17

    main

    The tomlc17 public API is contained in src/tomlc17.h. The library is a single-translation-unit implementation with no dependencies beyond the C standard library.

    Key components include:

    • toml_datum_t: The core type representing a node in the TOML value tree.
    • toml_result_t: The output structure containing the parsed/merged tree and error information.
    • Parsing/Merging: toml_parse*, toml_merge.
    • Navigation/Access: toml_get, toml_seek.
    • Memory Management: toml_free (the only public teardown call).
    • Configuration: toml_option_t for providing custom allocator hooks.
  3. Understand Bare Key Character differences between TOML v1.0 and v1.1

    main

    A bare key is an unquoted key in a TOML file. The definition of allowed characters for bare keys changed significantly between versions:

    TOML v1.0 (Legacy)

    Bare keys are restricted to strict ASCII characters:

    • ASCII letters (A-Z, a-z)
    • ASCII digits (0-9)
    • Underscores (_)
    • Dashes (-)

    Note: In v1.0, internationalized keys like café = "val" are invalid as bare keys and must be quoted: "café" = "val".

    TOML v1.1 (Expanded)

    TOML v1.1 supports non-ASCII Unicode characters in bare keys, aligning largely with the XML 1.1 NameChar specification. This allows for internationalized keys without quotes.

    Examples of valid v1.1 bare keys:

    • café = "coffee"
    • ñoño = "val"
    • 日本語 = "val"
  4. Accessing values in toml_datum_t

    main

    A toml_datum_t represents a node in the parsed document tree. To access the actual value, check the type field (of type toml_type_t) and then access the corresponding field in the u union.

    typefield(s) to read
    TOML_STRINGu.s or u.str.ptr / u.str.len
    TOML_INT64u.int64
    TOML_FP64u.fp64
    TOML_BOOLEANu.boolean
    TOML_DATEu.ts.year, .month, .day
    TOML_TIMEu.ts.hour, .minute, .second, .usec
    TOML_DATETIMEall u.ts fields except .tz
    TOML_DATETIMETZall u.ts fields
    TOML_ARRAYu.arr.size, u.arr.elem[]
    TOML_TABLEu.tab.size, u.tab.key[], u.tab.value[]
  5. How to merge two TOML results

    main

    The toml_merge function allows you to combine two existing toml_result_t objects into a new, third, independent result.

    Behavior:

    • It performs a deep copy of the first result's tree.
    • It folds the second result into the first using the following rules:
      • Arrays of Tables: Appended.
      • Tables: Recursively merged.
      • Other types: The value from the second result overrides the first.
    • Memory: All string bytes are copied into a fresh pool_t for the new result. The two input results remain independently owned and are not modified or aliased into the new result.
  6. Threading and Concurrency in tomlc17

    main

    The library is designed to be thread-safe for concurrent operations on independent data, but it uses no internal locks.

    • Safe: You can perform multiple toml_parse* or toml_merge calls concurrently, provided each call uses its own independent input and produces its own toml_result_t.
    • Unsafe: Do not call toml_set_option() (to change allocator hooks) while any other parsing, merging, or freeing operations are in progress, as the global configuration is unsynchronized.
  7. Memory Management and Teardown

    main

    A parse or merge operation produces a single, self-contained toml_result_t. This result owns all its allocated memory, including the string bytes and the tree structure.

    To prevent memory leaks, you must call toml_free() to tear down the entire result. You cannot free individual nodes or strings within the tree; the entire tree and its associated memory pool are destroyed at once.

  8. Extract TOML values using get() and seek()

    main

    To extract data from a parsed TOML file, use toml::Result in combination with get() or seek() methods on the retrieved Datum objects.

    get() method

    Use result.get({"key1", "key2"}) to retrieve a value using a list of keys representing the path to the nested element. This returns a pointer to a Datum.

    seek() method

    Use result.seek("key1.key2") to retrieve a value using a dot-separated string path. This also returns a pointer to a Datum.

    Error Handling

    Both methods return std::optional wrappers (via methods like .as_str() or .as_intvec()). If the key does not exist or the type is incorrect, calling .value() on the optional will throw std::bad_optional_access. It is recommended to wrap these calls in try-catch blocks.

    Example

    // Parse the file
    toml::Result result = toml::parse_file_ex("simple.toml");
    
    if (!result.ok()) {
      // Handle parse error using result.errmsg()
    }
    
    try {
      // Accessing nested string via get()
      std::string host = result.get({"server", "host"})->as_str().value();
    
      // Accessing nested array via seek()
      std::vector<int64_t> ports = result.seek("server.port")->as_intvec().value();
    } catch (const std::bad_optional_access &ex) {
      // Handle missing or invalid properties
    }
    /*
     * Parse the config file simple.toml:
     *
     * [server]
     * host = "www.example.com"
     * port = [8080, 8181, 8282]
     *
     */
    #include "../src/tomlcpp.hpp"
    #include <iostream>
    #include <vector>
    
    using std::cout;
    
    static void error(const char *msg) {
      fprintf(stderr, "ERROR: %s\n", msg);
      exit(1);
    }
    
    int main() {
      // Parse the toml file
      toml::Result result = toml::parse_file_ex("simple.toml");
    
      // Check for parse error
      if (!result.ok()) {
        error(result.errmsg());
      }
    
      // Extract values
      std::string host;
      std::vector<int64_t> port;
    
      try {
        // use the Datum::get() method
        host = result.get({"server", "host"})->as_str().value();
      } catch (const std::bad_optional_access &ex) {
        error("missing or invalid 'server.host' property in config");
      }
    
      try {
        // use the Datum::seek() method
        port = result.seek("server.port")->as_intvec().value();
      } catch (const std::bad_optional_access &ex) {
        error("missing or invalid 'server.port' property in config");
      }
    
      // Print values
      cout << "server.host = " << host << "\n";
      cout << "server.port = [";
      for (size_t i = 0; i < port.size(); i++) {
        cout << (i ? ", " : "") << port[i];
      }
      cout << "]\n";
    
      // Done! No need to call toml_free(). Result::~Result() will handle destruction.
      return 0;
    }
  9. Typical usage pattern in tomlc17

    main

    To use tomlc17, follow these steps:

    1. Parse: Call toml_parse(), toml_parse_file(), or toml_parse_file_ex() to parse a document.
    2. Validate: Check result.ok. If false, use result.errmsg to inspect the error.
    3. Navigate: Use toml_get() or toml_seek() to traverse the parsed document tree.
    4. Cleanup: Call toml_free() on the result when finished.

    Memory Management Rule: You must call toml_free() on every toml_result_t returned by a parse or merge function, even if result.ok is false. All strings and data returned by the API are owned by the toml_result_t and remain valid only until toml_free() is called.

    #include "tomlc17.h"
    
    // 1. Parse
    toml_result_t result = toml_parse_file_ex("config.toml");
    
    // 2. Check result
    if (result.ok) {
        // 3. Navigate
        toml_datum_t val = toml_get(result.toptab, "key");
        // ... use val ...
    } else {
        printf("Error: %s\n", result.errmsg);
    }
    
    // 4. Free
    toml_free(result);
  10. Quick Start: Parsing and extracting TOML data

    main

    To use tomlc17, parse a TOML file into a tree structure using toml_parse_file_ex. You can then navigate this tree using toml_seek with dot-notation paths (e.g., "server.host") to retrieve toml_datum_t objects. Always check the ok field of the toml_result_t to ensure parsing succeeded, and verify the type of the returned datum before accessing its union members (like u.s for strings or u.int64 for integers). Finally, call toml_free to release the memory associated with the result.

    /*
     * Parse the config file simple.toml:
     *
     * [server]
     * host = "www.example.com"
     * port = [8080, 8181, 8282]
     *
     */
    #include "tomlc17.h"
    #include <errno.h>
    #include <inttypes.h>
    #include <stdlib.h>
    #include <string.h>
    #include <stdio.h>
    
    static void error(const char *msg, const char *msg1) {
      fprintf(stderr, "ERROR: %s%s\n", msg, msg1 ? msg1 : "");
      exit(1);
    }
    
    int main() {
      // Parse the toml file
      toml_result_t result = toml_parse_file_ex("simple.toml");
    
      // Check for parse error
      if (!result.ok) {
        error(result.errmsg, 0);
      }
    
      // Extract values
      toml_datum_t host = toml_seek(result.toptab, "server.host");
      toml_datum_t port = toml_seek(result.toptab, "server.port");
    
      // Print server.host
      if (host.type != TOML_STRING) {
        error("missing or invalid 'server.host' property in config", 0);
      }
      printf("server.host = %s\n", host.u.s);
    
      // Print server.port
      if (port.type != TOML_ARRAY) {
        error("missing or invalid 'server.port' property in config", 0);
      }
      printf("server.port = [");
      for (int i = 0; i < port.u.arr.size; i++) {
        toml_datum_t elem = port.u.arr.elem[i];
        if (elem.type != TOML_INT64) {
          error("server.port element not an integer", 0);
        }
        printf("%s%" PRId64, i ? ", " : "", elem.u.int64);
      }
      printf("]\n");
    
      // Done!
      toml_free(result);
      return 0;
    }
  11. Build tomlc17

    main

    You can build tomlc17 using make. Use the DEBUG environment variable to toggle between debug and release builds.

    Debug build:

    export DEBUG=1
    make

    Release build:

    unset DEBUG
    make
    export DEBUG=1
    make
  12. Configure custom memory allocators in tomlc17

    main

    By default, tomlc17 uses standard C library realloc() and free() for memory management. You can override this behavior using toml_set_option() to integrate with custom memory pools, arena allocators, or to track allocations for debugging and resource constraints.

    Implementation Requirements

    • mem_realloc: Must behave like standard realloc():
      • If size > 0: Allocate or resize memory; return the pointer or NULL on failure.
      • If size == 0: Free the memory and return NULL.
    • mem_free: Must handle NULL pointers gracefully.

    Usage Constraints

    • Initialization: Call toml_set_option() exactly once during program startup, before any parsing operations occur.
    • Thread Safety: toml_set_option() is not thread-safe. Ensure options are set during initialization in a single-threaded context.
    #include "tomlc17.h"
    #include <stdlib.h>
    
    // Custom realloc function
    void* my_realloc(void *ptr, size_t size) {
      if (size == 0) {
        free(ptr);
        return NULL;
      }
      return realloc(ptr, size);
    }
    
    // Custom free function
    void my_free(void *ptr) {
      free(ptr);
    }
    
    int main() {
      // Set custom allocators
      toml_option_t opt = toml_default_option();
      opt.mem_realloc = my_realloc;
      opt.mem_free = my_free;
      toml_set_option(opt);
    
      // Now all subsequent parse calls will use your custom allocators
      toml_result_t result = toml_parse_file_ex("config.toml");
      
      if (result.ok) {
        toml_free(result);
      }
    
      return 0;
    }