toml-elixir

repository·main·Indexed 41 days ago

https://github.com/bitwalker/toml-elixir

A TOML 1.0 compliant parser for Elixir providing efficient decoding from strings, files, and streams. It includes the Toml.Transform behavior for custom value transformations and Toml.Provider for loading TOML files as configuration providers in Elixir Releases (1.9+) and Distillery.

Tokens
4K
Snippets
13
Records
17
Agent score
50%

What's inside toml-elixir

  1. Extend TOML decoding with Toml.Transform

    main

    The Toml.Transform behavior allows you to convert TOML values into custom Elixir types (like structs or specific data formats) during the decoding process.

    Transforms are applied in the order provided using a depth-first, bottom-up traversal. This means leaf nodes (values) are transformed before their parent branches (tables/arrays).

    # Example of a transform module
    defmodule IPStringToCharlist do
      use Toml.Transform
      
      def transform(:ip, v) when is_binary(v) do
        String.to_charlist(v)
      end
      def transform(_k, v), do: v
    end
    
    # Using transforms in decode
    transforms = [IPStringToCharlist, ...]
    {:ok, result} = Toml.decode("example.toml", keys: :atoms, transforms: transforms)
  2. Use Toml as a configuration provider

    main

    You can use Toml.Provider to load TOML files into your application environment during boot. This is useful for Elixir Releases (1.9+) and Distillery.

    Configuration Requirements

    • The :path option is required.
    • :keys only supports :atoms and :atoms!.
    • Data Format: Keys at the root of the TOML document must be tables representing the applications to be configured. Non-table root keys are ignored.

    Example: Elixir Releases

    config_providers: [
      {Toml.Provider, [
        path: {:system, "XDG_CONFIG_DIR", "myapp.toml"},
        transforms: [...]
      ]}
    ]

    Example: Distillery

    release :myapp do
      set config_providers: [
        {Toml.Provider, [path: "${XDG_CONFIG_DIR}/myapp.toml", transforms: [...]]}
      ]
    end
    # TOML structure for config provider
    [myapp]
    key = "value"
    
    [myapp."MyApp.Endpoint"]
    port = "4000"
  3. Use Toml.Provider for configuration in releases

    main

    The Toml.Provider module allows you to use TOML files as configuration providers for Elixir releases. It supports both Distillery and the native Elixir 1.9+ Config.Provider behaviour.

    When the provider runs, it reads the TOML file and persists the settings into the application environment, allowing you to access them via Application.get_env/2.

    Important Requirements:

    • The TOML file must have keys at the root that are tables. These tables correspond to the Elixir applications being configured.
    • Keys at the root that are not tables are ignored.
    • If the TOML file is missing or inaccessible, the provider will raise an error and terminate the boot sequence.
    ## Distillery Usage
    
    Add the following to your `rel/config.exs`
    
    ```elixir
    release :myapp do
            # ...snip...
            set config_providers: [
              {Toml.Provider, [path: "${XDG_CONFIG_DIR}/myapp.toml", transforms: [...]]}
            ]
          end

    Elixir Usage

    config_providers: [
            {Toml.Provider, [
              path: {:system, "XDG_CONFIG_DIR", "myapp.toml"},
              transforms: [...]
            ]}
          ]
  4. Parse TOML strings, files, and streams

    main

    Use the Toml module to decode TOML data into Elixir maps.

    • Toml.decode(input): Decodes a string. By default, keys are strings. Use keys: :atoms to convert keys to atoms.
    • Toml.decode_file(path): Decodes a file from the given path.
    • Toml.decode_stream(stream): Decodes a stream (e.g., from File.stream!/1).

    If parsing fails, it returns {:error, {:invalid_toml, reason}}.

    # Decode string with string keys
    {:ok, %{"database" => %{"server" => "192.168.1.1"}}} = Toml.decode(input)
    
    # Decode string with atom keys
    {:ok, %{database: %{server: "192.168.1.1"}}} = Toml.decode(input, keys: :atoms)
    
    # Decode from a file
    {:ok, %{"database" => %{"server" => "192.168.1.1"}}} = Toml.decode_file("example.toml")
    
    # Decode from a stream
    stream = File.stream!("example.toml")
    {:ok, %{"database" => %{"server" => "192.168.1.1"}}} = Toml.decode_stream(stream)
  5. Configure Toml.Provider options

    main

    When using Toml.Provider, you can pass options similar to those accepted by Toml.parse/2.

    Supported options:

    • :path (required): The path to the TOML file to load. This can be a string or a {:system, env_var, path} tuple (for Elixir 1.9+).
    • :keys: Determines how keys are converted. Defaults to :atoms. Can be set to :atoms! if desired. All other key types are ignored as they result in an invalid configuration structure.
    • :transforms: Pass transformations as supported by the underlying TOML parser.
  6. TOML to Elixir type conversions

    main

    The following table describes how TOML types are mapped to Elixir types:

    TOMLElixir
    StringString.t() (binary)
    Integerinteger
    inf / +inf:infinity
    -inf:negative_infinity
    nan / +nan:nan
    -nan:negative_nan
    Booleanboolean
    Offset Date-TimeDateTime.t()
    Local Date-TimeNaiveDateTime.t()
    Local DateDate.t()
    Local TimeTime.t()
    Arraylist
    Tablemap
    Table Arraylist(map)

    Implementation Notes:

    • Infinity/NaN: Since Erlang/Elixir lacks native representations for all TOML infinity/NaN variants, :infinity, :negative_infinity, :nan, and :negative_nan are used as conventions.
    • Precision: Time types have a maximum precision of microseconds (6 decimal places). Higher precision (nanoseconds) will be lost.
    • Numbers: Hex, octal, and binary numbers are converted to integers and will be represented as decimals when serialized back.
  7. Handle TOML parsing errors with Toml.Error

    main

    When parsing TOML documents, toml-elixir may raise Toml.Error exceptions. This exception contains a message (a human-readable string) and a reason (the underlying error structure). You can catch these exceptions to handle syntax errors or invalid TOML data gracefully.

    Common error reasons include:

    • :unexpected_eof: Unexpected end of file.
    • :unclosed_array or :unclosed_inline_table: Missing closing brackets/braces, often accompanied by line and column numbers.
    • :invalid_token: Encountered a character that doesn't fit the expected grammar.
    • :key_exists: Attempting to redefine a key that already exists in a path.
    • :badarg: An invalid value was provided for a specific option.
  8. Example: Transforming TOML tables into Elixir structs

    main

    This example demonstrates how to transform a TOML table into a specific Elixir struct and convert IP address strings into Erlang address tuples.

    defmodule Server do
      defstruct [:name, :ip, :ports]
    end
    
    defmodule ServerTransform do
      use Toml.Transform
    
      # Transform IP strings to Erlang address tuples
      def transform(:ip, ip) when is_binary(ip) do
        case :inet.parse_ipv4_address(String.to_charlist(ip)) do
          {:ok, result} -> result
          {:error, reason} -> {:error, {:invalid_ip, ip, reason}}
        end
      end
    
      # Error on non-binary IP values
      def transform(:ip, ip), do: {:error, {:invalid_ip, ip, :expected_string}}
    
      # Transform the :servers table into a list of %Server{} structs
      def transform(:servers, servers) when is_map(servers) do
        for {name, server} <- servers do
          struct(Server, Map.put(server, :name, name))
        end
      end
    
      # Error on non-map values for :servers
      def transform(:servers, s), do: {:error, {:invalid_server, s}}
    
      # Ignore all other values
      def transform(_key, v), do: v
    end
    
    # Input TOML:
    # [servers.alpha]
    # ip = "192.168.1.1"
    # ports = [8080, 8081]
    
    # Decoding call:
    # Toml.decode!(content, keys: :atoms, transforms: [ServerTransform])
    
    # Result:
    # %{
    #   servers: [
    #     %Server{name: :alpha, ip: {192,168,1,1}, ports: [8080, 8081]},
    #     %Server{name: :beta, ip: {192,168,1,2}, ports: [8082, 8083]}
    #   ]
    # }
  9. Decode TOML from a file

    main

    Use Toml.Decoder.decode_file(path, opts) to read a file from the filesystem and decode its contents. It returns {:ok, map} or {:error, reason}. If the file cannot be opened, the error message will include the relative path.

    For a version that raises on error, use decode_file!/2.

    # Safe decoding
    case Toml.Decoder.decode_file("config.toml", []) do
      {:ok, map} -> IO.inspect(map)
      {:error, reason} -> IO.puts("Error: #{reason}")
    end
  10. Implement custom transformations with Toml.Transform

    main

    To modify decoded TOML values (e.g., converting strings to structs or IP tuples), implement the Toml.Transform behaviour.

    Your implementation must define a transform/2 function. This function is called for every key/value pair in a depth-first, bottom-up traversal.

    Return Values:

    • term: Replace the value for the given key with this returned value.
    • {:error, term}: Stop decoding and return this error.

    To use your transformer, pass the module name in the transforms option to Toml.decode! or Toml.decode.

    defmodule MyTransformer do
      use Toml.Transform
    
      def transform(:my_key, value) do
        # logic to transform value
        transformed_value
      end
    
      def transform(_key, value) do
        value # return original value to ignore
      end
    end
    
    # Usage
    Toml.decode!(content, transforms: [MyTransformer])
  11. Decode TOML from a binary string

    main

    Use Toml.Decoder.decode!(bin, opts) to parse a raw binary into an Elixir map. This function will raise a Toml.Error if the TOML is invalid.

    Options:

    • :filename: A binary representing the filename (defaults to "nofile"). This is used to provide context in error messages. If provided, it must be a binary.
    # Basic usage
    map = Toml.Decoder.decode!("key = ""value""", [])
    
    # Usage with filename for better error reporting
    map = Toml.Decoder.decode!("key = ""value""", filename: "config.toml")