DOCSEARCHbyAlgolia
Submit Docs

Tomlyn

repository·main·Indexed 20 days ago

https://github.com/xoofx/tomlyn

A high-performance .NET TOML 1.1 parser and serializer featuring a System.Text.Json-style API. It provides two layers: an object serialization layer using TomlSerializer for mapping to POCOs and a low-level parsing layer (TomlLexer, TomlParser, SyntaxParser) for full-fidelity syntax trees. Tomlyn is designed to be NativeAOT and trimming-ready through source generation via TomlSerializerContext, and it supports a dynamic DOM model using TomlTable and TomlArray.

Tokens
20.6K
Snippets
54
Records
93
Agent score
68%

What's inside Tomlyn

  1. Use low-level parsing APIs in Tomlyn

    main

    Tomlyn provides three distinct low-level parsing approaches depending on your performance and fidelity requirements:

    1. TomlLexer: Use this for an allocation-free token stream. It provides precise source locations for every token, making it ideal for high-performance scanning or custom error reporting.
    2. TomlParser: Use this for a pull-based, incremental event stream. By calling MoveNext(), you can consume TOML events one by one, which is highly efficient for building custom readers or serializers without loading the entire structure into memory.
    3. SyntaxParser: Use this when you need a full-fidelity, lossless parse. It builds a DocumentSyntax tree that preserves all 'trivia' (comments, whitespace, and formatting), allowing you to modify the document and write it back out (round-tripping) without losing the original styling.

  2. Use Tomlyn for object serialization

    main

    Tomlyn provides an object serialization layer designed to align with System.Text.Json concepts, attributes, and source generation. You can use TomlSerializer to convert C# objects to and from TOML format.

    The serialization engine is built upon several core components:

    • Metadata: Uses TomlTypeInfo<T>, resolver interfaces, and source-generated contexts to understand type structures.
    • Attribute Support: Supports both native Tomlyn attributes and standard System.Text.Json.Serialization attributes.
    • Converters: Extensible via TomlConverter<T> and TomlConverterFactory.
    • Primitives: Uses TomlReader and TomlWriter for low-level data handling.

  3. Overview of Tomlyn layers

    main

    Tomlyn is a high-performance TOML 1.1 library for .NET designed with an API shape similar to System.Text.Json. It provides two distinct layers depending on your use case:

    1. Object serialization: Use TomlSerializer to map TOML documents to and from .NET objects (POCOs, records, or dictionaries). This layer is optimized for data mapping and does not preserve comments or formatting.
    2. Low-level parsing: Use TomlLexer, TomlParser, and SyntaxParser to access token streams, incremental events, or a full-fidelity syntax tree. This layer is intended for building tooling like formatters, analyzers, or editors that must preserve every character of the original source.

  4. What is the Tomlyn DOM model?

    main

    The DOM (Document Object Model) in Tomlyn allows for dynamic TOML manipulation without the need to define POCO (Plain Old CLR Object) classes. This is ideal for scenarios where the TOML structure is unknown at compile time or when you want to avoid reflection-based overhead. The DOM model is compatible with NativeAOT and trimmed applications because it does not rely on reflection.

    Core DOM Types

    TypeDescriptionImplementation
    TomlTableA TOML tableIDictionary<string, object>
    TomlArrayA TOML arrayIList<object?>
    TomlTableArrayAn array of tables ([[array_of_tables]])IList<TomlTable>

    Scalar Values

    Scalar values are mapped to standard .NET types:

    • string $\rightarrow$ string
    • long $\rightarrow$ long
    • double $\rightarrow$ double
    • bool $\rightarrow$ bool
    • Date/Time $\rightarrow$ TomlDateTime
    using Tomlyn.Model;

// Example of the types available
var table = new TomlTable();
var array = new TomlArray();
var tableArray = new TomlTableArray();

  5. Control Property Mapping Order

    main

    The MappingOrder option (or [TomlMappingOrder] attribute) determines the order in which properties appear in the serialized output:

    • Declaration (Default): Properties appear in CLR declaration order.
    • Alphabetical: Properties are sorted alphabetically.
    • OrderThenDeclaration: Properties with [TomlPropertyOrder] appear first, followed by declaration order.
    • OrderThenAlphabetical: Properties with [TomlPropertyOrder] appear first, followed by alphabetical order.

  6. Implement discriminator-based polymorphism

    main

    Tomlyn supports polymorphism using a discriminator key (defaulting to $type) to determine which derived type to instantiate. You can implement this using three different methods depending on your architecture and runtime requirements:

    1. Attribute-based (Standard)

    Use [TomlPolymorphic] and [TomlDerivedType] on your base class. This is the simplest method for most applications.

    2. Cross-project runtime mappings (Reflection)

    If the base type and derived types are in different projects (e.g., a plugin architecture), use TomlPolymorphismOptions.DerivedTypeMappings to register types at runtime using the reflection resolver.

    3. Cross-project source generation (NativeAOT/Trimming)

    For NativeAOT or trimming-safe code, use [TomlDerivedTypeMapping] on a TomlSerializerContext to register mappings during source generation.

    Key Features:

    • Custom Discriminators: Change the key name via TomlPolymorphicAttribute.TypeDiscriminatorPropertyName or TomlPolymorphismOptions.TypeDiscriminatorPropertyName.
    • Default Types: Register a derived type without a discriminator to act as the default. If the TOML lacks a discriminator or has an unknown one, it will deserialize as this type. When serializing the default type, no discriminator is emitted.
    • Integer Discriminators: [TomlDerivedType] accepts an int which is stored as a string in the TOML file.
    • JSON Compatibility: JsonPolymorphicAttribute and JsonDerivedTypeAttribute are supported. If both Toml and Json attributes are present, Toml attributes take precedence.
    // Attribute-based example
[TomlPolymorphic]
[TomlDerivedType(typeof(Cat), "cat")]
[TomlDerivedType(typeof(Dog), "dog")]
public abstract class Animal
{
    public string Name { get; set; } = "";
}

public sealed class Cat : Animal { public bool Indoor { get; set; } }
public sealed class Dog : Animal { public string Breed { get; set; } = ""; }

  7. Understand Tomlyn's architectural layers

    main

    Tomlyn's architecture is layered to provide different levels of abstraction and performance characteristics:

    • TomlLexer: Provides allocation-free, struct-based token iteration. Ideal for low-level tasks like syntax highlighting.
    • TomlParser: A pull-based event stream that avoids creating an intermediate DOM. Useful for custom streaming or validation.
    • TomlSerializer: The primary high-performance API. It reads directly from the parser stream and uses source-generated metadata to avoid reflection.
    • SyntaxParser: Builds a full tree of nodes. This is resource-intensive and should only be used when full-fidelity round-tripping (preserving exact formatting/comments) is required.

  8. Leverage System.Text.Json attributes

    main

    Tomlyn supports most common System.Text.Json.Serialization attributes out of the box. You can use these to control how your .NET types are mapped to TOML. Supported attributes include:

    • [JsonPropertyName]
    • [JsonIgnore]
    • [JsonConstructor]
    • [JsonObjectCreationHandling]

  9. Choose between Syntax and Object mapping

    main

    Decide which API to use based on your requirements:

    • Tomlyn.Syntax / SyntaxParser: Use for lossless round-trips, preserving comments/whitespace, or building tooling that requires source span information.
    • TomlSerializer: Use for standard mapping to .NET objects. Note that this approach does not preserve formatting or comments.

  10. Handle cross-project polymorphism in source generation

    main

    If a polymorphic base type cannot reference its derived types (e.g., they are in different projects), register the derived types on the context using [TomlDerivedTypeMapping]. This avoids the need for separate [TomlSerializable] roots for the derived types.

    Note: If the base type already has [TomlDerivedTypeAttribute] or [JsonDerivedTypeAttribute] registrations, those take precedence over context-level mappings.

    using Tomlyn.Serialization;

[TomlSerializable(typeof(Animal))]
[TomlDerivedTypeMapping(typeof(Animal), typeof(Cat), "cat")]
[TomlDerivedTypeMapping(typeof(Animal), typeof(Dog), "dog")]
internal partial class AnimalContext : TomlSerializerContext
{
}

  11. Use the untyped TOML DOM model

    main

    When you need a dynamic representation of a TOML document without defining Plain Old CLR Objects (POCOs), use the untyped model layer. This model consists of TomlTable, TomlArray, and related container types. It is ideal for:

    • Dynamic data manipulation where the schema is unknown at compile time.
    • Building generic tools that inspect, traverse, or transform TOML values.
    • Scenarios where defining a dedicated class for every TOML structure is impractical.

  12. Handle Object Creation and Population

    main

    Tomlyn follows System.Text.Json semantics for object creation via PreferredObjectCreationHandling:

    • Replace (Default): Writable members are assigned fresh values; read-only members are left untouched.
    • Populate: Reuses existing mutable object and collection instances. Collection population appends items rather than clearing the collection first.

    Usage Levels:

    1. Global: Set via TomlSerializerOptions.PreferredObjectCreationHandling.
    2. Type-level: Use [JsonObjectCreationHandling(JsonObjectCreationHandling.Populate)] on a class.
    3. Member-level: Use [JsonObjectCreationHandling(JsonObjectCreationHandling.Populate)] on a specific property. Property-level attributes override type and global settings.

    Note: Populate semantics do not apply to types deserialized through a parameterized constructor.

    using System.Text.Json.Serialization;
using Tomlyn;

[JsonObjectCreationHandling(JsonObjectCreationHandling.Populate)]
public sealed class ReleaserConfiguration
{
    public List<string> Channels { get; } = ["stable"];
}

var options = new TomlSerializerOptions
{
    PreferredObjectCreationHandling = JsonObjectCreationHandling.Populate,
};