DOCSEARCHbyAlgolia
Submit Docs

.NEXT (dotNext) Libraries

repository·master·Indexed 23 days ago

https://github.com/dotnet/dotnext

A suite of high-performance, 100% managed .NET libraries optimized for low-allocation, high-throughput scenarios such as distributed systems, database engines, and microservices. Key offerings include the .NEXT Cluster Programming Suite for building clustered microservices with Raft consensus and HyParView gossip protocols, as well as specialized tools for memory and IO, concurrency, metaprogramming, and caching.

Tokens
3.3K
Snippets
2
Records
24
Agent score
83%

What's inside dotNext

  1. Overview of .NEXT (dotNext) libraries

    master

    .NEXT (dotNext) is a collection of high-performance, 100% managed libraries designed for scenarios requiring near-zero memory allocation and high flexibility. It is optimized for high-load microservices, database engines, actors, and distributed applications.

    Key feature areas include:

    • Memory & IO: Buffer manipulations, string building, Base64 streaming encoder/decoder, and IBufferWriter<T> capacity limiting via the Limit extension method.
    • Low-level Memory: Alignment detection, type-safe unmanaged allocators, and Span<T> manipulations.
    • Concurrency & Threading: Async locks, Atomic<T> types, and advanced GC notifications.
    • Distributed Systems: Raft Consensus Algorithm, Write Ahead Log (WAL), TCP multiplexing, and HyParView gossip protocol.
    • Metaprogramming: Extended LINQ Expression Trees and dynamic code generation with async lambdas.
    • Caching: Async-friendly SIEVE cache.

  2. Overview of the .NEXT Cluster Programming Suite

    master

    .NEXT Cluster Programming Suite is a collection of libraries designed for building clustered microservices. It provides implementations of consensus and membership protocols to manage state and communication across multiple nodes.

    There are two primary packages:

    1. DotNext.Net.Cluster: The core library containing the cluster programming model, a transport-agnostic implementation of the Raft consensus algorithm, TCP and UDP transport bindings for Raft, and a transport-agnostic implementation of the HyParView membership protocol for Gossip-based messaging.

    2. DotNext.AspNetCore.Cluster: A concrete implementation of Raft and HyParView specifically designed for building ASP.NET Core applications.

  3. Use Raft for consensus and log replication

    master

    The Raft implementation in .NET Cluster allows for replicated state machines and consistent log entries across a cluster. It is highly extensible and supports various network transports and storage mechanisms.

    Key Features:

    • Network Transports: Supports TCP, UDP, HTTP 1.1, HTTP/2, HTTP/3, and custom transports built on the ASP.NET Core Connections abstraction.
    • Security: TLS support is available for TCP and HTTP 1.1, HTTP/2, and HTTP/3.
    • Persistence: Includes a high-performance, general-purpose Persistent Write-Ahead Log (WAL) that supports log compaction.
    • Extensibility: You can implement custom write-ahead logs, custom network transports, and custom cluster member discovery mechanisms.
    • Deployment: Designed to be friendly to Docker, LXC, and Windows containers.

  4. Use HyParView for Gossip-based messaging and peer discovery

    master

    HyParView is a membership protocol used for Gossip-based messaging and peer discovery within a cluster. It allows nodes to maintain a view of the cluster and communicate efficiently via gossip.

    Key Features:

    • Network Transports: Supports HTTP 1.1, HTTP/2, and HTTP/3.
    • Security: TLS support is available for HTTP 1.1, HTTP/2, and HTTP/3.
    • Integration: Provides tight integration with the ASP.NET Core framework.
    • Capabilities: Supports broadcasting within the cluster.

  5. Implement a connection-oriented Raft Client

    master

    The DotNext.Net.Cluster.Consensus.Raft.NetworkTransport.ConnectionOriented.Client is an abstract base class used to implement a client for Raft cluster members using connection-oriented transport.

    To use this class, you must inherit from it and provide an implementation for the ConnectAsync method. The class manages connection lifecycle, request/response synchronization via an AsyncExclusiveLock, and automatic connection timeouts.

    Key properties and methods:

    • ConnectTimeout: A configurable TimeSpan that determines how long to wait for a connection. It must be greater than TimeSpan.Zero.
    • CancelPendingRequestsAsync(): An asynchronous method to cancel all pending requests and dispose of the current connection context.
    • RequestAsync<TResponse, TExchange>(...): An internal method used by the implementation to execute requests using a specific exchange logic defined by IClientExchange<TResponse>.

  6. Implement a connection-oriented Raft Server

    master

    To use the Raft network transport in a connection-oriented manner, you must implement the Server abstract class. The Server class handles the low-level orchestration of Raft protocol messages (such as Vote, AppendEntries, and InstallSnapshot) by routing them to an ILocalMember implementation.

    When implementing your custom Server, you are responsible for:

    1. Providing an Address (the EndPoint the server listens on).
    2. Implementing StartAsync(CancellationToken token) to begin accepting connections.
    3. Defining a BufferAllocator to manage memory for incoming requests.
    4. Setting a ReceiveTimeout to control how long the server waits for data.

    The server uses an ILocalMember to execute the actual Raft logic (voting, log appending, etc.) once a message is parsed from the ProtocolStream.

  7. Implement a custom Raft CommandInterpreter

    master

    To handle Raft log entries in a custom state machine, you can implement a CommandInterpreter. There are two ways to define command handlers:

    1. Inheritance: Create a class that inherits from CommandInterpreter. Define your command handlers as public instance methods marked with the [CommandHandler] attribute. The interpreter will automatically discover these methods during construction.
    2. Builder Pattern: Use the CommandInterpreter.Builder (implied by the class documentation) to manually register handlers.

    Handler Method Signatures

    When using inheritance, your handler methods must return ValueTask and follow one of these two patterns:

    • Standard Handler: ValueTask HandlerName(TCommand command, CancellationToken token)
    • Contextual Handler: ValueTask HandlerName(TCommand command, object context, CancellationToken token)

    Where TCommand is a type implementing ICommand<TCommand> and provides a unique Id via its interface.

  8. Use the Limit extension method for IBufferWriter<T>

    master
    In DotNext.IO (version 6.5.0+), the Limit extension method is available for all classes implementing the IBufferWriter<T> interface. This method returns a wrapper that limits the capacity of the underlying writer.

  10. Check .NEXT compatibility and support levels

    master

    The .NEXT libraries follow Semantic Versioning 2.0. Compatibility with .NET runtimes varies by major version:

    Version.NET compatibilitySupport Level
    0.x.NET Standard 2.0unsupported
    1.x.NET Standard 2.0unsupported
    2.x.NET Standard 2.1unsupported
    3.x.NET Standard 2.1, .NET 5unsupported
    4.x.NET 6unsupported
    5.x.NET 8bug and security fixes only
    6.x.NET 10active development

  12. Register Raft command handlers with CommandInterpreter.Builder.Add

    master

    The Add<TCommand> method allows you to register handlers for commands that implement ICommand<TCommand>. There are three overloads available:

    1. Using a CommandHandler: Add<TCommand>(CommandHandler<TCommand> handler)
    2. Using a simple delegate: Add<TCommand>(Func<TCommand, CancellationToken, ValueTask> handler)
    3. Using a delegate with state: Add<TCommand>(Func<TCommand, object?, CancellationToken, ValueTask> handler)

    Note: If the command type TCommand has IsSnapshot set to true, the builder will automatically track its ID as the snapshotCommandId for the interpreter.