IdempotentAPI

repository·master·Indexed 43 days ago

https://github.com/ikyriak/idempotentapi

A .NET library for ASP.NET Core that ensures non-idempotent HTTP methods (POST and PATCH) can be safely retried without side effects. It uses an Idempotency-Key header and request hashing to cache initial responses and prevent duplicate processing. The library supports Controller-Based and Minimal APIs, with extensible caching options via IDistributedCache or FusionCache, and distributed locking support through RedLock.net and MadelsonDistributedLock.

Tokens
5K
Snippets
9
Records
15
Agent score
6%

What's inside IdempotentAPI

  1. Overview of IdempotentAPI

    master

    IdempotentAPI is an open-source NuGet library designed for ASP.NET Core. It provides an attribute (filter) that handles HTTP write operations (POST and PATCH) to ensure they are processed correctly even if a client sends the same request multiple times.

    By using an idempotency-key and the request data, the library ensures that operations affecting state are only executed once per unique request, making your Web APIs resilient to transient faults and network issues.

  2. What is Idempotency and why use IdempotentAPI?

    master

    Idempotency ensures that an API operation can be applied multiple times without changing the result beyond the initial application. This is critical for building fault-tolerant distributed systems where network failures or client retries might cause the same request (like a payment) to be sent multiple times.

    IdempotentAPI is a .NET library that provides an ASP.NET Core attribute to handle HTTP write operations (POST and PATCH) so they only execute once for a given combination of an idempotency key and request data. It prevents accidental duplicate processing by caching the initial response and returning it for subsequent identical requests.

  3. How IdempotentAPI works

    master

    The library operates as an ASP.NET Core filter. The workflow is as follows:

    1. Client Request: The consumer sends a request including an Idempotency-Key header (default name: IdempotencyKey).
    2. Server Check: The library checks if that unique identifier has been used previously for that specific request.
    3. Cache Hit: If the key exists, the library returns the cached response (including HTTP status code, body, and headers) without executing the controller logic.
    4. Cache Miss: If the key is new, the library executes the controller, then saves the resulting response along with the unique identifier and a request hash-key to the cache.

    To prevent misuse, the library validates a request hash-key. This ensures that if a client sends the same Idempotency-Key but with different request body data, the cached response is not returned, preventing accidental data mismatch.

  4. Register IdempotentAPI Core services

    master

    Register the core services in your Program.cs or Startup.cs depending on whether you are using Controller-Based APIs or Minimal APIs.

    For Controller-Based APIs

    You can either use the default registration or provide custom IIdempotencyOptions to set global defaults.

    For Minimal APIs

    You can define default options or implement IIdempotencyOptionsProvider to provide options dynamically (e.g., per endpoint).

    // For Controller-Based APIs
    // Option 1: Use the options defined per Controller (i.e., in the attributes).
    services.AddIdempotentAPI();
    
    // Option 2: Register the `IIdempotencyOptions` that will enable the use of the `[Idempotent(UseIdempotencyOption = true)]` option to set global default options.
    services.AddIdempotentAPI(idempotencyOptions);
    
    // OR
    
    // For Minimal APIs:
    // Option 1: Define the default options:
    builder.Services.AddIdempotentMinimalAPI(new IdempotencyOptions());
    
    // Option 2: Configure the idempotent options by implementing the `IIdempotencyOptionsProvider` to provide the `IIdempotencyOptions` based on our needs (e.g., per endpoint).
    builder.Services.AddIdempotentMinimalAPI(new IdempotencyOptionsProvider());
  5. Best practices for Idempotent Web Consumers

    master

    To implement an idempotent consumer (the client side), follow these recommendations:

    • Generate Unique Keys: Use V4 UUIDs for the Idempotency-Key header (e.g., "07cd2d27-e0dc-466f-8193-28453e9c3023").
    • Retry Strategy: Use techniques like exponential backoff and random jitter to manage continuous requests and avoid overwhelming the server during recovery.
  6. Enable Idempotency in Controllers and Minimal APIs

    master

    You can apply idempotency to your API endpoints using attributes (for Controllers) or endpoint filters (for Minimal APIs).

    Controller-Based APIs

    • Class Level: Adding [Idempotent(Enabled = true)] to a controller class makes all POST and PATCH actions idempotent.
    • Action Level: Adding [Idempotent] to a specific method allows fine-grained control and custom options per action.

    Minimal APIs

    Install the IdempotentAPI.MinimalAPI package and use the .AddEndpointFilter<IdempotentAPIEndpointFilter>() extension method on your routes.

    [ApiController]
    [Idempotent(Enabled = true)]
    public class SimpleController : ControllerBase { ... }
    
    // Controller: Action Level
    [HttpPost]
    [Idempotent(ExpireHours = 48)]
    public IActionResult Post([FromBody] SimpleRequest request) { ... }
    
    // Minimal API
    app.MapPost("/example", () => Results.Ok(new ResponseDTO()))
       .AddEndpointFilter<IdempotentAPIEndpointFilter>();
  7. Prepare DTOs for Idempotency

    master

    Response Data Transfer Objects (DTOs) must be decorated with the [Serializable] attribute so they can be serialized and cached by the library.

    
    [Serializable]
    public class SimpleResponse
    {
        public int Id { get; set; }
        public string Message { get; set; }
        public DateTime CreatedOn { get; set; }
    }
  8. Configure Distributed Locks for Cluster Environments

    master

    If running in a cluster (multiple server instances), you should register a distributed lock provider to ensure idempotency across the cluster.

    Supported implementations:

    • RedLock.net: Uses the Redis Redlock algorithm.
    • MadelsonDistributedLock: Supports multiple technologies including Redis, SqlServer, Postgres, and more.
    List<DnsEndPoint> redisEndpoints = new List<DnsEndPoint>()
    {
    	new DnsEndPoint("localhost", 6379)
    };
    services.AddRedLockNetDistributedAccessLock(redisEndpoints);
    
    // Choice 3: madelson/DistributedLock (Multiple Technologies)
    var redicConnection = ConnectionMultiplexer.Connect("localhost:6379");
    services.AddSingleton<IDistributedLockProvider>(_ => new RedisDistributedSynchronizationProvider(redicConnection.GetDatabase()));
    services.AddMadelsonDistributedAccessLock();
  9. Configure Caching Storage for Idempotency

    master

    IdempotentAPI requires an implementation of IIdempotencyCache to store data. You can choose between two main implementations:

    Choice 1: DistributedCache (Default)

    Uses IDistributedCache as the primary storage. This is suitable for most standard scenarios.

    Choice 2: FusionCache

    Provides advanced features like 2nd-level caching (using IDistributedCache as a secondary layer), Fail-Safe, and Soft/Hard timeouts. To use 2nd-level cache, you must also register a FusionCache Serialization provider (e.g., NewtonsoftJson or SystemTextJson).

    // Register an implementation of the IDistributedCache (e.g., Memory Cache).
    services.AddDistributedMemoryCache();
    // Register the IdempotentAPI.Cache.DistributedCache.
    services.AddIdempotentAPIUsingDistributedCache();
    
    // Choice 2: FusionCache
    // Register the IdempotentAPI.Cache.FusionCache.
    services.AddIdempotentAPIUsingFusionCache();
    
    // Example: FusionCache with 2nd-level cache using Redis
    services.AddStackExchangeRedisCache(options =>
    {
        options.Configuration = "YOUR CONNECTION STRING";
    });
    services.AddFusionCacheNewtonsoftJsonSerializer();
    services.AddIdempotentAPIUsingFusionCache();
  10. Install Distributed Locking NuGet Packages

    master

    To handle distributed access locks, you can choose from several implementations provided by the IdempotentAPI project:

    • IdempotentAPI.DistributedAccessLock.Abstractions: Contains the core distributed access lock definitions.
    • IdempotentAPI.DistributedAccessLock.MadelsonDistributedLock: Implementation using Madelson's DistributedLock.
    • IdempotentAPI.DistributedAccessLock.RedLockNet: Implementation using RedLockNet.
    | Package Name | Description | Release |
    | ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
    | [IdempotentAPI.DistributedAccessLock.Abstractions](https://www.nuget.org/packages/IdempotentAPI.DistributedAccessLock.Abstractions/) | The distributed access lock definition of the IdempotentAPI project. | [![Nuget](https://img.shields.io/nuget/v/IdempotentAPI.DistributedAccessLock.Abstractions)](https://www.nuget.org/packages/IdempotentAPI.DistributedAccessLock.Abstractions/) |
    | [IdempotentAPI.DistributedAccessLock.MadelsonDistributedLock](https://www.nuget.org/packages/IdempotentAPI.DistributedAccessLock.MadelsonDistributedLock/) | The Madelson DistributedLock implementation for the definition of the IdempotentAPI.DistributedAccessLock. | [![Nuget](https://img.shields.io/nuget/v/IdempotentAPI.DistributedAccessLock.MadelsonDistributedLock)](https://www.nuget.org/packages/IdempotentAPI.DistributedAccessLock.MadelsonDistributedLock/) |
    | [IdempotentAPI.DistributedAccessLock.RedLockNet](https://www.nuget.org/packages/IdempotentAPI.DistributedAccessLock.RedLockNet/) | The RedLockNet implementation for the definition of the IdempotentAPI.DistributedAccessLock. | [![Nuget](https://img.shields.io/nuget/v/IdempotentAPI.DistributedAccessLock.RedLockNet)](https://www.nuget.org/packages/IdempotentAPI.DistributedAccessLock.RedLockNet/) |
  11. Reference: Idempotent Attribute Options

    master

    The [Idempotent] attribute allows you to configure the behavior of the idempotency logic. Use these options to control expiration, header names, and locking behavior.

    | Name | Type | Default Value | Description |
    | --------------------------- | ----------------- | ----------------- | ----------------- |
    | Enabled | bool | true | Enable or Disable the Idempotent operation on an API Controller’s class or method. |
    | ExpireHours (Obsolete) | int | 24 hours | The retention period (in hours) of the idempotent cached data. This option will be deprecated. |
    | ExpiresInMilliseconds | double | 24 hours | The retention period (in milliseconds) of the idempotent cached data. |
    | HeaderKeyName | string | IdempotencyKey | The name of the Idempotency-Key header. |
    | DistributedCacheKeysPrefix | string | IdempAPI_ | A prefix for the DistributedCache key names. |
    | CacheOnlySuccessResponses | bool | True | When true, only the responses with 2xx HTTP status codes will be cached. |
    | DistributedLockTimeoutMilli | double | NULL | The time the distributed lock will wait for the lock to be acquired (in milliseconds). This is Required when a `IDistributedAccessLockProvider` is provided. |
    | IsIdempotencyOptional | bool | False | Set the idempotency as optional to be introduced to existing endpoints easily (which should be backward compatible). |