Brighter Command Dispatcher and Processor

repository·master·Indexed 25 days ago

https://github.com/brightercommand/brighter

A Command Dispatcher and Command Processor framework for .NET designed for building loosely coupled applications. It supports in-process messaging and out-of-process messaging for microservices via transports such as RabbitMQ, Kafka, AWS SQS, Azure Service Bus, and Redis Streams. Brighter implements Command-Query Separation (CQS) and provides features including a middleware pipeline for logging and resilience, the Outbox pattern for reliable messaging, and integration with Quartz.NET.

Tokens
160.3K
Snippets
178
Records
743
Agent score
82%

What's inside Brighter

  1. Overview of Paramore.Brighter.AsyncAPI

    master

    The Paramore.Brighter.AsyncAPI NuGet package automatically generates AsyncAPI 3.0 JSON or YAML documents from a Brighter service's runtime configuration. This eliminates the need for manual documentation of messaging contracts by extracting information directly from the service's configuration.

    Key capabilities include:

    • Automatic Contract Collection: Gathers messaging contracts from subscriptions (consumers), publications (producers), and [PublicationTopic]-decorated IRequest types discovered via assembly scanning.
    • Schema Generation: Automatically generates JSON Schema payloads for IRequest types to define message shapes.
    • Pluggable Schema Logic: Allows developers to substitute custom schema generation logic via a pluggable interface.
    • Standard Compliance: Uses the official AsyncAPI .NET SDK as the underlying document model to ensure full support for bindings, tags, and protocol-specific metadata.
  2. Generate AsyncAPI 3.0 documents from Brighter configuration

    master

    The Paramore.Brighter.AsyncAPI package allows you to automatically generate AsyncAPI 3.0 JSON documents based on your Brighter runtime configuration. It scans your subscriptions, publications, and assembly-scanned IRequest types (decorated with [PublicationTopic]) to build a complete specification of your messaging infrastructure.

    Key capabilities include:

    • Subscription Support: Generates receive operations from IAmConsumerOptions.Subscriptions.
    • Publication Support: Generates send operations from IAmAProducerRegistry.Producers.
    • Assembly Scanning: Automatically discovers IRequest types decorated with [PublicationTopic] to include them in the document.
    • Deduplication: Merges channels and messages when the same routing key or IRequest type is found via multiple sources (e.g., both DI and assembly scanning).
  3. Validate Pipeline Assembly and Provider Registration

    master

    Brighter provides a pipeline validation mechanism to ensure that your messaging infrastructure (Producers, Consumers, and Validation Steps) is correctly configured. This includes checking for resolvable transformers (wrap/unwrap) and ensuring that validation providers are properly registered for any declared validation steps.

    Key validation behaviors:

    • Transformer Resolvability: Checks if [Wrap] (on Publications) or [Unwrap] (on Subscriptions) transformers can be resolved by the DI container.
    • Validation Providers: Ensures that if a validation step is present in a pipeline, a corresponding provider is registered.
    • Severity Levels: New validation findings (like missing transformers) are surfaced as ValidationSeverity.Warning and do not block execution when throwOnError: true is used. However, critical issues like a subscription with no registered handler still produce a ValidationSeverity.Error and will block execution.
  4. Detect pipeline misconfigurations with Paramore.Brighter.Analyzer

    master

    The Paramore.Brighter.Analyzer package provides Roslyn analyzer diagnostics to catch common pipeline configuration mistakes at compile time. These diagnostics appear as Warnings in your IDE, providing immediate feedback before runtime.

    Key misconfigurations detected include:

    • Incorrect Backstop Ordering: Detecting when a backstop attribute (e.g., [RejectMessageOnError]) is ordered after a resilience pipeline (e.g., [UseResiliencePipeline]). If the resilience pipeline comes first, it may catch exceptions before the backstop can act, rendering the backstop ineffective.
    • Sync/Async Attribute Mismatches: Detecting when a synchronous attribute is applied to an asynchronous handler (e.g., using [RejectMessageOnError] on a RequestHandlerAsync<T>) or vice versa. In these cases, the attribute is silently ignored at runtime.
    • Subscription Pump Type Mismatches: Detecting when a Subscription configuration (e.g., MessagePumpType.Reactor) is incompatible with the handler type (e.g., an async handler), which would otherwise cause a "no handlers found" error at runtime.

    Note: The analyzer is distributed as a DevelopmentDependency NuGet package.

  5. How message and request scheduling works

    master

    Scheduling in Brighter is split into two parts: the Producer and the Consumer.

    1. Producer

    The producer is responsible for scheduling the message. Depending on the implementation, this might involve creating a timer (In-Memory) or a job (Quartz/Hangfire).

    • IAmAMessageScheduler: Used by IAmAMessageProducer to schedule raw Message objects. This is used if the producer doesn't natively support delays or if a specialized scheduler is provided.
    • IAmARequestScheduler: Used by IAmACommandProcessor to schedule Send, Publish, and Post operations.

    2. Consumer

    Once a message/request is scheduled, the scheduler must eventually route it back to the correct producer. To facilitate this, Brighter uses two internal command types:

    • FireSchedulerMessage: Wraps a Message.
    • FireSchedulerRequest: Wraps a request with metadata like RequestType and RequestData.

    A scheduler job (e.g., a Quartz IJob) handles these by calling ICommandProcessor.SendAsync to re-trigger the original intent.

    public class MessageSchedulerJob(ICommandProcessor processor) : IJob
    {
        public async Task ExecuteAsync(FireSchedulerMessage message)
        {
            await processor.SendAsync(message);
        }
        
        public async Task ExecuteAsync(FireSchedulerRequest message)
        {
            await processor.SendAsync(message);
        }
    }
  6. Verify Outbox pattern usage with SpyCommandProcessor

    master

    When testing handlers that use the Outbox pattern, SpyCommandProcessor tracks deposited requests and their movement to the observation queue:

    1. Depositing: When DepositPost<T>() is called, the request is stored in DepositedRequests and an Id is returned.
    2. Clearing: When ClearOutbox(Id[] ids) is called, the specified requests are moved from the DepositedRequests storage into the internal observation queue.
    3. Observing: Once cleared, these requests can be retrieved using Observe<T>() to verify their contents.

    SpyCommandProcessor also records a CommandType.Clear entry in the RecordedCalls history when the outbox is cleared.

  7. How Brighter handles concurrency and message ordering

    master

    Brighter uses two primary concurrency models to manage message processing:

    1. Reactor Pattern (Single-threaded Message Pump): Uses a single-threaded message pump to preserve message ordering and ensure sequential access to shared resources. This is suitable for blocking I/O scenarios.
    2. Proactor Pattern (Asynchronous Handlers): Supports asynchronous handlers to avoid blocking on I/O, allowing for higher throughput by yielding between performers.

    Brighter implements a custom BrighterSynchronizationContext (an internal fork of AsyncEx.AsyncContext) to ensure that asynchronous continuations are invoked back on the message pump thread, maintaining the integrity of the concurrency model.

  8. How the scheduler flows through the factory chain

    master

    The scheduler is passed downward through the Brighter factory hierarchy via constructor injection. The flow follows this pattern:

    1. DI Container: Resolves IAmAMessageSchedulerFactory (defaults to InMemorySchedulerFactory).
    2. BuildDispatcher: Resolves the IAmAMessageScheduler and injects it into the IAmAChannelFactory (if it implements IAmAChannelFactoryWithScheduler).
    3. Channel Factory: Receives the scheduler and passes it to the IAmAMessageConsumerFactory during construction.
    4. Consumer Factory: Receives the scheduler and passes it to the concrete XxxMessageConsumer.

    This architecture ensures that every consumer has access to the scheduler for features like delayed requeueing without requiring manual wiring for every component.

  9. Understand the Pipeline Description Model

    master

    The HandlerPipelineDescription and TransformPipelineDescription classes provide a read-only, reflection-based view of your configured pipelines. This model is used by the validation and diagnostic tools to ensure pipelines are correctly constructed without the overhead of full handler instantiation.

    Key Components

    HandlerPipelineDescription

    Represents a command or event pipeline.

    • RequestType: The type of the incoming request.
    • HandlerType: The main handler type.
    • BeforeSteps / AfterSteps: Lists of PipelineStepDescription defining the pipeline sequence.

    PipelineStepDescription

    Represents an individual step (attribute) in the pipeline.

    • AttributeType: The type of the attribute (e.g., RejectMessageOnErrorAsyncAttribute).
    • HandlerType: The type of the handler associated with this step.
    • Step: The integer position in the pipeline.
    • Timing: The HandlerTiming (Before/After).
    • InitializerParams: Parameters passed to the handler during initialization (e.g., policy names).

    TransformPipelineDescription

    Represents a transformation pipeline (mapping).

    • RequestType: The type of the request.
    • MapperType: The type of the mapper being used.
    • IsDefaultMapper: Whether it is the default mapper.
    • WrapTransforms: Outgoing transforms.
    • UnwrapTransforms: Incoming transforms.
  10. Fix RabbitMQ (RMQ) Shutdown Deadlock and Connection Pool Race Conditions

    master

    This ADR (Architectural Decision Record) documents the fixes for two critical bugs in the RabbitMQ messaging gateway within Brighter:

    1. Shutdown Deadlock: Resolved a sync-over-async deadlock occurring when a ServiceActivator using the Proactor (async message pump) receives a shutdown signal. The deadlock was caused by the BrighterSynchronizationContext being blocked by synchronous waits on async operations that required the same context to complete.
    2. Connection Pool Race Condition: Resolved a race condition in RmqMessageGatewayConnectionPool where a stale ConnectionShutdown event handler could inadvertently dispose of a new, active connection sharing the same endpoint credentials/host.

    Key architectural changes implemented to resolve these issues:

    • Added IAsyncDisposable to IAmAChannelAsync to allow proper asynchronous cleanup.
    • Updated the Proactor to await Channel.DisposeAsync() upon receiving MT_QUIT.
    • Applied ConfigureAwait(false) to connection pool async methods to prevent continuations from posting back to the BrighterSynchronizationContext.
    • Guarded ConnectionShutdown handlers with ReferenceEquals to ensure only the correct connection instance is handled.
    • Fixed a Time-of-Check to Time-of-Use (TOCTOU) vulnerability in the synchronous RemoveConnection method.
    • Ensured DisposeAsync is the primary path for the RmqMessageGateway async variant.
  11. SQLite Outbox and Inbox Provisioning

    master

    The SQLite backend provides lightweight provisioning for Outbox and Inbox tables:

    • Outbox Provisioning: SqliteOutboxProvisioner creates the outbox table on fresh databases. It uses sqlite_master to check for table existence and pragma_table_info for column/version introspection.
    • Inbox Provisioning: SqliteInboxProvisioner manages inbox table creation and bootstrapping.
    • Concurrency: SQLite relies on implicit file-level locking, so explicit advisory locks are not required for the SQLite implementation.