DotPulsar .NET Client for Apache Pulsar

repository·master·Indexed 42 days ago

https://github.com/apache/pulsar-dotpulsar

The official .NET client library for Apache Pulsar, written entirely in C#. It implements the Pulsar binary protocol to allow .NET applications to produce and consume messages from Pulsar clusters. Targeting .NET Standard 2.0, it is compatible with .NET Framework 4.6.2+, .NET Core, and modern .NET (5, 6, 7, 8+). Supports a wide range of features including various subscription types, TLS/JWT authentication, multiple compression formats (LZ4, ZLIB, ZSTD, SNAPPY), and diverse schema types.

Tokens
1.9K
Snippets
6
Records
12
Agent score
54%

What's inside DotPulsar

  1. Supported features in DotPulsar

    master

    DotPulsar supports a wide range of Apache Pulsar features, including:

    • Core Connectivity: Service discovery, automatic reconnect, TLS connections, and Pulsar Proxy support.
    • Producer Capabilities: Custom metadata, event time, sequence ID, delayed delivery, keys, ordering keys, and partitioned topics.
    • Consumer Capabilities: Various subscription types (exclusive, shared, failover, key shared), initial position, priority levels, multi-topic subscriptions, and compacted topics.
    • Data Handling: Read/Consume/Acknowledge for batched messages, Reader API, and seeking on message-id or publish time.
    • Authentication: TLS, JSON Web Token (JWT), and Custom Authentication.
    • Compression: LZ4, ZLIB, ZSTD, and SNAPPY.
    • Telemetry: Tracing and Metrics support.
    • Schemas: Support for Boolean, Bytes, String (UTF-8, UTF-16, US-ASCII), Integers (INT8-INT64), Floats/Doubles, Time (TimeSpan), Timestamp/Date (DateTime), JSON, and Protobuf.
  2. Compatibility with .NET Framework and .NET Standard

    master

    DotPulsar targets .NET Standard 2.0 to ensure broad compatibility across different .NET runtimes.

    By targeting .NET Standard 2.0, DotPulsar can be used in:

    • .NET Framework applications (version 4.6.2 and up).
    • .NET Core and modern .NET (5, 6, 7, 8+) applications.

    Note that while .NET Standard 2.1 is a newer version, it is not supported by .NET Framework, so DotPulsar uses 2.0 to maintain support for legacy Windows-based software.

  3. IDE Compatibility and Code Formatting

    master

    DotPulsar is designed to be compatible with both Visual Studio and JetBrains Rider. To ensure consistent code formatting and warning behavior across these different IDEs, the project uses a .editorconfig file and *.DotSettings files.

    When working on the codebase, you should use these configuration files to ensure that your IDE's auto-formatting and warning settings align with the project standards. This prevents conflicts when multiple developers use different IDEs.

  4. Produce and Consume messages in DotPulsar

    master

    This example demonstrates the basic workflow of connecting to a Pulsar instance, producing a string message to a topic, and consuming that message using a subscription. Note that topics and subscriptions are created automatically if they do not exist.

    using DotPulsar;
    using DotPulsar.Extensions;
    
    const string myTopic = "persistent://public/default/mytopic";
    
    // connecting to pulsar://localhost:6650
    await using var client = PulsarClient.Builder().Build();
    
    // produce a message
    await using var producer = client.NewProducer(Schema.String).Topic(myTopic).Create();
    await producer.Send("Hello World");
    
    // consume messages
    await using var consumer = client.NewConsumer(Schema.String)
        .SubscriptionName("MySubscription")
        .Topic(myTopic)
        .InitialPosition(SubscriptionInitialPosition.Earliest)
        .Create();
    
    await foreach (var message in consumer.Messages())
    {
        Console.WriteLine($"Received: {message.Value()}");
        await consumer.Acknowledge(message);
    }
  5. Setup auto clean in Visual Studio with ReSharper

    master

    To automatically enforce the DotPulsar coding style on file save using ReSharper in Visual Studio:

    1. Navigate to Extensions > ReSharper > Options...
    2. Under Code Editing, select Code Cleanup and click General.
    3. Check the option: Automatically run cleanup when saving a file (not supported for shared/linked files).
    4. Navigate to Profiles and ensure the DotPulsar: Full Cleanup profile is selected as the default.
    5. Click Save.
  6. Verify Code Formatting in Visual Studio or Rider

    master

    To ensure your IDE is correctly applying the project's formatting and cleanup rules, perform a solution-wide cleanup. The end result should be identical regardless of whether you use Visual Studio or Rider.

    In Visual Studio: Open the solution and run a solution-wide "Analyze and Code Cleanup".

    In Rider: Open the solution and run "Reformat and Cleanup".

  7. Install DotPulsar via NuGet

    master

    To use DotPulsar in your .NET project, install the official NuGet package. This library is a pure C# implementation of the Apache Pulsar binary protocol.

    # Use your preferred NuGet client to install DotPulsar
    dotnet add package DotPulsar
  8. Setup auto clean in JetBrains Rider

    master

    To ensure code consistency when committing changes in Rider, configure the commit process to use the project's specific cleanup profile:

    1. Open the Commit window.
    2. Click the gear icon (settings).
    3. Ensure that only the Cleanup 'DotPulsar: Full Cleanup' profile is checked. You may leave Check TODO enabled or disabled based on your preference.
    // No code snippet available for this setup task
  9. Create a DotPulsar client

    master

    To interact with Apache Pulsar, you must first instantiate a PulsarClient using the PulsarClient.Builder. By default, calling .Build() without additional configuration attempts to connect to pulsar://localhost:6650.

    using DotPulsar;
    using DotPulsar.Extensions;
    
    await using var client = PulsarClient.Builder().Build();
  10. Consume messages from a topic

    master

    You can consume messages by creating a consumer from the client. You must specify a Schema, a SubscriptionName, and a Topic. You can also set the InitialPosition (e.g., SubscriptionInitialPosition.Earliest).

    Messages are retrieved by iterating over the consumer.Messages() stream. After processing a message, you should call consumer.Acknowledge(message) to confirm receipt.

    await using var consumer = client.NewConsumer(Schema.String)
        .SubscriptionName("MySubscription")
        .Topic("persistent://public/default/mytopic")
        .InitialPosition(SubscriptionInitialPosition.Earliest)
        .Create();
    
    await foreach (var message in consumer.Messages())
    {
        Console.WriteLine($"Received: {message.Value()}");
        await consumer.Acknowledge(message);
    }
  11. Produce messages to a topic

    master

    To send data, create a producer using client.NewProducer(Schema). You must specify the target Topic. The Send method returns a messageId upon successful delivery.

    await using var producer = client.NewProducer(Schema.String)
        .Topic("persistent://public/default/mytopic")
        .Create();
    
    var messageId = await producer.Send("Hello World");