rabbitpy

repository·main·Indexed 42 days ago

https://github.com/gmr/rabbitpy

A pure-Python, thread-safe, and minimalistic RabbitMQ client library providing an object-oriented API for connections, channels, queues, exchanges, and messages. Version 3.0.2 requires Python 3.11+ and RabbitMQ 3.8+.

Tokens
17K
Snippets
60
Records
75
Agent score
54%

What's inside rabbitpy

  1. Thread-safety and channel usage in rabbitpy

    main

    The core network communication module (rabbitpy.io.IO) is thread-safe and runs as a daemonic Python thread. However, rabbitpy does not protect against cross-thread channel utilization.

    Because of how channel events are managed, you must restrict each channel to a single thread. Sharing a single channel across multiple threads can lead to corrupted channel state in the AMQP protocol. To use multiple threads concurrently, you should create a separate channel for each thread using the connection object.

  2. Use multiple threads for publishing and consuming

    main

    To perform concurrent operations like publishing and consuming in separate threads, pass the rabbitpy.Connection object to your worker threads. Inside each thread, use connection.channel() to create a thread-local channel. This ensures that each thread operates on its own isolated AMQP channel, maintaining protocol safety.

    import rabbitpy
    import threading
    
    EXCHANGE = 'threading_example'
    QUEUE = 'threading_queue'
    ROUTING_KEY = 'test'
    MESSAGE_COUNT = 100
    
    
    def consumer(connection):
        """Consume MESSAGE_COUNT messages then exit."""
        received = 0
        # Create a dedicated channel for this thread
        with connection.channel() as channel:
            for message in rabbitpy.Queue(channel, QUEUE):
                print(message.body)
                message.ack()
                received += 1
                if received == MESSAGE_COUNT:
                    break
    
    
    def publisher(connection):
        """Publish MESSAGE_COUNT messages."""
        # Create a dedicated channel for this thread
        with connection.channel() as channel:
            for index in range(MESSAGE_COUNT):
                message = rabbitpy.Message(channel, f'Message #{index}')
                message.publish(EXCHANGE, ROUTING_KEY)
    
    
    with rabbitpy.Connection() as connection:
        # Setup topology in the main thread
        with connection.channel() as channel:
            exchange = rabbitpy.Exchange(channel, EXCHANGE)
            exchange.declare()
    
            queue = rabbitpy.Queue(channel, QUEUE)
            queue.declare()
            queue.bind(EXCHANGE, ROUTING_KEY)
    
        kwargs = {'connection': connection}
    
        consumer_thread = threading.Thread(target=consumer, kwargs=kwargs)
        consumer_thread.start()
    
        publisher_thread = threading.Thread(target=publisher, kwargs=kwargs)
        publisher_thread.start()
    
        consumer_thread.join()
  3. Publish a message to RabbitMQ

    main

    To publish a message, establish a Connection using an AMQP URL, open a channel, create a Message object associated with that channel, and call .publish(exchange, routing_key). Using context managers (with statements) ensures that connections and channels are properly closed.

    import rabbitpy
    
    with rabbitpy.Connection('amqp://guest:guest@localhost:5672/%2f') as conn:
        with conn.channel() as channel:
            message = rabbitpy.Message(channel, 'Hello, RabbitMQ!')
            message.publish('my-exchange', 'routing-key')
  4. Consume messages from a queue

    main

    To consume messages, establish a Connection and a channel, then instantiate a Queue object with the target queue name. You can iterate directly over the Queue object to receive messages. After processing a message, call message.ack() to acknowledge it.

    import rabbitpy
    
    with rabbitpy.Connection('amqp://guest:guest@localhost:5672/%2f') as conn:
        with conn.channel() as channel:
            queue = rabbitpy.Queue(channel, 'my-queue')
            for message in queue:
                print(message.body)
                message.ack()
  5. Handle rabbitpy and AMQP exceptions

    main

    rabbitpy uses two categories of exceptions:

    1. rabbitpy Exceptions: Specific to the library's logic and state.
    2. AMQP Exceptions: Raised when RabbitMQ sends a channel or connection close with a specific AMQP reply code (e.g., when a precondition fails during queue declaration).

    You should wrap your connection and channel operations in try/except blocks to handle these errors, especially when performing operations like declaring queues with specific parameters that might conflict with existing ones.

    import rabbitpy
    
    try:
        with rabbitpy.Connection() as connection:
            with connection.channel() as channel:
                queue = rabbitpy.Queue(channel, 'exception-test')
                queue.durable = True
                queue.declare()
                # Changing parameters and re-declaring will trigger an exception
                queue.durable = False
                queue.declare()  # raises rabbitpy.exceptions.AMQPPreconditionFailed
    except rabbitpy.exceptions.AMQPPreconditionFailed:
        print('Queue already exists with different parameters')
  6. Manage RabbitMQ connections with Connection objects

    main

    rabbitpy.Connection objects provide a thread-safe connection to RabbitMQ used for authentication and executing channel-based RPC commands. You can specify connection details using the AMQP URI syntax.

    Connections can be managed in two ways:

    1. Manual management: Create the object and call .close() manually.
    2. Context manager: Use the with statement to ensure the connection is automatically closed when the block exits.

    Note on Heartbeats: If heartbeats are enabled (default is 300 seconds), a ConnectionResetException will be raised if RabbitMQ does not send a heartbeat within $\ge 2$ heartbeat intervals.

    # Manual management
    conn = rabbitpy.Connection('amqp://guest:guest@localhost:5672/%2F')
    conn.close()
    
    # Context manager (Recommended)
    with rabbitpy.Connection('amqp://guest:guest@localhost:5672/%2F') as conn:
        with conn.channel() as channel:
            # use channel
            pass
  7. Use the Message class to publish and consume messages

    main

    The Message class is the primary interface for interacting with RabbitMQ messages. It is used both to create messages for publishing and to represent messages received from the broker.

    Key Capabilities

    • Publishing: Send messages to an exchange with a routing key.
    • Consuming: Access metadata like delivery_tag, routing_key, and exchange from received messages.
    • Acknowledgment: Use ack(), nack(), or reject() to manage message lifecycle after consumption.
    • Serialization: Automatically serializes dict or list bodies to JSON and sets content_type to application/json.
    • Opinionated Mode: When opinionated=True, the class automatically generates a message_id (UUID) and a timestamp if they are not provided.

    Message Properties

    You can pass a dictionary of properties to the constructor. Note that the AMQP type property is referred to as message_type in rabbitpy to avoid Python keyword conflicts.

    Available properties:

    • app_id
    • content_type
    • content_encoding
    • correlation_id
    • delivery_mode
    • expiration
    • headers
    • message_id
    • message_type
    • priority
    • reply_to
    • timestamp
    • user_id
    # Example: Creating and publishing a message
    from rabbitpy import Message
    
    msg = Message(
        channel=channel,
        body_value={'key': 'value'},
        properties={'app_id': 'my_app'},
        opinionated=True
    )
    msg.publish(exchange=my_exchange, routing_key='my_key')
  8. How transactions work in rabbitpy

    main

    Transactions in rabbitpy allow for grouping multiple AMQP commands (like publish and ack) into a single atomic unit.

    Key Behaviors:

    • Atomicity: Servers aim to ensure all requests in a transaction succeed or fail together. However, transactions covering multiple queues may be non-atomic because queue creation/destruction is asynchronous and not part of the transaction.
    • Lifecycle: A transaction is activated via Tx.select(). Once commit() or rollback() is called, a new transaction is immediately started on that channel.
    • Error Handling: If an exception occurs while using Tx as a context manager, it attempts a rollback(). If the channel is already closed, it logs a warning and skips the rollback to avoid secondary errors.
  9. Configure SSL certificate validation levels

    main

    When using amqps URLs, you can specify the level of server certificate validation using the verify or ssl_validation query parameter. The allowed values are:

    • ignore: Ignore the certificate if provided (default).
    • optional: The certificate is validated if provided.
    • required: The certificate is required and must be validated.

    These map to Python's ssl.VerifyMode constants.

    # Using 'required' validation in a URL
    url = "amqps://guest:guest@localhost:5671/?verify=required"
    args = parse(url)
  10. Use the core rabbitpy API (Connection, Channel, Exchange, Queue)

    main

    The core API for rabbitpy follows a hierarchical structure: Connection -> Channel -> Exchange/Queue.

    • Connection: Manages the physical connection to the RabbitMQ broker.
    • Channel: A virtual connection within a Connection where most AMQP operations occur.
    • Exchange: An entity that receives messages and routes them to queues. Supported types include DirectExchange, FanoutExchange, HeadersExchange, and TopicExchange.
    • Queue: A buffer that stores messages.
    • Message: The data unit being sent or received.
    • AMQP: The underlying AMQP implementation layer.
    from rabbitpy import Connection, Channel, DirectExchange, Queue, Message
    
    # Typical workflow pattern:
    connection = Connection(host='localhost')
    with connection:
        channel = connection.channel()
        exchange = channel.exchange('my_exchange', type='direct')
        queue = channel.queue('my_queue')
        queue.bind(exchange)
        
        message = Message(body='Hello World')
        exchange.publish(message)
        
        # Consuming
        for msg in queue.consume():
            print(msg.body)
  11. Run RabbitMQ via Docker Compose

    main

    To run a RabbitMQ instance compatible with rabbitpy, use the following compose.yml configuration. This setup uses the rabbitmq:4-management image, which includes the management plugin for web-based monitoring.

    Exposed Ports:

    • 5672: Default AMQP port for message communication.
    • 15672: Management UI port for accessing the RabbitMQ dashboard.

    Healthcheck:

    • Uses rabbitmq-diagnostics -q check_running to ensure the service is ready for connections.
    services:
      rabbitmq:
        image: rabbitmq:4-management
        ports:
          - 5672
          - 15672
        healthcheck:
          test: rabbitmq-diagnostics -q check_running