AMQPStorm

repository·main·Indexed 42 days ago

https://github.com/eandersson/amqpstorm

A thread-safe Python 3 client and management library for RabbitMQ. It provides the ManagementApi for broker administration—including management of queues, exchanges, users, and virtual hosts—as well as tools for implementing robust AMQP consumers, publishers, and RPC patterns.

Tokens
6.1K
Snippets
13
Records
29
Agent score
52%

What's inside AMQPStorm

  1. Install AMQPStorm via pip

    main

    You can install the core AMQPStorm library using pip. If you require the management API features, install the library with the [management] extra.

    # Install the core library
    pip install amqpstorm
    
    # Install with management dependencies
    pip install amqpstorm[management]
  2. Implement a simple RPC server with AMQPStorm

    main

    To implement a Remote Procedure Call (RPC) server using AMQPStorm, you can use a pattern where the server listens on a specific queue, processes incoming messages, and sends a response back to the reply_to queue specified in the message properties.

    In a typical RPC implementation:

    1. The server consumes messages from a dedicated request queue.
    2. For each message, the server performs the requested logic.
    3. The server publishes the result to the queue name provided in the reply_to property of the incoming message.
    4. The server uses the correlation_id from the request to ensure the client can match the response to the original request.
    import amqpstorm
    
    def on_request(channel, method, properties, body):
        print(f'Received request: {body.decode()}
    ')
        # Perform some work
        response = body.decode().upper()
    
        # Send the response back to the reply_to queue
        channel.basic_publish(
            exchange='',
            routing_key=properties.reply_to,
            body=response.encode(),
            correlation_id=properties.correlation_id
        )
        # Acknowledge the message
        channel.basic_ack(delivery_tag=method.delivery_tag)
    
    def main():
        connection = amqpstorm.Connection({
            'host': 'localhost',
            'port': 5672,
            'login': 'guest',
            'password': 'guest'
        })
        channel = connection.channel()
    
        # Declare the queue for requests
        channel.queue_declare(queue='rpc_queue')
    
        # Set up the consumer
        channel.basic_consume(queue='rpc_queue', on_message_callback=on_request)
    
        print('RPC Server is running. To exit press CTRL+C')
        channel.start_consuming()
    
    if __name__ == '__main__':
        main()
  3. Basic AMQP message publishing example

    main

    To publish a message to RabbitMQ, use amqpstorm.Connection to establish a connection, create a channel using connection.channel(), declare a queue, and then use amqpstorm.Message.create() to prepare and publish the message.

    import amqpstorm
    
    with amqpstorm.Connection('rmq.eandersson.net', 'guest', 'guest') as connection:
        with connection.channel() as channel:
            channel.queue.declare('fruits')
            message = amqpstorm.Message.create(
                channel, body='Hello RabbitMQ!', properties={
                    'content_type': 'text/plain'
                })
            message.publish('fruits')
  4. Implement a Flask RPC Client using AMQPStorm

    main

    You can implement a Remote Procedure Call (RPC) client within a Flask application by using a threaded approach to handle AMQP communication. This allows the Flask web server to remain responsive while waiting for responses from an AMQP-based RPC server.

    In this pattern, the client typically:

    1. Creates an AMQPConnection.
    2. Uses a thread or a non-blocking mechanism to manage the request-response lifecycle.
    3. Sends a message to a specific queue and waits for a response on a callback queue.
    # Note: The actual implementation is contained in examples/flask_threaded_rpc_client.py
    # This pattern involves integrating AMQPStorm within Flask routes or background threads.
  5. Implement a simple AMQP publisher

    main

    To publish messages to an AMQP broker using amqpstorm, you need to establish a connection, open a channel, declare an exchange, and then use the basic_publish method. This pattern is used for sending data to specific routing keys or exchanges in a standard AMQP workflow.

    import amqpstorm
    
    # Establish connection and open a channel
    with amqpstorm.Connection('localhost', username='guest', password='guest') as connection:
        with connection.channel() as channel:
            # Declare an exchange (e.g., a direct exchange named 'my_exchange')
            channel.exchange_declare(exchange='my_exchange', exchange_type='direct')
    
            # Publish a message to the exchange with a specific routing key
            channel.basic_publish(
                exchange='my_exchange',
                routing_key='my_routing_key',
                body='Hello, AMQP!'
            )
  6. Implement a simple AMQP consumer

    main

    To consume messages from an AMQP queue using amqpstorm, you need to establish a connection, create a channel, and then use the consume method on the channel. The consume method accepts a callback function that is invoked whenever a message is received.

    Key steps:

    1. Create an AMQPConnection with your broker credentials.
    2. Open an AMQPChannel from the connection.
    3. Call channel.consume(queue_name, callback) to start listening for messages.
    4. The callback function receives the AMQPMessage object, which contains the body (the message payload) and other metadata.
    import amqpstorm
    
    def callback(message):
        print(f'Received message: {message.body}')
        message.ack()
    
    connection = amqpstorm.Connection(host='localhost', username='guest', password='guest')
    connection.connect()
    
    channel = connection.channel()
    channel.queue.declare(name='test_queue')
    
    # Start consuming from the queue
    channel.consume('test_queue', callback)
    
    # Keep the script running to listen for messages
    connection.process_events()
  7. Create a RabbitMQ User

    main

    You can programmatically create a RabbitMQ user using the management module. This requires the requests library to interact with the RabbitMQ Management HTTP API.

    Refer to the example below for the implementation pattern.

    # Example implementation from examples/management/create_user.py
    # Note: This requires amqpstorm[management] to be installed
  8. Implement a robust AMQP consumer

    main

    To build a resilient AMQP consumer that can handle connection interruptions and unexpected errors, you should implement a loop that manages the connection and channel lifecycle. A robust pattern involves:

    1. Connection Management: Wrapping the AMQPStorm.Connection instantiation and the connection.start() call in a retry loop.
    2. Channel Management: Creating the channel within the connection context.
    3. Consumer Loop: Using channel.consume() to start receiving messages.
    4. Error Handling: Catching exceptions within the message processing logic to prevent the entire consumer from crashing, and catching connection-level exceptions to trigger a reconnection attempt.
    5. Graceful Shutdown: Using try...finally blocks to ensure connection.stop() is called, which closes the connection and cleans up resources.
    import AMQPStorm
    import time
    
    def run_consumer():
        while True:
            connection = None
            try:
                # 1. Establish connection
                connection = AMQPStorm.Connection(
                    host='localhost',
                    login='guest',
                    password='guest'
                )
                
                with connection:
                    channel = connection.channel()
                    
                    # 2. Setup queue and consumer
                    channel.queue_declare(queue='robust_queue')
                    
                    print('[*] Waiting for messages. To exit press CTRL+C')
                    
                    # 3. Consume messages
                    for method, properties, body in channel.consume('robust_queue'):
                        try:
                            print(f' [x] Received {body!r}')
                            # Simulate work
                            time.sleep(1)
                            # 4. Acknowledge message
                            channel.basic_ack(delivery_tag=method.delivery_tag)
                        except Exception as e:
                            print(f' [!] Error processing message: {e}')
                            # 5. Nack message on processing error
                            channel.basic_nack(delivery_tag=method.delivery_tag, requeue=True)
    
            except (AMQPStorm.exceptions.AMQPError, ConnectionError) as e:
                print(f' [!] Connection error: {e}. Retrying in 5 seconds...')
                time.sleep(5)
            except KeyboardInterrupt:
                print(' [!] Stopping consumer...')
                break
            finally:
                if connection:
                    connection.stop()
    
    if __name__ == '__main__':
        run_consumer()
  9. Establish an SSL connection using an SSLContext

    main

    To establish a secure SSL connection with an AMQP broker using amqpstorm, you can provide a pre-configured ssl.SSLContext object. This is useful when you need fine-grained control over certificate validation, loading CA certificates, or configuring specific SSL/TLS settings. Pass the context object to the ssl_context parameter when initializing the AMQPConnection.

    import ssl
    from amqpstorm import AMQPConnection
    
    # Create an SSL context
    ssl_context = ssl.create_default_context()
    # Optionally load specific CA certificates or configure settings
    # ssl_context.load_verify_locations(cafile='path/to/ca_certificate.pem')
    
    # Connect using the SSL context
    with AMQPConnection(host='your-broker-host', port=5671, ssl_context=ssl_context) as connection:
        channel = connection.channel()
        # Perform AMQP operations
        print("Connected securely via SSL")