hmq Documentation

repository·master·Indexed 23 days ago

https://github.com/fhmq/hmq

hmq is a free, high-performance MQTT broker written in Go. It supports clustering, shared subscriptions, TLS/SSL, and Kafka bridging. The broker features a sequential Access Control List (ACL) system for managing client permissions based on clientid, username, or IP address, and provides extensibility through custom SessionsProvider and TopicsProvider interfaces.

Tokens
4.8K
Snippets
16
Records
25
Agent score
79%

What's inside hmq

  1. Use Shared Subscriptions

    master

    hmq supports shared subscriptions using the $share/<group>/<topic> syntax. This allows multiple clients in a group to share the load of a topic.

    Pattern:

    • Topic format: $share/<group>/topic
    • Subscribe command: mosquitto_sub -t '$share/<group>/topic'
    • Publish command: mosquitto_pub -t 'topic'
    | Prefix              | Examples                                  | Publish                      |
    | -------------------| ----------------------------------------- | ---------------------------- |
    | $share/<group>/topic | mosquitto_sub -t ‘$share/<group>/topic’ | mosquitto_pub -t ‘topic’     |
  2. How ACL rule matching works

    master

    hmq processes ACL rules using a sequential matching strategy. When a client attempts an action, the broker checks the client against the rules in the order they are defined in the configuration file.

    1. The client is compared against Rule 1.
    2. If Rule 1 matches, the action (allow or deny) is applied immediately, and no further rules are checked.
    3. If Rule 1 does not match, the client is compared against Rule 2, and so on.
    4. If no rules match, the default behavior is determined by the final state of the matching logic (typically implicit deny if no allow rules match).
  3. Configure ACL rules in hmq

    master

    Access Control List (ACL) rules in hmq allow you to define permissions for clients based on their identity or network location. Rules are evaluated sequentially (one by one) from top to bottom. The first rule that matches the client's attributes determines whether the action is allowed or denied.

    Rule Format: [Action] | [Type] | [Value] | [PubSub Mode] | [Topics]

    Rule Components:

    • Action: allow or deny.
    • Type: The attribute to match against. Supported types are clientid, username, and ipaddr.
    • Value: The specific value to match (e.g., a specific IP, username, or * for wildcard).
    • PubSub Mode: Defines the permission level. Note: In v2.0, the mapping has changed.
      • 1: sub (Subscribe only)
      • 2: pub (Publish only)
      • 3: pubsub (Both Publish and Subscribe)
    • Topics: A comma-separated list of topics or a single topic. Use # for multi-level wildcards.

    Placeholders:

    • %c: Represents the client's clientid.
    • %u: Represents the client's username.
    allow      ip          127.0.0.1   2     $SYS/#
    allow      clientid    0001        3     #
    allow      username    admin       3     #
    allow      username    joy         3     /test,hello/world 
    allow      clientid    *           1     toCloud/%c
    allow      username    *           1     toCloud/%u
    deny       clientid    *           3     #
  4. Install and run hmq

    master

    To install and run the hmq broker from source, use the following commands:

    $ go get github.com/fhmq/hmq
    $ cd $GOPATH/github.com/fhmq/hmq
    $ go run main.go

    hmq is compatible with the Eclipse Paho client and mosquitto-client.

  5. Set up an hmq Cluster

    master

    To run hmq in a cluster, you must first start a router.

    1. Start the router:
      $ go get github.com/fhmq/router
      $ cd $GOPATH/github.com/fhmq/router
      $ go run main.go
    2. Configure hmq: In your hmq.config file, set the router field to the router's address (e.g., "router": "127.0.0.1:9888").
    1, start router for hmq  (https://github.com/fhmq/router.git)
    	$ go get github.com/fhmq/router
    	$ cd $GOPATH/github.com/fhmq/router
    	$ go run main.go
    2, config router in hmq.config  ("router": "127.0.0.1:9888")
  6. Configure the CSVLog bridge plugin

    master

    The CSVLog plugin allows you to log MQTT messages to a CSV file at runtime for debugging, monitoring, or auditing. Configuration is handled via the csvBridgeConfig structure.

    Key configuration options include:

    • FileName: The output destination. Use a specific path, or one of the special tokens: {LOG} (to the system log), {STDOUT} (to standard output), or {NULL} (to discard messages).
    • LogFileMaxSizeMB: Size in MB for log rotation.
    • LogFileMaxFiles: Number of rotated files to keep.
    • WriteIntervalSecs: Delay before flushing pending writes to the file.
    • CommandTopic: The MQTT topic used to send control commands to the plugin (e.g., bridge/CSVLOG/command).
    • Filters: An array of topic filters. If empty, all messages are bridged (defaults to #).
  7. How hmq handles Shared Subscriptions

    master

    hmq supports MQTT Shared Subscriptions using a specific topic prefix. When a client subscribes to a topic starting with $share/, the broker treats it as a shared subscription group.

    The topic format is: $share/<group_name>/<topic>.

    Example topic: $share/my-group/sensors/temperature

    • groupName: my-group
    • topic: sensors/temperature

    When messages are published to sensors/temperature, the broker will distribute them among the clients subscribed to the shared group using a load-balancing approach (random selection from the queue of subscribers).

  8. Configure hmq using hmq.config

    master

    The broker can be configured using a JSON configuration file. Key fields include:

    • workerNum: Number of workers.
    • port: Client port.
    • host: Network host.
    • cluster: Object containing host and port for cluster communication.
    • router: URL of the cluster router.
    • wsPort / wsPath / wsTLS: Websocket configuration.
    • tlsPort / tlsHost / tlsInfo: TLS/SSL configuration. tlsInfo includes verify, caFile, certFile, and keyFile.
    • plugins: Object defining auth (e.g., authhttp) and bridge (e.g., kafka) plugins.
    {
    	"workerNum": 4096,
    	"port": "1883",
    	"host": "0.0.0.0",
    	"cluster": {
    		"host": "0.0.0.0",
    		"port": "1993"
    	},
    	"router": "127.0.0.1:9888",
    	"wsPort": "1888",
    	"wsPath": "/ws",
    	"wsTLS": true,
    	"tlsPort": "8883",
    	"tlsHost": "0.0.0.0",
    	"tlsInfo": {
    		"verify": true,
    		"caFile": "tls/ca/cacert.pem",
    		"certFile": "tls/server/cert.pem",
    		"keyFile": "tls/server/key.pem"
    	},
    	"plugins": {
    		"auth": "authhttp",
    		"bridge": "kafka"
    	}
    }
  9. Client connection types in hmq

    master

    The hmq broker distinguishes between different types of connections using the following constants. Understanding the client type is crucial as it determines how the broker processes publishes, subscribes, and cluster communications:

    • CLIENT (0): A standard end-user MQTT client.
    • ROUTER (1): Another router within the hmq cluster.
    • REMOTE (2): A router connecting to a different cluster.
    • CLUSTER (3): A connection used for cluster-wide operations.
    const (
    	CLIENT = 0
    	ROUTER = 1
    	REMOTE  = 2
    	CLUSTER = 3
    )
  10. Register and use a custom TopicsProvider

    master

    Custom topic providers are managed via a global registry. You can register your implementation using Register(name string, provider TopicsProvider) and then instantiate a Manager to interact with it using NewManager(providerName string).

    1. Register: Call Register with a unique name and your implementation. Note that registering the same name twice or registering a nil provider will cause a panic.
    2. Initialize: Use NewManager with the registered name to get a *Manager instance.
    3. Operate: Use the Manager methods (Subscribe, Unsubscribe, Retain, etc.) to perform topic operations. The Manager acts as a proxy to your registered provider.
  11. Register and use a custom session provider

    master

    You can register a custom SessionsProvider implementation using the Register function, making it available to the Manager by a unique name. Once registered, you can instantiate a Manager to interact with that provider.

    Note on Panics:

    • Calling Register with a nil provider will cause a panic.
    • Calling Register twice with the same name will cause a panic.
  12. Start the hmq broker via CLI

    master
    The hmq binary can be started by passing configuration arguments to the command line. The broker initializes its configuration from the command-line arguments provided, creates a new broker instance, and starts the service. The process will continue running until it receives an interrupt signal (like Ctrl+C) or a SIGTERM signal, at which point the broker will close gracefully.