repository·master·Indexed 23 days ago
https://github.com/python-zk/kazooA high-level Python client library for Apache ZooKeeper that provides advanced abstractions and a developer-friendly API for distributed coordination tasks. It includes the KazooClient for connection management, a comprehensive set of exceptions, and distributed recipes such as Election, Lock, Barrier, Counter, and TreeCache.
The kazoo.recipe.lock module provides distributed synchronization primitives for coordinating access to shared resources across multiple processes or nodes.
Available primitives include:
Lock: A standard mutual exclusion lock.ReadLock: A lock that allows multiple readers but requires exclusive access for writers (shared/exclusive locking).WriteLock: An exclusive lock used in conjunction with ReadLock to ensure no other readers or writers are active.Semaphore: A primitive that limits the number of concurrent holders of a resource.kazoo.security module provides tools for managing Access Control Lists (ACLs) and identities within ZooKeeper. It allows you to define permissions for ZNodes using ACL objects and Id objects, which are essential for securing your ZooKeeper paths.The client transitions through several states, which can be checked via the zk.state property.
LOST: The initial state or a state reached when a session expires or the client is stopped. Ephemeral nodes are removed by Zookeeper in this state.CONNECTED: A successful connection is established.SUSPENDED: Connection issues occurred or the node is no longer part of the quorum. Commands cannot currently be run. You should pause operations that require agreement (like Locks) during this state.Valid Transitions:
LOST -> CONNECTED: New connection or recovery.CONNECTED -> SUSPENDED: Connection loss occurred.CONNECTED -> LOST: Occurs if invalid credentials are provided after connection.SUSPENDED -> LOST: Connection resumed but session expired.SUSPENDED -> CONNECTED: Connection restored.Kazoo uses specialized handlers to manage callbacks via three separate queues. This design prevents deadlocks and ensures consistent execution order by separating different types of events. The three queue types are:
DataWatch, and ChildrenWatch functions.IAsyncResult objects.Because of this separation, you can safely make calls to Zookeeper from within most callbacks (such as Watch events or Completion callbacks) without blocking critical session events. However, you should not make calls to Zookeeper from within a state listener (Session event callback) if you want to avoid potential blocking issues.
Important: If you write code that blocks inside any of these callback functions, no other queued functions of that same type will execute until the blocking code finishes. To avoid stalling an entire queue, run blocking code in a separate greenlet or thread.
The kazoo.recipe.party module provides the Party and ShallowParty classes, which are used to coordinate a group of distributed workers (a "party"). These recipes allow nodes to participate in a collective group, often used for tasks like distributed processing where you need to know how many members are currently active or want to iterate over the members of the group.
Party: A full implementation of the party recipe.ShallowParty: A lighter-weight implementation of the party recipe.Both classes support standard Python container protocols, including __iter__ for iterating over members and __len__ to get the current size of the party.
The TreeCache recipe uses two primary data structures to communicate state changes:
TreeEvent: Represents an event occurring within the cached subtree. It typically encapsulates information about which node changed and the type of change (e.g., creation, deletion, or data update).NodeData: Represents the state of a specific node within the cache, containing the node's path and its associated data.zake library. zake provides a mock client that implements the same interface as a Kazoo client. This allows you to test application layers that interact with Kazoo and perform introspection (e.g., checking what was stored or which watchers are active) without a running Zookeeper instance.Kazoo provides a higher-level API for persistent watches that do not require manual re-registration. These watches are called immediately upon registration and then every time a change occurs. They stop when the decorated function returns False.
DataWatch: Watches for data modifications on a node. The callback receives (data, stat) where stat is a kazoo.protocol.states.ZnodeStat.ChildrenWatch: Watches for changes to a node's children. The callback receives the list of children.Both are available as decorators on the KazooClient instance.
@zk.ChildrenWatch("/my/favorite/node")
def watch_children(children):
print("Children are now: %s" % children)
@zk.DataWatch("/my/favorite")
def watch_node(data, stat):
print("Version: %s, data: %s" % (stat.version, data.decode("utf-8")))_async) return an IAsyncResult object. This object allows you to monitor the status of the operation or attach callbacks. To retrieve the actual result from an IAsyncResult object, you must call its .get() method. Note that .get() may raise an exception if the asynchronous operation encountered an error (e.g., ConnectionLossException or NoAuthException), so it should be wrapped in a try-except block.Kazoo provides two primary handler implementations to manage the event queues, depending on your concurrency model:
kazoo.handlers.gevent.SequentialGeventHandler: Runs a separate greenlet for each of the three queues to process callbacks in order.kazoo.handlers.threading.SequentialThreadingHandler: Runs a separate thread for each of the three queues to process callbacks in order.Both ensure that callbacks within a specific category (Session, Watch, or Completion) are processed sequentially.
To connect to Zookeeper without blocking the main thread, use KazooClient.start_async(). This method returns an IAsyncResult object immediately. You should use the .wait(timeout=...) method on the returned object to wait for the connection to be established. It is highly recommended to always use a timeout to handle cases where a connection cannot be established gracefully.
When using asynchronous frameworks like gevent or eventlet, you must pass the appropriate handler to the KazooClient constructor. Kazoo does not rely on monkey patching.
gevent: Use kazoo.handlers.gevent.SequentialGeventHandler.eventlet: Use kazoo.handlers.eventlet.SequentialEventletHandler.kazoo.handlers.threading.SequentialThreadingHandler.from kazoo.client import KazooClient
from kazoo.handlers.gevent import SequentialGeventHandler
zk = KazooClient(handler=SequentialGeventHandler())
# returns immediately
event = zk.start_async()
# Wait for 30 seconds and see if we're connected
event.wait(timeout=30)
if not zk.connected:
# Not connected, stop trying to connect
zk.stop()
raise Exception("Unable to connect.")