greenlet Documentation

repository·master·Indexed 23 days ago

https://github.com/python-greenlet/greenlet

A C extension module providing lightweight coroutines for in-process concurrent programming. It offers a primitive micro-thread abstraction with no implicit scheduling, allowing fine-grained control over execution flow via the greenlet class, switch() and throw() methods, and a comprehensive C API for extension modules.

Tokens
9K
Snippets
17
Records
54
Agent score
83%

What's inside greenlet

  1. What is a greenlet and how does it work?

    master

    A greenlet is a small, independent pseudo-thread represented by a greenlet object. Conceptually, it is a stack of frames where the bottom frame is the initial function called and the innermost frame is where the greenlet is currently paused.

    Greenlets allow for explicit execution switching (jumping between stacks). Unlike threads, jumps are never implicit; a greenlet must explicitly choose to switch to another greenlet, causing the former to suspend and the latter to resume. Switching can also pass objects between greenlets, similar to generator.send(val).

  2. How GreenletExit works during garbage collection

    master

    When all references to a greenlet object are removed (including references from the parent attribute of other greenlets), the greenlet can no longer be switched back to. In this scenario, a GreenletExit exception is automatically injected into the greenlet.

    This is the only case where a greenlet receives execution asynchronously without an explicit call to greenlet.switch(). It allows try/finally blocks to clean up resources or enables a pattern where greenlets run infinite loops that automatically terminate when the greenlet is garbage collected.

    Important: A greenlet is expected to either die or be resurrected by storing a new reference to it. Simply catching and ignoring GreenletExit may result in an infinite loop.

    >>> from greenlet import getcurrent, greenlet, GreenletExit
    >>> def run():
    ...     print("Beginning greenlet")
    ...     try:
    ...         while 1:
    ...             print("Switching to parent")
    ...             getcurrent().parent.switch()
    ...     except GreenletExit:
    ...          print("Got GreenletExit; quitting")
    
    >>> glet = greenlet(run)
    >>> _ = glet.switch()
    Beginning greenlet
    Switching to parent
    >>> glet = None
    Got GreenletExit; quitting
  3. Connect synchronous and asynchronous loops using greenlets

    master

    Greenlets allow you to bridge the gap between a synchronous, blocking "pull"-based application (like a command-line loop) and an asynchronous, callback-based "push"-based system (like a GUI event loop).

    Instead of using threads and complex locking/queuing, you can run both loops in a single thread by switching between them. This is achieved by:

    1. Wrapping your synchronous logic in a greenlet.
    2. Using greenlet.switch() from the asynchronous callback to jump into the synchronous greenlet and pass data.
    3. Using main_greenlet.switch() from within the synchronous logic to jump back to the event loop and wait for the next event.

    This approach preserves the call stack of your synchronous logic, meaning you don't have to rewrite your code into a state machine.

    from greenlet import greenlet
    
    # 1. Define the synchronous logic in a greenlet
    g_processor = greenlet(process_commands)
    
    # 2. Get the current (main) greenlet to allow switching back
    main_greenlet = greenlet.getcurrent()
    
    # 3. The asynchronous callback (e.g., GUI event) switches TO the processor
    def event_keydown(key):
        g_processor.switch(key)
    
    # 4. The synchronous blocking function switches BACK to the main loop
    def read_next_char():
        # This suspends g_processor and resumes the main_greenlet
        next_char = main_greenlet.switch('blocking in read_next_char')
        return next_char
  4. Identify the main greenlet

    master

    The "main greenlet" is the implicit greenlet that exists initially in every thread of a process. It is the root of the greenlet tree and is the only greenlet that has a parent of None. The main greenlet can never be dead.

    from greenlet import getcurrent
    
    def am_i_main():
        current = getcurrent()
        return current.parent is None
    
    print(am_i_main())  # Returns True in top-level code
  5. Managing garbage collection and cycles in greenlets

    master

    Greenlets participate in Python's garbage collection in a limited way. Specifically, cycles involving data present in a greenlet's frames may not be detected by the garbage collector.

    Warning: Storing references to other greenlets cyclically may lead to memory leaks.

    Key Behaviors:

    • Active Greenlets: Cycles can be found and cleared while a greenlet is active if the top-level references to the cycle are manually removed (e.g., using del) and gc.collect() is called.
    • Suspended Greenlets: Cycles within the frames of a suspended greenlet (one that has been switched away from and not switched back to) will not be detected by the garbage collector. These cycles are only freed when the greenlet itself becomes garbage.
  6. What are greenlets and how do they work?

    master

    Greenlets are lightweight coroutines designed for in-process sequential concurrent programming. They allow for cooperative multitasking where the programmer, rather than the operating system, controls when execution switches between tasks.

    Key Characteristics:

    • Cooperative Scheduling: Unlike preemptive threads, greenlets only switch when explicitly told to do so. This eliminates many race conditions and simplifies programming.
    • Low Resource Overhead: Because they do not involve the operating system for stack management or bookkeeping, you can run significantly more greenlets than native threads.
    • C-Library Compatibility: Greenlets can switch execution even when C functions are in the call stack, making them ideal for integrating with GUIs, event loops, or C-based I/O libraries.
    • No Special Syntax: Unlike Python's async def or generators, greenlets do not require special keywords or language-level support to function as coroutines.
  7. What are Greenlets?

    master

    Greenlets are lightweight coroutines used for in-process concurrent programming. Unlike Stackless tasklets, which have implicit scheduling and synchronization via channels, a greenlet is a primitive micro-thread with no implicit scheduling. This allows developers to have exact control over when code runs.

    Greenlets can be used to build custom scheduled micro-threads or advanced control flow structures. For example, they can be used to implement a version of generators that allows nested functions to yield values without requiring the yield keyword in every function.

  8. Switching to dead greenlets

    master

    If you attempt to switch to a greenlet that has already finished execution (a "dead" greenlet), the switch will not fail. Instead, the execution jumps to the dead greenlet's parent, or its parent's parent, and so on, until it reaches a living greenlet (eventually the "main" greenlet).

    When switching to a dead greenlet, the switch() call typically returns an empty tuple ().

    def inner():
        print("Entering inner.")
        print("Returning from inner.")
        return 42
    
    def outer():
        print("Entering outer and spawning inner.")
        inner_glet = greenlet(inner)
        print("Switching to inner.")
        result = inner_glet.switch()
        print("Got from inner value: %s" % (result,))
        print("Switching to inner again.")
        result = inner_glet.switch() # This switches to the parent (outer) because inner is dead
        print("Got from inner value: %s" % (result,))
        return inner_glet
    
    outer_glet = greenlet(outer)
    inner_glet = outer_glet.switch()
    
    # Output:
    # Entering outer and spawning inner.
    # Switching to inner.
    # Entering inner.
    # Returning from inner.
    # Got from inner value: 42
    # Switching to inner again.
    # Got from inner value: ()
  9. Understand greenlet parents and exception propagation

    master

    Greenlets are organized in a tree structure. Every greenlet (except the main greenlet) has a "parent". By default, the parent is the greenlet in which the new greenlet was created.

    Key behaviors of parents:

    • Execution Flow: When a greenlet dies (via return or falling off the end of its function), execution continues in its parent.
    • Exception Propagation: If a greenlet raises an uncaught exception, that exception is raised in its parent. The traceback will show the error within the child greenlet, but the exception itself is caught/raised by the parent's execution context.
    >>> def test2():
    ...    print(this_should_be_a_name_error)
    >>> gr1 = greenlet(test1)
    >>> gr2 = greenlet(test2)
    >>> gr1.switch()
    Traceback (most recent call last):
      File "<doctest default[3]>", line 1, in <module>
        gr1.switch()
      File "<doctest default[0]>", line 2 in test2
        print(this_should_be_a_name_error)
    NameError: name 'this_should_be_a_name_error' is not defined
  10. Pass objects and control between greenlets using switch()

    master

    You can pass data between greenlets by using the switch() method. When a greenlet calls g.switch(value), execution jumps to greenlet g, and g receives value as the return value of its previous switch() call.

    Key Switching Scenarios:

    • g.switch(*args, **kwargs): Switches execution to greenlet g. If g has not started yet, these arguments are passed to its run() function.
    • g.throw(*args, **kwargs): Switches execution to g and raises an exception in it.
    • Greenlet Death: When a greenlet's run() method finishes, its return value is sent to its parent. If it terminates with an exception, the exception is propagated to the parent (unless it is a greenlet.GreenletExit exception, which is caught and returned as an object).

    Note that x = g.switch(y) sends y to g, but x will eventually contain whatever object some other greenlet passes back to the caller when it eventually switches back.

    from greenlet import greenlet
    
    def test1(x, y):
        z = gr2.switch(x + y)
        print(z)
    
    def test2(u):
        print(u)
        gr1.switch(42)
    
    gr1 = greenlet(test1)
    gr2 = greenlet(test2)
    gr1.switch("hello", " world")
    # Output:
    # hello world
    # 42
  11. How Greenlets and Python Threads interact

    master

    Greenlets can be used within Python threads, but they are bound to the thread in which they were created. Each Python thread contains its own independent "main" greenlet and a tree of sub-greenlets.

    Crucial Constraint: You cannot switch between greenlets that belong to different threads. Attempting to do so will raise a greenlet.error: cannot switch to a different thread.

    from greenlet import getcurrent
    from greenlet import greenlet
    from threading import Thread
    
    class T(Thread):
        def run(self):
            self.main_glet = getcurrent()
            self.child_glet = greenlet(lambda: None)
            self.child_glet.switch()
    
    t = T()
    t.start()
    # Attempting to switch to a greenlet in thread 't' from the main thread will fail:
    t.main_glet.switch()
    # Raises: greenlet.error: cannot switch to a different thread
  12. Understand the greenlet lifecycle

    master

    A greenlet's lifecycle follows these stages:

    1. Creation: A greenlet is initialized with an empty stack.
    2. Execution: When switched to, it starts running its specified function.
    3. Death: A greenlet becomes "dead" when its outermost function finishes execution, it dies from an uncaught exception, or it is garbage collected (which raises an exception).

    You can check if a greenlet is dead using the .dead attribute.

    >>> from greenlet import greenlet
    >>> def test1():
    ...     return 'test1 done'
    >>> gr1 = greenlet(test1)
    >>> gr1.switch()
    'test1 done'
    >>> gr1.dead
    True