birdseye

repository·master·Indexed 23 days ago

https://github.com/alexmojaki/birdseye

A Python debugger that records expression values during function execution, enabling developers to visually inspect data flow and loop iterations in a browser. It uses AST modification to instrument functions via the @eye decorator and provides a web server for visualization. It includes integrations for Jupyter/IPython notebooks via %%eye magic, a Visual Studio Code extension, and support for snoop via the @spy decorator.

Tokens
4.5K
Snippets
9
Records
33
Agent score
82%

What's inside birdseye

  1. Feature highlights of birdseye

    master

    birdseye provides a non-linear debugging experience focused on data flow rather than line-by-line stepping:

    • Loop Iteration Navigation: Move back and forth through loop iterations to observe how expression values change over time.
    • Exception Highlighting: Identify which specific expressions raise exceptions, including those that are suppressed by try/except blocks.
    • Data Structure Exploration: Expand concrete data structures and objects to inspect their contents (depth and length are automatically limited to prevent data overload).
    • Chronological Call Organization: Function calls are organized by file and function, then ordered by time, providing a clear timeline of execution.
  2. How BirdsEye executes a function call

    master

    During function execution, the following steps manage data collection:

    1. Entry: Upon the first statement, TreeTracerBase._enter_call is called. A FrameInfo object is created for the current frame.
    2. Arguments: BirdsEye.enter_call records the function arguments into FrameInfo. If the parent frame is also being traced, it is marked as an inner call.
    3. Statement Context: A _StmtContext is created for every statement, triggering BirdsEye.before_stmt and BirdsEye.after_stmt.
    4. Expression Evaluation: For every expression, BirdsEye.before_expr and BirdsEye.after_expr are called. Values are expanded in NodeValue.expression and stored in an Iteration (either directly in FrameInfo for top-level or via an IterationList if inside a loop).
    5. Exit: When the call ends, BirdsEye.exit_call is triggered, and the FrameInfo data is gathered and stored in the database as a new Call row.
  3. How BirdsEye traces a function

    master

    When a function is decorated with BirdsEye.trace_function, the following lifecycle occurs:

    1. Parsing: The file is parsed via the ast module. The tree is modified: expressions are wrapped in two function calls (_NodeVisitor.visit_expr) and statements are wrapped in with blocks (_NodeVisitor.visit_stmt).
    2. Positioning: An ASTTokens object is created to map AST node positions to source code.
    3. Compilation: The modified tree is compiled into a code object. The code object corresponding to the traced function is identified.
    4. Globals Update: The function's __globals__ are updated to include references to the newly inserted tracing functions.
    5. Object Creation: A new function object is created as a copy of the original, but using the new code object.
    6. HTML Construction: An HTML document is built where source expressions and statements are wrapped in <span> tags.
    7. Storage: A Function row is stored in the database with the HTML and metadata, and a CodeInfo object is kept in memory.
  4. How Birdseye limits data collection

    master

    To manage memory and performance, Birdseye uses sampling and truncation strategies for data collection:

    • Loop Sampling: Only the first and last 3 iterations of loops are stored. (Note: If an expression is evaluated in the middle of a loop, up to two additional iterations where it was evaluated may be included).
    • Value Representation: A limited version of repr() is used via the cheap_repr package to reduce data size.
    • Nesting Depth: Nested data structures and objects are expanded by a maximum of 3 levels. This depth is further decreased inside loops, unless all current loops are in their first iteration.
    • Object Recording: Only specific pieces of objects are recorded rather than the entire object state.
  5. How Birdseye instrumentation works

    master

    Birdseye works by parsing a decorated function's source code into a standard Python Abstract Syntax Tree (AST). It then modifies the AST by wrapping every statement in a with statement and every expression in a function call. This modified tree is compiled into a new code object used to construct a brand new function.

    Because Birdseye reconstructs the function rather than wrapping it, the @eye decorator must be applied first. If other decorators are applied before @eye, they may be bypassed by the AST reconstruction or have no effect on the tracing.

  6. Debug specific iterations in the middle of a loop

    master

    By default, birdseye saves data from the first and last three iterations of a loop. If you need to inspect a specific iteration in the middle of a long loop, you can use these strategies:

    1. Conditional Logic: Use an if or try/except statement to isolate the specific iteration. birdseye ensures that for every statement/expression node in a loop, at least two iterations where that node was evaluated are saved. If a statement is only evaluated during your specific target iteration, those iterations will be captured.
    2. Function Wrapping: Wrap the loop's contents in a function and decorate that function with @eye. This allows you to find the specific call you need by inspecting the arguments and return values in the calls table, though this incurs a performance cost.
  7. Understand Birdseye performance and memory limitations

    master

    Birdseye traces every function call and nontrivial expression, which introduces significant overhead. When using Birdseye, be aware of the following:

    • Execution Speed: Programs will run significantly slower. Avoid tracing functions that are called frequently or contain many loop iterations.
    • Memory Usage: Large amounts of data are collected for every call. Functions with many loop iterations or large nested objects can consume significant memory during execution and when viewing results in a browser.
    • Visibility: Function calls are only visible in the Birdseye interface once they have completed execution.
  8. Use the Visual Studio Code extension

    master

    The birdseye VS Code extension allows you to view the birdseye UI directly inside your editor.

    1. Install the extension from the Visual Studio Marketplace.
    2. Open the Command Palette (F1 or Cmd+Shift+P).
    3. Select Show birdseye to start the server and open the browser pane within VS Code.
  9. Run the Birdseye server

    master

    The Birdseye server provides the web interface for viewing traces. You can run it via the birdseye CLI command.

    To run a remote server accessible from any network interface, use the --host 0.0.0.0 flag.

    For production environments, it is recommended to use a WSGI server like gunicorn instead of the default Flask development server. The WSGI application is located at birdseye.server:app.

  10. Trace an entire module or file

    master

    Instead of decorating individual functions with @eye, you can trace an entire module or its top-level execution by adding a specific import statement at the top of the file.

    • To trace every function in the file and the module execution itself, use import birdseye.trace_module_deep.
    • To trace only the module execution (and reduce performance impact), use import birdseye.trace_module.

    Important Caveats:

    • These import statements must be unindented (at the top level), not inside if or try blocks.
    • If the module is being imported by another module (rather than run directly):
      • In Python 2, the module will not be traced.
      • birdseye must be imported somewhere before the traced module is imported.
      • The execution of the entire module will be traced, not just the code following the import statement.
    import birdseye.trace_module_deep
  11. Deploy birdseye on PythonAnywhere

    master

    To access the birdseye UI on PythonAnywhere, the birdseye server must run within a web app. You can use a dedicated web app or integrate it into an existing one using WSGI middleware.

    Option 1: Dedicated Web App In your WSGI configuration file (/var/www/your_domain_com_wsgi.py), use:

    from birdseye.server import app as application

    Option 2: Combine with an existing Web App Append the following to your existing WSGI file to mount birdseye at a specific prefix (e.g., /birdseye):

    import birdseye.server
    from werkzeug.wsgi import DispatcherMiddleware
    
    application = DispatcherMiddleware(application, {
        '/birdseye': birdseye.server.app
    })

    Security Note: Because birdseye exposes your code and data, ensure your PythonAnywhere web app has Force HTTPS and Password protection enabled under the Security settings.