Bowler Documentation

Bowler is a refactoring tool for manipulating Python at the syntax tree level. It is designed for safe, large-scale code modifications, ensuring that the resulting code compiles and runs.

It provides two primary ways to interact with it:

1. A command line interface.
2. A fluent Python API for generating complex code modifications.

Bowler uses a Query API to build refactoring scripts through a series of selectors, filters, and modifiers. You can use built-in tools or provide custom selectors, filters, and modifiers for complex refactorings.

Bowler is a refactoring tool that manipulates Python code at the syntax tree level using the Concrete Syntax Tree (CST) from lib2to3. It allows for safe, large-scale code modifications by ensuring the resulting code remains syntactically correct.

You can use Bowler via a command-line interface or a fluent Python API. The fluent API allows you to chain selectors, filters, and modifiers to build complex transformations.

query = (
    Query([<file paths>])
    # rename class Foo to Bar
    .select_class("Foo")
    .rename("Bar")
    # change method buzz(x) to buzzard(x: int)
    .select_method("buzz")
    .rename("buzzard")
    .modify_argument("x", type_annotation="int")
)

query.diff()  # generate unified diff on stdout
query.write()  # write changes directly to files

Queries use a fluent API to build a series of transforms over a set of file paths. A query consists of a sequence of transforms, where each transform includes a selector, optional filters, and one or more modifiers.

To build a query, follow this pattern:

1. Initialize: Create a Query object with a list of file paths.
2. Select: Use a selector to define broad search criteria.
3. Filter (Optional): Use filters to refine the scope of the modification.
4. Modify: Use modifiers to apply changes to the matched nodes.
5. Repeat: Add more transforms by repeating steps 2-4.
6. Execute: Trigger a terminal action like .diff() or .write() to apply the changes.

Filters are intermediate functions used to restrict the set of syntax tree elements matched by selectors before any modifications are applied.

Key behaviors:

• Short-circuiting: Matched elements must pass all applied filters to be modified. If an element fails a single filter, it is dropped immediately and subsequent filters are not evaluated for that element.
• Signature: All custom filter functions must follow this signature:

def some_filter(node: LN, capture: Capture, filename: Filename) -> bool:
    ...

Arguments:

• node: The matched syntax tree element.
• capture: Sub-elements captured by the selector pattern.
• filename: The file being considered for modification.
• Return Value: Return True to keep the element for modification, or False to drop it.

Modifiers are functions used to modify, add, remove, or replace syntax tree elements that were matched by selectors after they have passed all filters.

Modifications can occur anywhere in the syntax tree (above or below the matched element) and can include multiple changes.

All modifier functions must follow this signature:

def modifier(node: LN, capture: Capture, filename: Filename) -> Optional[LN]:
    ...

Arguments

• node: The matched syntax tree element.
• capture: Sub-elements captured by the selector pattern.
• filename: The file being modified.
• Return Value: The leaf or nodes returned will automatically replace the matched element.

Bowler is built on top of lib2to3 (specifically using fissix, a backport with improved features). This provides a Concrete Syntax Tree (CST) implementation.

Because it uses a CST rather than an Abstract Syntax Tree (AST), Bowler can modify the syntax tree while preserving all original formatting and comments. This prevents refactoring operations from destroying valuable code metadata or stylistic information.

Compatibility Notes:

• Source Code: Can read and modify source files written for both Python 2 and Python 3 (back to version 2.6).
• Runtime Requirement: Bowler itself requires Python 3.6 or newer to run.

Selectors are lib2to3 search patterns used to identify nodes in the CST. They can capture child nodes or leaves using named assignments.

Example pattern to find print function calls and capture their arguments:

pattern = """
    power< "print"
        trailer< "(" print_args=any* ")" >
    >
"""

(
    Query()
    .select(pattern)
    ...
)

In this example, print_args captures everything inside the parentheses as a list.

Bowler refactors Python code by building queries against the Concrete Syntax Tree (CST). A query is composed of one or more transforms. Each transform follows a specific lifecycle:

1. Selector: A pattern used to find specific nodes in the syntax tree (e.g., finding all print calls).
2. Filters: Functions that inspect the matched nodes and return True to keep them or False to exclude them.
3. Modifiers: Functions that perform the actual tree transformation (modifying, removing, or inserting nodes).

Bowler uses a fluent API, allowing you to chain these operations together in a single statement.

(
    Query()
    .select(pattern)
    .filter(filter_func)
    .modify(modifier_func)
    .execute()
)

(
    Query()
    .select(...)
    .modify(...)
    .execute()
)

Bowler uses a "fluent" Query API, meaning most methods return the Query object itself. This allows you to chain multiple refactoring operations in a single expression without needing to assign the object to a variable at every step.

Both fluent and imperative styles are supported:

Fluent style (preferred):

(
    Query()
    .select_function("foo")
    .rename("bar")
    .diff()
)

Imperative style:

query = Query()
query.select_function("foo")
query.rename("bar")
query.diff()