autotyping

repository·master·Indexed 43 days ago

https://github.com/jellezijlstra/autotyping

A tool for automatically adding Python type annotations to codebases using heuristics. It can infer types from default values, magic methods, and return statements, and supports integration with pyanalyze reports. Built as a LibCST codemod, it can be run via CLI, as a pre-commit hook, or through the libcst.tool interface. Version 24.9.0.

Tokens
2.7K
Snippets
7
Records
13
Agent score
53%

What's inside autotyping

  1. Understand autotyping limitations

    master

    Autotyping uses heuristics to find annotations. Because it does not model full code flow through a function, it may miss implicit None returns.

    Important: Always run a type checker (like mypy or pyright) after running autotyping to verify that the added annotations are correct.

  2. Use autotyping shortcut flags

    master

    To avoid listing many individual flags, use these shortcuts:

    • --safe: Enables transformations that are considered safe: --none-return, --scalar-return, and --annotate-magics.
    • --aggressive: Enables all --safe transformations plus riskier ones: --bool-param, --int-param, --float-param, --str-param, --bytes-param, and --annotate-imprecise-magics.
    # Example of using the safe shortcut
    python -m autotyping --safe /path/to/code
  3. Install and use autotyping via CLI

    master

    You can install autotyping via pip and run it directly against your codebase using the python -m autotyping command. By default, the tool does nothing unless you provide specific transformation flags. Use the following command structure:

    pip install autotyping
    python -m autotyping /path/to/my/code
  4. Run autotyping as a LibCST codemod

    master

    Autotyping is built as a LibCST codemod. To run it via the libcst.tool interface:

    1. Ensure you have a .libcst.codemod.yaml file in your project with 'autotyping' in the modules list.
    2. Execute the command:
    python -m libcst.tool codemod autotyping.AutotypeCommand /path/to/my/code
  5. Configure autotyping as a pre-commit hook

    master

    To automatically add type annotations during the commit process:

    1. Install pre-commit:

      pip install pre-commit
    2. Add the following configuration to your .pre-commit-config.yaml:

    - repos:
      - repo: https://github.com/JelleZijlstra/autotyping
        rev: 24.9.0
        hooks:
          - id: autotyping
            stages: [pre-commit]
            types: [python]
            args: [--safe] # Use --safe, --aggressive, or specific flags
    1. Install the hook in your repository:
      pre-commit install
    - repos:
      - repo: https://github.com/JelleZijlstra/autotyping
        rev: 24.9.0
        hooks:
          - id: autotyping
            stages: [pre-commit]
            types: [python]
            args: [--safe]
  6. Define parameter annotations using the `name:type` format

    master

    When using --annotate-optional or --annotate-named-param, use the format name:type_path. The type_path can be a simple name or a fully qualified path.

    • foo:int -> Annotates foo as int.
    • foo:bar.Baz -> Annotates foo as bar.Baz (imports bar.Baz).
    • foo:typing.Optional[int] -> Annotates foo with the specific type path.
  7. Integrate pyanalyze suggestions into AutotypeCommand

    master

    You can provide a JSON report generated by pyanalyze to AutotypeCommand to apply high-fidelity type suggestions. The command parses the report and applies suggested_type and necessary imports to the corresponding code positions.

    To use this, pass the path to the report via the --pyanalyze-report flag. If you want to avoid adding new imports to your files, use the --only-without-imports flag.

  8. Autotyping CLI transformation flags

    master

    Autotyping uses flags to determine which type annotations to add.

    Return Type Annotations

    • --none-return: Adds -> None to functions without return, yield, or raise in their body.
    • --scalar-return: Adds return annotations for functions returning literal bool, str, bytes, int, or float objects.

    Parameter Type Annotations

    • --bool-param: Adds : bool to parameters with a default of True or False.
    • --int-param, --float-param, --str-param, --bytes-param: Adds annotations for parameters with literal int, float, str, or bytes defaults.
    • --annotate-optional <param_name>:<module>.<Type>: For parameters like foo=None, adds the specified type (e.g., --annotate-optional uid:my_types.Uid adds Optional[my_types.Uid]).
    • --annotate-named-param <param_name>:<module>.<Type>: For parameters with no default, adds the specified type (e.g., --annotate-named-param uid:my_types.Uid adds my_types.Uid).
    • --guess-common-names: Infers types from common names (e.g., verbose becomes bool).

    Magic Method Annotations

    • --annotate-magics: Adds standard annotations to magic methods like __str__ (str), __len__ (int), __init__ (None), etc.
    • --annotate-imprecise-magics: Adds typing.Iterator return annotations to __iter__, __await__, and __reversed__.

    External Integrations

    • --pyanalyze-report: Applies type suggestions from pyanalyze's suggested_parameter_type and suggested_return_type codes.
    • --only-without-imports: Only applies pyanalyze suggestions that do not require adding new imports.
  9. Use AutotypeCommand arguments

    master

    When running via python -m autotyping, the command accepts additional arguments defined by AutotypeCommand.add_args(parser). These arguments are passed directly to the AutotypeCommand constructor as keyword arguments.

    Note: The positional path argument is handled separately and is used to identify the files to be processed (including stubs).

  10. Guess types from argument names with `guess_type_from_argname`

    master

    The guess_type_from_argname function attempts to infer a Python type and a collection container based on a string argument name. It uses pattern matching and a set of predefined name lists (for booleans, integers, floats, and strings) to make a 'best guess'.

    Return Value Returns a Tuple[Optional[str], List[str]]:

    1. Optional[str]: The inferred element type (e.g., 'int', 'str', 'float', 'bool'). Returns None if no type can be determined.
    2. List[str]: A list of container types (e.g., ['List'], ['Set'], ['Iterable']). If the type is a scalar (not a container), this list is empty [].

    Matching Logic Examples:

    • Container + Type: list_ints $\rightarrow$ ('int', ['List'])
    • Type + Container: latitude_list $\rightarrow$ ('float', ['List'])
    • Container of Type: set_of_widths $\rightarrow$ ('int', ['Set'])
    • Boolean Prefixes: is_valid $\rightarrow$ ('bool', [])
    • Plurals: users $\rightarrow$ ('Any', ['Sequence']) (via recursive singular check)
    • Filesystem: path_to_file $\rightarrow$ (None, []) (explicitly avoids guessing str for paths to encourage pathlib.Path)
    from autotyping.guess_type import guess_type_from_argname
    
    # Example: Container with type suffix
    element_type, containers = guess_type_from_argname("list_ints")
    print(element_type, containers)  # Output: int ['List']
    
    # Example: Type with container suffix
    element_type, containers = guess_type_from_argname("latitude_list")
    print(element_type, containers)  # Output: float ['List']
    
    # Example: Boolean name
    element_type, containers = guess_type_from_argname("is_active")
    print(element_type, containers)  # Output: bool []
    
    # Example: Scalar integer name
    element_type, containers = guess_type_from_argname("width")
    print(element_type, containers)  # Output: int []
  11. Use AutotypeCommand to automatically add type annotations

    master

    The AutotypeCommand is a LibCST VisitorBasedCodemodCommand designed to automatically add simple type annotations to Python code. It can infer types from default values, magic methods, return statements, and pyanalyze reports. It also supports manual configuration for specific parameter names and types.

    from libcst.codemod import CodemodContext
    from autotyping.autotyping import AutotypeCommand
    
    # Assuming you have a CodemodContext and a target file
    command = AutotypeCommand(
        context,
        annotate_optional=["foo:bar.Baz"],
        none_return=True,
        safe=True
    )
    # The command is then used within a LibCST codemod execution flow
  12. Run autotyping via CLI

    master

    You can run autotyping as a module using python -m autotyping. The tool accepts one or more paths as positional arguments and uses AutotypeCommand to perform type transformations.

    It leverages LibCST's parallel_exec_transform_with_prettyprint to process files in parallel. The tool will output a summary of successes, skips, failures, and warnings to stderr upon completion.

    python -m autotyping <path1> <path2> ...