DOCSEARCHbyAlgolia
Submit Docs

pyrefly

repository·main·Indexed 27 days ago

https://github.com/facebook/pyrefly

A high-performance type checker and language server for Python designed for large-scale codebases. It provides autocomplete, code navigation, and fast type checking with built-in support for Pydantic, Django, and pytest. The project includes a VS Code extension, WASM build targets, and performance benchmarks using PyTorch.

Tokens
104.2K
Snippets
316
Records
636
Agent score
92%

What's inside pyrefly

  1. Overview of Pyrefly features and capabilities

    main

    Pyrefly is a PEP 484 compliant type checker and language server for Python. Key advantages include:

    • Rich IDE experience: Advanced refactoring capabilities (comparable to Pylance or PyCharm).
    • Strong Inference: Mature type system with the ability to infer types on un-annotated code.
    • Scale: Proven in large-scale environments like Instagram and PyTorch; default server for Antigravity and Positron editors.
    • Native Framework Support: Built-in support for Pydantic and Django to handle specialized type-checking behaviors.
    • Dual-use: Can be used as a standalone type checker (CLI/CI) or as a standalone language server (IDE features like hover and go-to-definition).

  2. Introduction to Pyrefly

    main
    Pyrefly is a fast type checker and language server for Python designed to catch type-related errors before runtime. It provides powerful IDE features and supports both IDE integration and CLI usage, helping to improve code reliability, documentation, and developer experience through accurate autocomplete and safer refactoring.

  3. Overview of Pyrefly features

    main

    Pyrefly is a high-performance type checker and language server for Python.

    Key Capabilities:

    • High Performance: Capable of checking over 1.85 million lines of code per second.
    • Language Server Features: Provides code navigation, autocomplete, hover information, inlay hints, and semantic highlighting.
    • Framework Support: Built-in support for Pydantic (model validation/field types), Django, and pytest (fixture navigation/autocomplete).
    • Scalability: Used in production for large-scale codebases like Meta's Instagram and open-source projects like PyTorch and JAX.

  4. Overview of Pyrefly Architecture

    main
    Pyrefly is a next-generation Python type checker and language server designed for both high-throughput CLI workloads (like CI) and low-latency IDE workloads. Unlike its predecessor Pyre, which was throughput-optimized, Pyrefly is built with a language-server-first architecture to ensure responsiveness in editors. It uses the Ruff parser for fast, robust, and error-tolerant parsing, which is critical for maintaining functionality while a user is mid-edit.

  5. Supported Django ORM features

    main

    Pyrefly automatically recognizes Django constructs without requiring plugins or manual configuration. Supported features include:

    • Model Classes: Recognition of classes inheriting from models.Model.
    • Field Types: Understanding of CharField, IntegerField, ForeignKey, etc.
    • Auto-Generated Fields: Inference for id, pk, and X_id (for ForeignKey named X).
    • Relationships: Handling of ForeignKey (including null=True nullable types) and ManyToManyField.
    • Choices: Support for Choices, IntegerChoices, and TextChoices enums, including the get_<fieldname>_display() method.
    • Class-based Views: Typing of generic views and mixins when parameterized by a model.

  6. How Pyrefly Processes Modules

    main

    The type checker operates in three sequential steps for every module:

    1. Export Resolution: Determine what each module exports by transitively solving all import * statements.
    2. Binding Conversion: Convert the module into bindings, capturing all statements and both static and flow-based scope information.
    3. Binding Solving: Solve the generated bindings, which may depend on solutions from other modules.

    Pyrefly uses a module-centric design focused on raw performance and parallelism rather than fine-grained, identifier-level incrementality.

  7. Core Concepts of Pyrefly Tensor Typing

    main

    Pyrefly's tensor shape system relies on three primary technical pillars:

    1. Symbolic Arithmetic: The type system supports symbolic integers as type arguments. This allows dimensions to be represented as symbols or arithmetic expressions (e.g., 4 * config.n_embedding) rather than just concrete integers. Equality is determined via a normalization procedure that simplifies expressions and checks for syntactic equality.

    2. User-Facing Types:

      • Tensor Type: A generic type that accepts dimensions as arguments (e.g., Tensor[dim1, dim2, ...]). You can use Any for unknown or unspecified dimensions.
      • Dim Type: A type used to propagate integers (both concrete and symbolic) into the type system, acting similarly to Literal but supporting symbolic expressions.

    3. Fake Ops (Python DSL): To handle the thousands of complex operators in PyTorch, Pyrefly uses "fake ops." These are programmatic definitions (using a tiny Python DSL, primarily list comprehensions over integers) that describe how an operation transforms input shapes into output shapes without executing the actual data operations.

  8. Understand the FwPython language model

    main

    FwPython is a compact, Python-like object-oriented language formalized in Lean 4. It is designed to model specific aspects of Python typing—such as nominal inheritance, runtime method lookup, and first-class bound methods—without the complexity of modules, mutation, or complex argument binding.

    A FwPython program consists of two components:

    1. A class table: Contains class declarations (name, base classes, and methods).
    2. A main expression: The entry point evaluated when the program runs.

    Key language features include:

    • Nominal inheritance: Uses C3 linearization (the same MRO algorithm as Python).
    • Methods: Unary methods (one explicit argument plus a receiver) that support dynamic dispatch.
    • First-class bound methods: Accessing a method via attribute access (e.m) returns a heap object containing the receiver and the method, which can then be called as a value.

  9. Use Project vs. Per-file type checking modes

    main

    Pyrefly operates in two modes depending on how you invoke it:

    • Project mode: Triggered when no specific files are provided in the CLI. Pyrefly attempts to load a configuration file, uses project-includes and project-excludes from that config, and type checks the resulting file set.
    • Per-file (Single-file) mode: Triggered when you provide specific files or patterns (e.g., pyrefly path/to/file.py). Pyrefly finds the relevant config file for each file provided. In this mode, project-includes and project-excludes from the config file are ignored; only the remaining configuration options are applied.

  10. Understand the FwPython runtime value model

    main

    In the FwPython runtime, all values follow a single shape consisting of an object reference (objRef) and an object ID (oid).

    By looking up the oid in the heap, you can recover:

    • A runtime class name.
    • A payload, which is either an ordinary payload (carrying no extra data) or a bound-method payload (carrying the receiver object ID and the method declaration chosen by lookup).

    Fixed object IDs in the initial heap:

    • None: 0
    • True: 1
    • False: 2

  11. Measure type coverage with pyrefly coverage

    main

    Use the pyrefly coverage command to measure how much of your Python code is annotated with types. Coverage is calculated based on typables (function return types, parameters, module-level variables, and class attributes).

    There are two primary subcommands:

    • pyrefly coverage check: Used to validate coverage against a threshold (ideal for CI gates).
    • pyrefly coverage report: Used to generate a detailed JSON report with per-module statistics.

    Coverage Definitions:

    • Coverage: The percentage of typables that are annotated. Any counts as covered.
    • Strict coverage: The percentage of typables that are typed. Any does not count as covered unless the --strict flag is used (which treats Any as untyped).

  12. Use pyrefly-shape-extensions for runtime tensor shape annotations

    main

    The pyrefly-shape-extensions package provides the shape_extensions module, which contains lightweight, no-op runtime helpers for Pyrefly tensor shape annotations.

    These helpers allow you to use advanced shape typing primitives in your Python code—such as Tensor[B, T], IntVar("B"), and assert_shape(x, (2, 3))—without causing runtime errors. While Pyrefly uses the corresponding stubs for static shape checking during development, these extensions ensure the code remains valid and executable by the Python interpreter at runtime.