DOCSEARCHbyAlgolia
Submit Docs

Pyre

repository·main·Indexed 27 days ago

https://github.com/facebook/pyre-check

A performant, incremental Python type checker compliant with PEP 484, designed for large codebases. It includes Pysa, a static analyzer for taint analysis that identifies security vulnerabilities by mapping TaintSources to TaintSinks using .pysa files and taint.config rules. The toolset supports sanitizers to remove false positives, feature tagging for filtering via the Static Analysis Post Processor (SAPP), and dynamic model generators for automating the creation of taint models.

Tokens
70.8K
Snippets
179
Records
459
Agent score
90%

What's inside pyre-check

  1. Overview of Pysa Tutorial exercises

    main

    The Pysa tutorial consists of 5 exercises designed to teach the Python Static Analyzer (Pysa) features using a Django project as a target. The exercises cover:

    • Running and observing results
    • Adding sinks and rules
    • Using sanitizers to filter results
    • Adding features to and browsing results with SAPP
    • The model generation concept

  3. Understand Pysa Taint Analysis

    main

    Pysa (Python Static Analyzer) performs Taint Analysis to identify security vulnerabilities by tracking the flow of tainted data from sources (where data originates) to sinks (dangerous termination points).

    Key behaviors:

    • Propagation: Taint propagates through operations. If x is tainted, y = x + 10 and s = str(x) will also be tainted.
    • Object Tainting: When an object is tainted, all its attributes (including .__class__) are considered tainted. This may lead to false positives.
    • Scope: Pysa only analyzes code in your repository and directories listed in the search_path of your .pyre_configuration file. It does not see the source of your dependencies. If a function's source is unavailable, Pysa assumes its return type has the same taint as its input to prevent false negatives.

  4. Understand Pysa warning codes

    main
    Pysa identifies data flows from specific sources to sinks. Each unique flow tracked from a source to a sink is assigned a unique warning code. When Pysa detects such a flow, it emits an issue associated with that specific code. To understand the meaning of a warning code emitted during a run, you must refer to the taint_config.json files used in that specific execution, as they serve as the source of truth for these codes.

  5. Understand Pysa Summaries

    main

    Pysa tracks the flow of tainted data by computing summaries for all functions. These summaries cover the entire call graph, meaning if function foo calls bar, foo's summary includes information about sources and sinks reachable within bar.

    Summaries describe three key behaviors:

    1. Which function arguments hit sinks.
    2. Which sources the function returns.
    3. Which arguments propagate their taint to the return value (Taint In Taint Out).

  7. Use Zstandard (zstd) for compression and decompression

    main

    Zstandard (zstd) is a fast lossless compression algorithm. It provides a command line utility for producing and decoding .zst, .gz, .xz, and .lz4 files.

    To trade compression ratio for speed, you can use negative compression levels with the --fast=# flag. Higher numbers provide faster compression and decompression at the cost of a lower compression ratio.

  8. Understand Pyre's Typeshed integration

    main

    Pyre uses a vendored copy of the Python typeshed stubs rather than relying on a system-installed version. This allows Pyre to:

    • Pin a specific typeshed version to ensure compatibility with Python's standard library stubs.
    • Apply patches to typeshed code to ensure it works correctly with Pyre.
    • Extend standard stubs, such as modifying builtins.pyi to support experimental tensor shape type arithmetic.

    The source code for the original project is available at http://github.com/python/typeshed/.

  9. Learn about Pysa through public talks and blogs

    main

    To understand the motivation, architecture, and usage of Pysa, you can review several public resources:

    • PyCon 2018: Covers the open-sourcing of Pyre and the deeper static analysis (Pysa) it enables.
    • PyCon 2019 & Security @Scale: Explains the basics of how Pysa works.
    • F8 (at 16:00): Demonstrates how Pyre is used at Instagram.
    • DEF CON 28: Provides a tutorial on how to get started with Pysa.
    • Engineering Blog: A detailed post covering Pysa's development, usage, mechanics, and results.

  10. Run Pyre tests

    main

    Pyre uses two different test suites depending on the component you are testing:

    • OCaml Unit Tests: Covers the main binary (pyre.bin). These are executed via make test after the sources have been configured.
    • Python Tests: Covers the Python wrappers. These are executed via make python_tests.
    make test
make python_tests

  11. Install the Pyre Visual Studio Code Client

    main

    The Pyre VSCode extension allows you to view Pyre errors within any workspace that contains a .pyre_configuration file.

    To use the extension, you must first ensure pyre and watchman are installed and configured on your system:

    1. Install pyre: Run pip install pyre-check.
    2. Install watchman: Follow the official installation guide (e.g., apt-get install -y watchman on Debian/Ubuntu).
    3. Initialize configuration: Run pyre init in your project directory to create the necessary configuration for both pyre and watchman.
    pip install pyre-check
# Install watchman via your package manager
pyre init