pydeps

repository·master·Indexed 24 days ago

https://github.com/thebjorn/pydeps

A tool for visualizing Python module dependencies by analyzing Python bytecode. It helps developers understand project structure, identify import cycles, and map external dependencies. It generates graphs in SVG or PNG format using Graphviz and provides advanced filtering options such as Bacon scoring, module depth limiting, and clustering of external modules.

Tokens
5.2K
Snippets
9
Records
16
Agent score
85%

What's inside pydeps

  1. What is Bacon (Scoring) and how to use it

    master

    The Bacon scoring function (based on the Erdős-like Bacon number) allows you to filter the dependency graph to show only modules within a certain number of 'hops' from your target module. This is useful for isolating the interface a module has to the rest of the world.

    • --max-bacon n: Excludes nodes that are more than n hops away.
    • The default is 2.
    • Setting --max-bacon 0 results in an infinite filter (no nodes excluded by distance).
  2. How to display the generated graph

    master

    By default, pydeps calls an appropriate opener for your platform (e.g., xdg-open) to display the resulting .svg or .png file.

    You can customize this behavior in two ways:

    1. Use the --display PROGRAM flag to specify an executable.
    2. Set the PYDEPS_DISPLAY or BROWSER environment variables.
  3. How to identify import cycles

    master

    By default, pydeps highlights any cycles in the dependency graph with blue boxes.

    To isolate and view only the import cycles (the nodes participating in a cycle and the edges between them), use the --show-cycles flag. This is particularly helpful for debugging cyclic imports in large projects.

  4. Install pydeps

    master

    Install pydeps using pip.

    Note: To generate graphs, you must also have Graphviz installed on your system, and the dot command must be available in your system's PATH.

    pip install pydeps
  5. Use the pydeps CLI

    master

    The pydeps command is used to visualize Python module dependencies. It analyzes Python bytecode to find imports.

    Important Usage Note: If you provide options with a variable number of arguments (like -x) before the filename (fname), you must separate the arguments from the filename using --. Otherwise, the filename will be treated as an argument to the option.

    Example: $ pydeps -x os sys -- pydeps

  6. Use the pydeps CLI to generate dependency graphs

    master

    The pydeps command-line tool analyzes Python module dependencies and generates visual graphs.

    Basic Usage

    To analyze a specific file or package:

    pydeps <filename_or_package>

    If you want to automatically detect the current package name, use the --find-package flag:

    pydeps --find-package

    Common Output Options

    • Specify output file: Use -o <file> to write the graph to a file.
    • Choose format: Use -T <format> to select svg (default) or png.
    • Display graph: By default, pydeps attempts to open the graph in an external program. Use --noshow or --no-show to prevent this, or --display <PROGRAM> to specify a specific viewer.
    • Skip file creation: Use --no-output to skip creating .svg or .png files (this implies --noshow).

    Filtering and Inclusion

    • Exclude modules: Use -x <PATTERN> or -xx <MODULE> (for exact matches) to skip specific modules.
    • Include only specific modules: Use --only <MODULE_PATH> to only include modules starting with the specified path.
    • Include Python Standard Library: Use --pylib to include stdlib modules, or --pylib-all to include all (including C modules).
    • Include missing modules: Use --include-missing to include modules that are not installed or cannot be found on sys.path.

    Graph Customization

    • Direction: Use --rankdir <DIRECTION> to set graph direction. Options: TB (default, top-to-bottom), BT (bottom-to-top), LR (left-to-right), RL (right-to-left).
    • Reverse arrows: Use --reverse to draw arrows to (instead of from) imported modules.
    • Clustering:
      • --cluster: Draw external dependencies as separate clusters.
      • --keep-target-cluster: Draw the target module as a cluster.
      • --collapse-target-cluster: Collapse the target module into a single node.
      • --min-cluster-size <INT>: Minimum nodes required to form a cluster.
      • --max-cluster-size <INT>: Maximum nodes before a cluster is collapsed.
  7. Configure pydeps via configuration files

    master

    pydeps loads configuration from multiple files with a specific precedence order. If multiple files are present, the one with the highest precedence wins.

    Precedence Order (Highest to Lowest):

    1. Explicitly specified file via --config <FILE>
    2. .pydeps file in the current working directory
    3. pydeps.yml file in the current working directory
    4. pyproject.toml file in the current working directory
    5. setup.cfg file in the current working directory
    6. .pydeps file in the user's home directory

    You can disable all configuration file processing using the --no-config flag.

  8. Configure pydeps using configuration files

    master

    You can set pydeps options in several configuration files. The order of precedence (highest to lowest) is:

    1. Command line options
    2. .pydeps file in the current directory (using .ini syntax)
    3. .pydeps file in the user's home directory (%USERPROFILE% on Windows, ${HOME} otherwise)

    pydeps also supports configuration in:

    • pyproject.toml under the [tool.pydeps] section.
    • setup.cfg under the [pydeps] section.
    [pydeps]
    max_bacon = 2
    no_show = True
    verbose = 0
    pylib = False
    exclude =
        os
        re
        sys
        collections
        __future__
  9. Get external dependencies programmatically or via CLI

    master

    To quickly identify the direct external dependencies of a package, use the --externals flag. This prints a JSON-formatted list of module names to the screen.

    Alternatively, you can use the externals function from pydeps.pydeps in your Python code.

    pydeps --externals dktasklib
    import os
    from pydeps.pydeps import externals
    # the directory that contains setup.py (one level up from actual package):
    os.chdir('package-directory')
    print(externals('mypackage'))
  10. Reference: pydeps CLI options

    master

    The following options are available for the pydeps command line interface:

    usage: pydeps [-h] [--debug] [--config FILE] [--no-config] [--version] [-L LOG] [--find-package] [-v] [-o file] [-T FORMAT] [--display PROGRAM] [--noshow] [--show-deps] [--show-raw-deps] [--deps-output DEPS_OUT] [--show-dot] [--dot-output DOT_OUT] [--nodot] [--no-output] [--debug-mf INT] [--noise-level INT] [--max-bacon INT] [--max-module-depth INT] [--pylib] [--pylib-all] [--include-missing] [-x PATTERN [PATTERN ...]] [-xx MODULE [MODULE ...]] [--only MODULE_PATH [MODULE_PATH ...]] [--externals] [--reverse] [--rankdir {TB,BT,LR,RL}] [--cluster] [--min-cluster-size INT] [--max-cluster-size INT] [--keep-target-cluster] [--collapse-target-cluster] [--rmprefix PREFIX [PREFIX ...]] [--start-color INT] fname
    
    positional arguments:
      fname                 filename
    
    optional arguments:
      -h, --help                             show this help message and exit
      --debug                                turn on all the show and verbose options (mainly for debugging pydeps itself)
      --config FILE                          specify config file
      --no-config                            disable processing of config files
      --version                              print pydeps version
      -L LOG, --log LOG                      set log-level to one of CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET.
      --find-package                         tries to automatically find the name of the current package.
      -v, --verbose                          be more verbose (-vv, -vvv for more verbosity)
      -o file                                write output to 'file'
      -T FORMAT                              output format (svg|png)
      --display PROGRAM                      program to use to display the graph (png or svg file depending on the T parameter)
      --noshow, --no-show                    don't call external program to display graph
      --show-deps                            show output of dependency analysis
      --show-raw-deps                        show output of dependency analysis before removing skips
      --deps-output                          write output of dependency analysis to file (instead of screen)
      --show-dot                             show output of dot conversion
      --dot-output DOT_OUT                   write dot code to write to file (instead of screen)
      --nodot, --no-dot                      skip dot conversion
      --no-output                            don't create .svg/.png file, implies --no-show (-t/-o will be ignored)
      --debug-mf INT                         set the ModuleFinder.debug flag to this value
      --noise-level INT                      exclude sources or sinks with degree greater than noise-level
      --max-bacon INT                        exclude nodes that are more than n hops away (default=2, 0 -> infinite)
      --max-module-depth INT                 coalesce deep modules to at most n levels
      --pylib                                include python std lib modules
      --pylib-all                            include python all std lib modules (incl. C modules)
      --include-missing                      include modules that are not installed (or can't be found on sys.path)
      --only MODULE_PATH                     only include modules that start with MODULE_PATH, multiple paths can be provided
      --externals                            create list of direct external dependencies
      --reverse                              draw arrows to (instead of from) imported modules
      --rankdir                              set the direction of the graph, legal values are TB (default, imported modules above importing modules), BT (opposite direction of TB), LR (left-to-right), and RL (right-to-left)
      --cluster                              draw external dependencies as separate clusters
      --min-cluster-size INT                 the minimum number of nodes a dependency must have before being clustered (default=0)
      --max-cluster-size INT                 the maximum number of nodes a dependency can have before the cluster is collapsed to a single node (default=0)
      --keep-target-cluster                  draw target module as a cluster
      --collapse-target-cluster              collapse target module (this implies --cluster)
      --rmprefix PREFIX                     remove PREFIX from the displayed name of the nodes (multiple prefixes can be provided)
      -x PATTERN, --exclude PATTERN          input files to skip (e.g. `foo.*`), multiple patterns can be provided
      --exclude-exact MODULE                 (shorthand -xx MODULE) same as --exclude, except requires the full match. `-xx foo.bar` will exclude foo.bar, but not foo.bar.blob
  11. View the intermediate dependency format

    master

    When using pydeps --show-deps, the tool outputs an intermediate JSON-like format representing the dependency graph. Each entry in the output contains:

    • imported_by: A list of modules that import this module.
    • imports: A list of modules that this module imports.
    • kind: The type of module (e.g., imp.PY_SOURCE).
    • name: The full module name.
    • path: The file system path to the module.
    {
        "pydeps.py2depgraph": {
            "imported_by": [
                "__main__",
                "pydeps.pydeps"
            ],
            "imports": [
                "pydeps.depgraph",
                "pydeps.mf27"
            ],
            "kind": "imp.PY_SOURCE",
            "name": "pydeps.py2depgraph",
            "path": "pydeps\\py2depgraph.py"
        }
    }
  12. Reference: pydeps CLI flags

    master

    A complete list of available command-line arguments for pydeps.

    Global & Debugging

    • --debug: Turn on all show and verbose options (mainly for debugging pydeps itself).
    • --version: Print pydeps version.
    • --config FILE: Specify a custom config file.
    • --no-config: Disable processing of config files.
    • -L, --log <LEVEL>: Set log-level to one of CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET.
    • -v, --verbose: Increase verbosity (e.g., -vv, -vvv).

    Analysis & Output

    • fname: The input filename or package.
    • -o, --output <file>: Write output to the specified file.
    • -T, --format <format>: Output format (svg or png).
    • --display <PROGRAM>: Program to use to display the graph.
    • --noshow, --no-show: Don't call external program to display graph.
    • --show-deps: Show output of dependency analysis.
    • --show-raw-deps: Show output of dependency analysis before removing skips.
    • --deps-output <file>: Write output of dependency analysis to file.
    • --show-dot: Show output of dot conversion.
    • --dot-output <file>: Write dot code to file.
    • --nodot, --no-dot: Skip dot conversion.
    • --no-output: Don't create .svg/.png file (implies --no-show).
    • --show-cycles: Show only import cycles.
    • --externals: Create a list of direct external dependencies.
    • --reverse: Draw arrows to (instead of from) imported modules.

    Filtering & Scoring

    • --noise-level <INT>: Exclude sources or sinks with degree greater than noise-level.
    • --max-bacon <INT>: Exclude nodes that are more than n hops away (default=2, 0 -> infinite).
    • --max-module-depth <INT>: Coalesce deep modules to at most n levels.
    • -x, --exclude <PATTERN>: Input files to skip (supports multiple patterns).
    • -xx, --exclude-exact <MODULE>: Same as --exclude, but requires an exact match.
    • --only <MODULE_PATH>: Only include modules that start with MODULE_PATH.
    • --pylib: Include python std lib modules.
    • --pylib-all: Include python all std lib modules (incl. C modules).
    • --include-missing: Include modules that are not installed.

    Graph Layout

    • --rankdir <TB|BT|LR|RL>: Set the direction of the graph.
    • --cluster: Draw external dependencies as separate clusters.
    • --min-cluster-size <INT>: Minimum nodes for a cluster.
    • --max-cluster-size <INT>: Maximum nodes before a cluster is collapsed.
    • --keep-target-cluster: Draw target module as a cluster.
    • --collapse-target-cluster: Collapse target module.
    • --rmprefix <PREFIX>: Remove PREFIX from the displayed name of the nodes.
    • --start-color <INT>: Starting value for hue from 0 (red/default) to 360.