Awesome Python

If you need to distribute your Python programs as packaged executables, consider these tools:

• cx-Freeze: Converts scripts into standalone executables and installers for Windows, macOS, and Linux.
• Nuitka: Compiles Python programs into high-performance standalone executables (cross-platform).
• pyarmor: Obfuscates Python scripts and allows binding them to specific machines or expiration dates.
• pyinstaller: Converts Python programs into stand-alone executables (cross-platform).
• shiv: A CLI utility for building fully self-contained zipapps (PEP 441) including all dependencies.

Build Tools

Tools for compiling software or managing tasks:

• bitbake: Build tool for embedded Linux.
• doit: Task runner and build tool.
• invoke: Manage shell-oriented subprocesses and CLI-invokable tasks.
• platformio: Build code for different development platforms.
• pybuilder: Continuous build tool.
• scons: Software construction tool.

Documentation Generation

Libraries for creating project documentation:

• diagrams: Diagram as Code.
• mkdocs: Markdown-friendly documentation generator.
• pdoc: Auto-generates API documentation.
• sphinx: Standard Python documentation generator.

Tools for managing Python versions, environments, and dependencies:

• uv: Extremely fast Python version, package, and project manager (Rust-written).
• pyenv: Simple Python version management.
• virtualenv: Tool to create isolated Python environments.
• KillPy: Analyze, detect, and clean unused Python environments and pipx packages.

The repository is organized into the following core components:

• README.md: The single source of truth containing hierarchical categories and alphabetically ordered entries.
• CONTRIBUTING.md: Contains submission guidelines and review criteria.
• SPONSORSHIP.md: Details sponsor tiers, placement rules, and editorial-independence policies.
• website/: The static site generator directory used to build awesome-python.com from the README.md.
  • build.py: Parses README.md and renders HTML using Jinja2.
  • readme_parser.py: Converts Markdown into structured data.
  • fetch_github_stars.py: Updates star counts in website/data/.
• Makefile: Provides automation commands for the development workflow.
• pyproject.toml: Defines dependencies using uv (requires Python >=3.13).

Use the following tools to manage Python versions, dependencies, and environments:

• conda: Cross-platform, Python-agnostic binary package manager.
• hatch: Modern, extensible project manager for environments, builds, and publishing.
• pip: The standard package installer for Python.
• pipx: Install and run Python applications in isolated environments (similar to npx in Node.js).
• poetry: Dependency management and packaging.
• uv: An extremely fast Python version, package, and project manager written in Rust.

Libraries for Computer Vision tasks:

• General CV: opencv (Open Source Computer Vision Library).
• Deep Learning/PyTorch: kornia (differentiable CV library for PyTorch).
• Object Detection & Segmentation: ultralytics (YOLO for detection, segmentation, pose estimation, and classification).
• OCR (Optical Character Recognition): easyocr (40+ languages) and pytesseract (wrapper for Google Tesseract OCR).
• Dataset Management: fiftyone (tool for building high-quality datasets and models).

Use these libraries to validate data structures and types in Python:

• pydantic: Data validation using Python type hints.
• pandera: Data validation library for dataframes (supports pandas, polars, and Spark).
• jsonschema: Implementation of JSON Schema for Python.
• cerberus: Lightweight and extensible data validation library.
• voluptuous: Primarily intended for validating data from untrusted sources.

Distributed Computing

• Batch Processing: dask (parallel computing), luigi (pipeline management), mpi4py (MPI bindings), pyspark (Apache Spark API), joblib (pipelining), and ray (parallel/distributed Python).

Task Queues and Messaging

• Task Queues: celery (distributed message passing), dramatiq (background tasks), huey (multi-threaded queue), rq (simple job queues), and taskiq (asyncio support).
• Messaging: faststream (asynchronous services for Kafka, RabbitMQ, NATS, MQTT, and Redis).

Job Schedulers

• airflow: Programmatic workflow monitoring.
• apscheduler: In-process task scheduling.
• dagster: Data asset orchestration.
• prefect: Workflow orchestration for data pipelines.
• schedule: Human-friendly job scheduling.
• SpiffWorkflow: Pure Python workflow engine.

Use these libraries to store and parse configuration options:

• configparser: (Python standard library) INI file parser.
• dynaconf: Configuration manager with plugins for Django, Flask, and FastAPI.
• hydra: A framework for elegantly configuring complex applications.
• python-decouple: Ensures strict separation of settings from code.
• python-dotenv: Reads key-value pairs from a .env file and sets them as environment variables.

Django-based APIs

• django-rest-framework: Powerful and flexible toolkit.
• django-ninja: Fast, based on type hints and Pydantic.
• django-modern-rest: Modern REST with support for msgspec and pydantic.
• strawberry-django: Strawberry GraphQL integration.

Flask-based APIs

• apiflask: Lightweight framework based on Flask and Marshmallow.

Framework Agnostic

• fastapi: Modern, fast framework using standard Python type hints.
• falcon: High-performance framework for cloud APIs.
• sanic: High-performance web server and framework.
• connexion: Spec-first framework using OpenAPI specifications.
• strawberry: GraphQL library leveraging Python type annotations.
• webargs: Parsing HTTP request arguments.

Select tools for processing, analyzing, and ingesting data:

Data Analysis

• Core Libraries: pandas (High-performance structures), polars (Fast Rust-based DataFrame), and ibis (Portable API for 20+ backends).
• Scaling & Profiling: modin (Drop-in pandas replacement), aws-sdk-pandas (Pandas on AWS), data-profiling (Pandas profiling), desbordante (Complex pattern discovery), and percentify (Stats/quality diagnostics for pandas/Polars).
• Real-time: pathway (Reactive dataflow framework).
• Exploration: datasette (Exploring and publishing data).

Data Ingestion / ETL

• General: dlt (Data Load Tool) for pipelines with automatic schema inference.
• Financial Data: akshare (Financial interface), edgartools (SEC EDGAR/XBRL), lumibot (Algorithmic trading), openbb (Financial platform), and yfinance (Yahoo Finance data).