DOCSEARCHbyAlgolia
Submit Docs

CPython Reference Implementation Documentation

repository·main·Indexed 32 days ago

https://github.com/python/cpython

Documentation for the CPython reference implementation of the Python programming language. Covers the C API for interpreter configuration, embedding Python, and customizing runtime behavior. Includes technical guides for Mypy type-checking path symlinks, building Python with Tcl/Tk and OpenSSL on RHEL/CentOS, HACL* cryptographic implementations in hashlib, mimalloc event tracing on Windows, and using the PC/layout script for creating Python distributions (NuGet, APPX, Embeddable, etc.).

Tokens
1.1M
Snippets
1.3K
Records
7.9K
Agent score
97%

What's inside python-cpython

  1. Overview of the _decimal module

    main
    The _decimal module is a C wrapper for the libmpdec library. It provides fast, correctly-rounded, arbitrary-precision decimal floating-point arithmetic. It is a complete implementation of the General Decimal Arithmetic Specification by Mike Cowlishaw/IBM.

  2. Overview of Internet Protocols and Support modules

    main

    Python provides a suite of modules for implementing internet protocols and related technologies. Most of these modules are implemented in Python and rely on the system-dependent socket module, which is supported on most popular platforms.

    Key functional areas include:

    • Web Browsing & HTTP: webbrowser, http, http.client, http.server, http.cookies, http.cookiejar, and urllib (including urllib.request, urllib.parse, urllib.error, and urllib.robotparser).
    • Web Server Interfaces: wsgiref.
    • Email Protocols: smtplib (SMTP), poplib (POP3), and imaplib (IMAP).
    • File Transfer: ftplib (FTP).
    • Remote Procedure Calls: xmlrpc and xmlrpc.client.
    • Networking & IP: socketserver, ipaddress, and uuid.

  3. Getting started with the Python/C API

    main

    The Python/C API allows C and C++ programmers to interact with the Python interpreter. It is primarily used for two purposes:

    1. Writing extension modules: Creating C modules that extend the Python interpreter (the most common use case).
    2. Embedding Python: Using Python as a component within a larger C or C++ application.

    Language Compatibility

    • Compatible with C11 and C++11.
    • You do not need to enable specific C11 modes in your compiler.
    • For C++ users, the API is defined using extern "C", so no special configuration is required to use it from C++.

  4. Overview of the email package

    main

    The email package is used for parsing, manipulating, and generating email messages. It is designed to be RFC-compliant (supporting RFC 5322, RFC 6532, and various MIME RFCs) but does not send emails. To send emails via SMTP, use the smtplib module instead.

    The package consists of four main components:

    1. Object Model (email.message): The central interface. It represents email messages as a tree structure of EmailMessage objects. You use this to construct new emails, query existing ones, or manage subcomponents (like attachments).
    2. Parser (email.parser): Converts serialized email messages (byte streams) into a tree of EmailMessage objects.
    3. Generator (email.generator): Converts an EmailMessage object back into a serialized byte stream.
    4. Policy (email.policy): Controls the behavior of the parser, generator, and message objects. You typically specify a policy when creating a new message or parsing an input stream to define how the message should behave (e.g., using standard SMTP settings during serialization).

  5. Introduction to the `decimal` module

    main

    The decimal module provides support for fast, correctly rounded decimal floating-point arithmetic. It is designed to follow the General Decimal Arithmetic Specification and is preferred over the built-in float type for applications requiring exactness, such as accounting or monetary calculations.

    Key advantages include:

    • Exact Representation: Unlike binary floating point, numbers like 1.1 and 2.2 can be represented exactly.
    • Predictable Arithmetic: Operations like 0.1 + 0.1 + 0.1 - 0.3 result in exactly 0.0.
    • Significant Places: The module preserves trailing zeros to indicate significance (e.g., 1.30 + 1.20 results in 2.50).
    • User-Alterable Precision: You can control the precision (default is 28 places) for any given problem.

  6. Overview of idlelib and IDLE architecture

    main

    IDLE (Integrated Development and Learning Environment) is Python's built-in IDE. The idlelib package contains the implementation details for the editor, shell, debugger, and configuration systems.

    Key architectural components include:

    • Startup: __main__.py is the entry point when running with -m idlelib.
    • Implementation: Modules like editor.py, pyshell.py, and debugger.py handle core functionality.
    • Configuration: Uses .def files (e.g., config-keys.def, config-highlight.def) to manage defaults for keybindings, colors, and fonts.
    • Extensions: IDLE supports extensions that add submenu items to the main menu.

    For developers, the idlelib implementation follows PEP 8 guidelines, specifically regarding import grouping (stdlib, tkinter, and idlelib).

  7. Overview of the cases_generator tooling

    main

    The cases_generator toolset is used to process the instruction definitions defined in Python/bytecodes.c (the DSL) and generate various C and Python files required for the CPython interpreter. This includes opcode IDs, metadata, and dispatch targets.

    Key components include:

    • Instruction Definition Processing:

      • tierN_generator.py: Reads Python/bytecodes.c to write Python/generated_cases.c.h and other files.
      • optimizer_generator.py: Reads Python/bytecodes.c and Python/optimizer_bytecodes.c to write Python/optimizer_cases.c.h.
      • opcode_id_generator.py: Generates opcode lists in Include/opcode_ids.h.
      • opcode_metadata_generator.py: Writes metadata to Include/internal/pycore_opcode_metadata.h.
      • py_metadata_generator.py: Writes metadata to Lib/_opcode_metadata.py.
      • target_generator.py: Generates targets for computed goto dispatch in Python/opcode_targets.h.
      • uop_id_generator.py: Generates uop IDs in Include/internal/pycore_uop_ids.h.
      • uop_metadata_generator.py: Writes uop metadata to Include/internal/pycore_uop_metadata.h.

    • Parsing and Lexing Infrastructure:

      • lexer.py: C lexer.
      • plexer.py: OO interface for the lexer (main class: PLexer).
      • parsing.py: Parser for the instruction definition DSL (main class: Parser).
      • analyzer.py: Converts AST from the Parser into a high-level structure.
      • cwriter.py: Formats C code based on tokens (main class: CWriter).
      • stack.py: Handles generalized stack effects.

  8. Overview of XML processing modules in Python

    main

    Python provides several modules for processing XML, grouped under the xml package. Note that these modules require a SAX-compliant XML parser; Python includes the Expat parser via xml.parsers.expat by default.

    Available submodules include:

    • xml.etree.ElementTree: A simple and lightweight ElementTree API.
    • xml.dom: The DOM API definition.
    • xml.dom.minidom: A minimal implementation of the DOM API.
    • xml.dom.pulldom: Support for building partial DOM trees.
    • xml.sax: SAX2 base classes and convenience functions.
    • xml.parsers.expat: Bindings for the Expat parser.

  9. Overview of wsgiref utilities

    main

    The wsgiref module is a reference implementation of the Web Server Gateway Interface (WSGI) specification (PEP 3333). It provides utilities for manipulating WSGI environment variables and response headers, base classes for implementing WSGI servers, a demo HTTP server, and a validation tool for conformance checking.

    Warning: wsgiref is a reference implementation and is not recommended for production use as it only implements basic security checks.

  10. Overview of the xml.dom module

    main

    The xml.dom module provides a Document Object Model (DOM) API for accessing and modifying XML documents. It represents an XML document as a tree structure, allowing for random-access manipulation of nodes, elements, and attributes. This implementation is substantially based on the W3C DOM Level 2 recommendation.

    If you need a middle ground between the event-driven SAX model and the full DOM (e.g., when you cannot load the entire tree into memory), consider using xml.dom.pulldom.

  11. Overview of asyncio high-level APIs

    main

    The asyncio module provides high-level APIs for managing asynchronous programming in Python. These APIs are categorized into several functional areas:

    • Tasks: Utilities for running programs, creating Task objects, managing concurrency with gather or TaskGroup, and handling timeouts with wait_for or timeout.
    • Queues: FIFO (Queue), priority (PriorityQueue), and LIFO (LifoQueue) queues for distributing work between tasks or implementing pub/sub patterns.
    • Subprocesses: Tools to spawn subprocesses and run shell commands asynchronously using create_subprocess_exec or create_subprocess_shell.
    • Streams: High-level network I/O APIs for TCP and Unix socket connections, including open_connection, start_server, and the StreamReader/StreamWriter objects.
    • Synchronization: Threading-like primitives for tasks, such as Lock, Event, Condition, Semaphore, BoundedSemaphore, and Barrier.
    • Exceptions: Specific error types like asyncio.CancelledError (raised when a task is cancelled) and asyncio.BrokenBarrierError (raised when a barrier is broken).

  12. Overview of Python 2.7 features and migration

    main

    Python 2.7 is the final major release of the 2.x series. It serves as a bridge to Python 3 by backporting several Python 3.1 features.

    Key Backported Features:

    • Set literal syntax (e.g., {1, 2, 3}).
    • Dictionary and set comprehensions.
    • Multiple context managers in a single with statement.
    • The collections.OrderedDict class.
    • The collections.Counter class.
    • The argparse module for command-line parsing.
    • The io library (rewritten in C).
    • The memoryview object.
    • New " , " format specifier for thousands separators.

    Migration Note: While many changes can be automated, migrating Unicode handling requires careful consideration and robust regression testing. For migration guidance, refer to the pyporting HOWTO.