mypy

mypyc is a compiler that transforms Python modules into fast C extensions using standard Python type hints. It is designed to provide performance gains while maintaining a high degree of compatibility with standard Python.

Key Characteristics

• Performance: Existing type-annotated code typically runs 1.5x to 5x faster. Code specifically tuned for mypyc can achieve 5x to 10x speedups.
• Type System: Uses mypy for type checking and inference. It supports most features in the typing module, including generics, union types, and optional types.
• Compatibility: Compiled modules can import any standard Python module or third-party library. You can run compiled modules as normal interpreted Python modules during development.
• Runtime Safety: Unlike standard Python where type hints are ignored at runtime, mypyc enforces type annotations at runtime, raising TypeError if values do not match. This provides protection against certain types of memory corruption and bugs.

The librt.vecs module (part of the librt PyPI package) provides the vec[T] type, a low-level, growable array designed for high performance in code compiled with mypyc.

Key characteristics:

• Efficient Memory: Uses packed binary encoding for value types (integers, floats, bools).
• Immutable Length: The length of a vec is immutable. Operations that change the length (like append) return a new vec value.
• Performance: Optimized for mypyc to allow inlined get/set operations and efficient machine register allocation.
• Usage Note: While usable in interpreted Python, vec is primarily intended for mypyc-compiled code. In interpreted code, it is 'boxed' and may not offer performance benefits over list or array.array.

from librt.vecs import append, vec

v = vec[float]([1.0, 2.5])  # Construct vec[float] with two items

# Appending returns a new vec because length is immutable
v = append(v, -0.5)
print(v)  # vec[float]([1.0, 2.5, -0.5])

The librt.base64 module (part of the librt package) provides high-efficiency Base64 encoding and decoding using SIMD (Single Instruction, Multiple Data). It is a wrapper around Alfred Klomp's base64 library and is significantly faster than the Python standard library base64 module, particularly for large inputs or when used in code compiled with mypyc.

Key Differences from the standard base64 module:

• Performance: Much faster, especially for large inputs.
• Error Handling: When padding is incorrect, librt.base64 raises ValueError. The standard library raises binascii.Error (which is a subclass of ValueError).
• Argument Support: Only commonly used functionality is provided. The optional altchars and validate arguments are not supported.
• Malformed Data: Decode functions may behave differently than the standard library when encountering malformed data.

Mypy is a static type checker for Python. It uses type hints (defined in PEP 484) to find bugs in your programs without running them.

Key Concepts

• Static Checking: Unlike Python's dynamic nature where errors appear at runtime, mypy finds type mismatches before execution.
• Gradual Typing: You can add type hints to your codebase incrementally. You can type-check parts of your program while leaving others dynamic.
• Non-intrusive: Type hints act similarly to comments; they do not change how the Python interpreter executes your code.
• Advanced Type System: Supports type inference, generics, callable types, tuple types, union types, and structural subtyping.

# Example of a type error caught by mypy
number = input("What is your favourite number?")
print("It is", number + 1)  # error: Unsupported operand types for + ("str" and "int")

stubtest is a tool included with mypy designed to detect discrepancies between Python stub files (.pyi) and the actual runtime implementation of your code.

How it works

Stubtest imports your code and uses runtime introspection (via the inspect module) to analyze code objects. It then compares these runtime objects against the type annotations in your stub files. This makes it particularly effective for checking extension modules.

Key Limitations

• Dynamic only: It relies on runtime introspection and does not perform static analysis of your actual code. It cannot verify if a return type is accurately typed; it only checks if the runtime object matches the stub's description.
• Not a type checker: Use mypy for static type checking.
• Not a generator: Use stubgen or pyright --createstub to generate stubs, or monkeytype to generate stubs based on running code.
• No code transformation: It does not apply stubs to code to produce inline types (use retype or libcst for that).

Warning: stubtest will import and execute Python code from the packages it checks.

python3 -m mypy.stubtest library

To increase code re-usability in mixin classes, define a Protocol that describes the required interface of the host class. Use this Protocol as the self type in your mixin methods. This allows the mixin to be used by any class that implements the protocol, without requiring the mixin to inherit from the host class directly.

from typing import Protocol

class Lockable(Protocol):
    @property
    def lock(self) -> Lock: ...

class AtomicCloseMixin:
    # The mixin requires the host to satisfy the Lockable protocol
    def atomic_close(self: Lockable) -> int:
        with self.lock:
            # perform actions
            pass

class File:
    def __init__(self) -> None:
        self.lock = Lock()

# File satisfies Lockable, so it can use the mixin
class ValidFile(AtomicCloseMixin, File): pass

# A class without 'lock' will fail mypy checks when using the mixin
class Invalid:
    pass

b = ValidFile()
b.atomic_close()  # OK

c = Invalid()
c.atomic_close()  # Error: Invalid self type