Python Mastery Advanced Programming Course

repository·main·Indexed 11 days ago

https://github.com/dabeaz-course/python-mastery

An exercise-driven, advanced Python programming course designed to help developers build a deep mental model of the language and its internal mechanics. Targeting Python 3.6, the course includes instructional slides, data files, and a comprehensive set of exercises and solutions covering topics such as file I/O, error handling, and data structure manipulation.

Tokens
76.7K
Snippets
248
Records
272
Agent score
96%

What's inside Python Mastery

  1. Course content and directory structure

    main

    The repository is organized into the following key directories and files:

    • PythonMastery.pdf: The core instructional material containing presentation slides, exercises, and timings.
    • Exercises/: Contains all course exercises (see Exercises/index.md for details).
    • Solutions/: Contains the complete solution code for the exercises.
    • Data/: Contains data files required for various course exercises.
  2. Understand the behavior of 'from module import symbol'

    main

    When using from module import symbol, the entire module is still loaded into memory, but the module object itself is not added to your local namespace. Only the specific names (variables, functions, classes) you requested are available directly.

    Important Note on Name References: When you import a variable using from module import x, you are creating a local name reference to the object. If the module's internal state changes (e.g., the module reassigns x to a new object), your local reference x will still point to the original object. This can lead to inconsistencies between local variables and the module's current state.

    # If simplemod.x is 42
    from simplemod import x, foo
    
    x = 13       # This reassigns the LOCAL name 'x', not simplemod.x
    foo()        # If foo() uses 'import simplemod; simplemod.x', it still sees 42
  3. Extend formatters with ColumnFormatMixin and UpperHeadersMixin

    main

    The system uses Mixins to add functionality to formatters without modifying their core logic:

    1. ColumnFormatMixin: When applied, it overrides the row() method. It iterates through rowdata and applies a list of format strings stored in the formats attribute to each corresponding item before passing the data to the base class row() method.
    2. UpperHeadersMixin: When applied, it overrides the headings() method to convert all provided header strings to uppercase before passing them to the base class headings() method.
  4. How `validate_attributes` automates class construction

    main

    The validate_attributes function is a class decorator that performs several automated setup steps for a class:

    1. Validator Scanning: It scans the class attributes for instances of Validator. These are collected into cls._fields (using v.name) and cls._types (using v.expected_type or an identity lambda lambda x: x).
    2. Callable Validation: It identifies any callable (like a function) that has type annotations (__annotations__) and wraps it using the validated() decorator.
    3. Dynamic __init__ Generation: If _fields are found, it calls cls.create_init() to dynamically construct and inject an __init__ method that assigns the fields to self.
    4. Attribute Protection: It works in tandem with Structure.__setattr__ to ensure only defined fields can be set.
  5. Understand Method Resolution Order (MRO) and super() in Python

    main

    Python uses a Method Resolution Order (MRO) to determine the sequence in which classes are searched when a method is called. This is accessible via the __mro__ attribute.

    Single Inheritance

    In a linear hierarchy, super() moves up the chain from child to parent.

    Multiple Inheritance and Cooperative Inheritance

    In multiple inheritance, super() does not necessarily delegate to the immediate parent of the current class. Instead, it moves to the next class in the MRO. The order of the MRO is determined by the order of classes provided in the subclass definition.

    Key behaviors:

    • The child class controls the order of the MRO.
    • super() follows the MRO sequence, allowing different classes to be composed together.
    • A common base class can serve as a terminator for a chain of super() calls if its methods do not call super().
    >>> class Base:
            def spam(self):
                print('Base.spam')
    
    >>> class X(Base):
            def spam(self):
                print('X.spam')
                super().spam()
    
    >>> class Y(Base):
            def spam(self):
                print('Y.spam')
                super().spam()
    
    >>> class Z(Base):
            def spam(self):
                print('Z.spam')
                super().spam()
    
    >>> class M(X, Y, Z):
            pass
    
    >>> M.__mro__
    (<class '__main__.M'>, <class '__main__.X'>, <class '__main__.Y'>, <class '__main__.Z'>, <class '__main__.Base'>, <class 'object'>)
    >>> m = M()
    >>> m.spam()
    X.spam
    Y.spam
    Z.spam
    Base.spam
  6. Convert methods to computed attributes using properties

    main

    You can use the @property decorator to turn a method into a read-only attribute. This allows you to access computed values without using parentheses, providing a cleaner interface for the consumer.

    # Before: method call
    s.cost()
    
    # After: property access
    s.cost
  7. Access nested lists

    main

    Lists can contain any object, including other lists. To access elements within a nested list, use chained indexing: list[outer_index][inner_index].

    nums = [101, 102, 103]
    symlist = ['AA', 'AAPL']
    items = [symlist, nums]
    
    # Accessing the first element of the first list
    # items[0] is ['AA', 'AAPL']
    # items[0][1] is 'AAPL'
    val = items[0][1]
  8. Control exported symbols using __all__

    main

    You can control which symbols are exported from a submodule by defining an __all__ list. This is useful for hiding internal implementation details and providing a cleaner API. When a submodule defines __all__, only the names listed in that variable are exported when using from submodule import * or when the submodule is imported into a package's __init__.py via from .submodule import *.

    # Example: structure.py
    __all__ = ['Structure']
    
    class Structure:
        pass
    
    # Example: reader.py
    __all__ = ['read_csv_as_instances', 'read_csv_as_json']
    
    def read_csv_as_instances():
        pass
    
    def read_csv_as_json():
        pass
  9. Implement data validation using Descriptors

    main

    You can implement attribute-level validation in Python classes by using the Descriptor protocol. By defining a class with a __set__ method, you can intercept attribute assignments and run validation logic via a check method.

    Key components of this pattern:

    • __set_name__(self, cls, name): Automatically captures the name of the attribute being assigned in the owner class.
    • __set__(self, instance, value): Intercepts the assignment and applies validation logic before updating the instance's __dict__.
    • check(cls, value): A class method used to perform the actual validation logic, which can be overridden by subclasses to implement specific constraints (like type checking or value ranges).
    class Validator:
        def __init__(self, name=None):
            self.name = name
    
        def __set_name__(self, cls, name):
            self.name = name
    
        @classmethod
        def check(cls, value):
            return value
    
        def __set__(self, instance, value):
            instance.__dict__[self.name] = self.check(value)
  10. Understand instance representation via __dict__

    main

    In Python, an instance is essentially a layer on top of a dictionary. You can inspect and manipulate the attributes of an instance by accessing its __dict__ attribute. This dictionary stores the instance-specific data (attributes) that were assigned during or after initialization.

    class SimpleStock:
        def __init__(self, name, shares, price):
            self.name = name
            self.shares = shares
            self.price = price
    
    goog = SimpleStock('GOOG', 100, 490.10)
    
    # Inspect the instance attributes
    print(goog.__dict__)
    # Output: {'name': 'GOOG', 'shares': 100, 'price': 490.1}
  11. Implement alternate constructors using @classmethod

    main

    You can use the @classmethod decorator to create alternate constructors for a class. This is useful for initializing an object from a different data format, such as a raw list or a CSV row. By using a class method like from_row(cls, row), you can encapsulate the logic for data conversion and type casting within the class itself, rather than performing it in the caller or the __init__ method.

    A common pattern is to use a class variable (e.g., types) to define the expected types for each field, allowing the class method to iterate through the row and apply the corresponding conversion functions.

    class Stock:
        types = (str, int, float)
        def __init__(self, name, shares, price):
            self.name = name
            self.shares = shares
            self.price = price
    
        @classmethod
        def from_row(cls, row):
            values = [func(val) for func, val in zip(cls.types, row)]
            return cls(*values)
  12. Understand the Descriptor Protocol

    main

    Descriptors are objects that manage attribute access (getting, setting, and deleting) for other objects. They implement the descriptor protocol using specific magic methods. When you access an attribute on an instance, if that attribute is a descriptor defined in the class, Python automatically invokes the descriptor's methods instead of the default dictionary lookup.

    Key methods in the protocol:

    • __get__(self, instance, cls): Invoked when the attribute is accessed.
    • __set__(self, instance, value): Invoked when the attribute is assigned a value.
    • __delete__(self, instance): Invoked when the attribute is deleted.
    class Descriptor:
        def __init__(self, name):
            self.name = name
        def __get__(self, instance, cls):
            print('%s:__get__' % self.name)
        def __set__(self, instance, value):
            print('%s:__set__ %s' % (self.name, value))
        def __delete__(self, instance):
            print('%s:__delete__' % self.name)
    
    class Foo:
        a = Descriptor('a')
    
    f = Foo()
    f.a  # Triggers __get__
    f.a = 23  # Triggers __set__
    del f.a  # Triggers __delete__