Marker Document Intelligence Tool

A Switch Transformer is a sparsely activated Transformer model designed to maximize parameter count while keeping floating point operations (FLOPs) per example constant.

Unlike dense Transformers, Switch Transformers replace the standard dense feed-forward network (FFN) layer with a sparse Switch FFN layer. This layer routes each token independently to a single expert (a specific FFN) based on a router's decision. This allows the model to increase its total parameter count (by adding more experts) without increasing the computational cost per token, as each token only interacts with one expert per layer.

Boolean expressions evaluate to True or False (type bool).

Relational Operators:

• ==: Equal to
• !=: Not equal to
• >: Greater than
• <: Less than
• >=: Greater than or equal to
• <=: Less than or equal to

Note: Use == for comparison and = for assignment. Do not use =< or =>.

Logical Operators:

• and: True if both operands are true.
• or: True if at least one operand is true.
• not: Negates the boolean expression.

# Relational
5 == 5  # True
5 == 6  # False

# Logical
(5 > 0 and 5 < 10)  # True
(5 % 2 == 0 or 5 % 3 == 0)  # False
not (5 > 10)  # True

Python offers different ways to access code from other modules:

1. Standard Import: Using import <module_name> creates a module object. You must use dot notation to access its contents (e.g., module.constant).
2. Specific Import: Using from <module_name> import <object> allows you to access specific functions or constants directly without the module prefix.
3. Wildcard Import: Using from <module_name> import * imports all objects from the module into the current namespace.

Warning: Wildcard imports can cause name conflicts between different modules or between a module's contents and your own variables.

# 1. Standard Import
import math
print(math.pi)

# 2. Specific Import
from math import pi
print(pi)

# 3. Wildcard Import
from math import *
print(cos(pi))

Attributes are named elements associated with an object. You can assign values to attributes or read them using dot notation (object.attribute).

Attributes can be simple values (like floats) or other objects (known as embedded objects).

# Assigning attributes
blank.x = 3.0
blank.y = 4.0

# Reading attributes
print(blank.x)

# Accessing embedded attributes
# If 'box' has an attribute 'corner' which is a Point object:
print(box.corner.x)

In Python, a user-defined type is called a class. A class acts as a factory for creating instances (objects).

To define a class, use the class keyword. The class object can then be called like a function to create a new instance, a process known as instantiation.

class Point(object):
    """Represents a point in 2-D space."""
    pass

# Instantiation
blank = Point()
print(blank)

To prevent errors (like IOError from missing files or permission issues) from terminating your program, use a try statement. Python executes the code inside the try block; if an exception occurs, it immediately jumps to the except block.

This pattern is known as catching an exception and allows you to handle errors gracefully, retry operations, or exit cleanly.

try:
    fin = open('bad_file')
    for line in fin:
        print line
    fin.close()
except:
    print 'Something went wrong.'

An update is a form of multiple assignment where the new value of a variable depends on its current value.

Common patterns:

• Increment: Adding 1 to a variable (x = x + 1).
• Decrement: Subtracting 1 from a variable (x = x - 1).

Requirement: You must initialize a variable (assign it an initial value) before you can update it. Attempting to update an undefined variable will result in a NameError because Python evaluates the right side of the assignment before assigning the result to the variable.

# This will fail with NameError: name 'x' is not defined
x = x + 1

# Correct way: Initialize first
x = 0
x = x + 1

To reduce debugging time in large programs, use "debugging by bisection." Instead of checking every line, find a logical midpoint in your program where you can verify an intermediate value (e.g., by adding a print statement).

• If the midpoint check is incorrect, the bug is in the first half of the program.
• If the midpoint check is correct, the bug is in the second half.

By repeatedly halving the search space, you can quickly narrow down the location of an error.

Class diagrams are abstract representations of a program's structure, showing classes and their relationships rather than individual objects. There are three primary types of relationships:

• IS-A (Inheritance): A child class inherits from a parent class (e.g., Hand is a kind of Deck). In diagrams, this is represented by an arrow with a hollow triangle head.
• HAS-A (Composition/Aggregation): One class contains references to objects of another class (e.g., a Deck has many Cards). This is represented by a standard arrow head.
• Dependency: One class depends on another in that changes to one require changes to the other.

Multiplicity: Indicated by a symbol (like * or a number like 52) near the arrow head, it specifies how many instances of one class are referenced by another.

Refactoring is the process of rearranging a program to improve function interfaces and facilitate code reuse.

If you find similar code in multiple functions (e.g., polygon and arc), you can "factor out" the common logic into a more general function (e.g., polyline).

Refactoring Pattern

1. Identify common logic between functions.
2. Create a new, highly general function (e.g., polyline(t, n, length, angle)).
3. Rewrite the original functions to call the new general function using specific parameters.

Layouts are managed by packing widgets into Frames. Gui.py uses a stack-based approach for Frames:

• row(): Creates a horizontal row Frame.
• col(): Creates a vertical column Frame.
• gr(cols=N): Creates a grid Frame with N columns. Widgets are placed left-to-right, top-to-bottom.
• endrow(), endcol(), endgr(): Closes the current Frame and returns to the previous one on the stack.

Use pady for vertical padding and provide a list of weights to row() to determine how extra space is allocated among widgets during resizing.

self.col() # Start a column
self.gr(cols=2) # Start a 2-column grid inside the column
self.bu(text='Button 1')
self.bu(text='Button 2')
self.endgr() # End the grid
self.endcol() # End the column