PaddlePaddle Deep Learning Platform

The Paddle High reusability operator library (PHI), also known as the 'functional operator library', is a refactored operator architecture designed to replace the original Paddle Fluid Framework operator library.

PHI provides a functional paradigm that allows developers to implement new operator kernels by combining existing kernel functions and the Kernel Primitives API (KPS). This design supports:

• High Reusability: Implementing new operators by calling functional interfaces instead of constructing complex ExecutionContext objects.
• Hardware Plug-ins: Support for new hardware or acceleration libraries via plug-in access.
• Unified Training and Inference: A single library that serves training, server-side inference, and mobile-side (Paddle-Lite) inference, reducing maintenance costs and bugs caused by implementation mismatches.
• Improved Performance: A function-based scheduling architecture that reduces the overhead of the dynamic graph and enables fine-grained execution scheduling for static graphs.

When implementing fusion kernels in PaddlePaddle, follow these architectural guidelines:

• Avoid Python APIs: Do not implement Python APIs for fusion kernels. They typically involve complex input/output arguments that are difficult to use in Python. Instead, trigger fusion kernels via optimization passes in dy2static mode or in static graph mode.
• Kernel Reuse: Do not reuse a fusion kernel as a building block for other kernels. Instead, implement fusion kernels by reusing existing individual kernels.
• Backend Specificity: Fusion kernels are intended to accelerate combined operations on specific backends. You are not required to implement them for all devices. If a kernel is only intended for a specific backend, append the backend name as a suffix to the kernel name (e.g., fused_matmul_onednn or fused_fc_xpu).
• Avoid Pseudo Kernels: Do not implement 'pseudo kernels' that simply throw exceptions. If a kernel is not required for a specific backend, it should simply not be implemented.
• Namespace Requirement: All fusion kernels must reside in the phi/fusion namespace.

The code checking process is divided into three main stages: Interface Extraction, Example Execution, and Result Comparison.

When using xdoctest, the responsibilities are distributed as follows:

1. Interface Extraction: Uses sampcd_processor_utils.py to extract docstrings.
2. Example Execution: Uses sampcd_processor_xdoctest.py (via Xdoctester) to run the extracted examples.
3. Result Comparison: Uses sampcd_processor_xdoctest.py (via Xdoctester) to compare results.

The execution flow follows these steps:

1. Initialize logger via init_logger.
2. Check test mode via check_test_mode.
3. Get test capacity via get_test_capacity.
4. Extract test docstrings via get_docstring.
5. Prepare the doctester via doctester.prepare(sample_code_test_capacity).
6. Run code checking and get results via get_test_results(doctester, docstrings_to_test).
7. Print summary via doctester.print_summary(test_results, whl_error).
8. (Optional) Generate documentation via exec_gen_doc().

The paddle/phi/kernels directory manages how operator logic is implemented across different backends:

• Root kernels/: Contains device-independent kernel declarations (kernel.h) and implementations (kernel.cc). If a kernel uses the Primitive API or reuses other kernels, it lives here.
• Device Subdirectories (cpu/, gpu/, xpu/, etc.): Contains the actual implementation for that specific backend. Note that gpu is used as a unified name for both cuda and hip to reduce code repetition.
• kernels/funcs/: Contains functors and functions that support multiple backends (legacy compatibility with original fluid operators).
• kernels/primitive/: Contains the Kernel Primitive API, which provides basic tools for unified multi-device kernel implementation.
• kernels/impl/: A special directory for kernel implementations that are shared between CPU and GPU but are not strictly device-independent. These are stored as header files with the xxx_kernel_impl.h suffix.

Implementation Details:

• Backward Kernels: Implementation of backward (gradient) kernels is placed in separate files with the *_grad_kernel.* suffix to facilitate split compilation (e.g., for inference-only builds).
• Auxiliary Functions: Functions used only by a specific kernel should be placed in the same backend folder as the kernel implementation.

paddle/phi/kernels
├── (Device-independent declarations/implementations)
├── cpu
├── gpu
├── xpu
├── onednn
├── gpudnn
├── impl (Shared CPU/GPU implementations, e.g., scale_kernel_impl.h)
├── funcs (Multi-device functors/functions)
└── primitive (Kernel Primitive API)

The PaddlePaddle 3.2 framework introduces several advanced capabilities designed for large-scale model development and scientific computing:

• Unified Dynamic/Static Execution with Automatic Parallelism: Automatically finds efficient distributed parallel strategies with minimal tensor splitting annotations, reducing the cost of industrial development and training.
• Unified Training and Inference for Large Models: Supports both training and inference within the same framework, enabling code reuse and seamless transitions for the entire large model lifecycle.
• High-Order Differentiation for Scientific Computing: Provides high-order automatic differentiation, complex number arithmetic, Fourier transforms, and compilation optimizations to accelerate differential equation solving in fields like mathematics, mechanics, and biology.
• Neural Network Compiler: Uses an integrated framework design to balance computational flexibility and high performance for generative and scientific models, lowering the cost of performance optimization.
• Heterogeneous Multi-core Adaptation: Provides a standardized interface to shield developers from the differences in software stacks across various hardware chips, enabling a plug-and-play architecture.

The JIT kernel implementation is organized into three main functional directories under paddle/phi/kernels/funcs/jit/:

• gen/: Contains code generated via JIT (using the xbyak library). This directory focuses on maximum performance.
• refer/: Contains the reference implementation. Every kernel MUST have a reference implementation here to serve as a baseline for correctness and unit testing.
• more/: Contains additional optimized implementations, such as those using mkl, mkldnn, openblas, or custom intrinsic sets. It can also contain combinations of existing kernels.

PaddlePaddle/Paddle/paddle/phi/kernels/
├── ...
└── funcs/
    ├── .../
    └── jit/
        ├── ...
        ├── gen/
        │   └── ...
        ├── more/
        │   ├── mkl/
        │   ├── mkldnn/
        │   ├── intrinsic/
        │   └── openblas/
        └── refer/
            └── ...