DOCSEARCHbyAlgolia
Submit Docs

Chanmufeng Technical Blog

website·Indexed 31 days ago

https://www.chanmufeng.com

A technical knowledge base covering backend development, network programming, and system internals. Key topics include I/O multiplexing (select, poll, epoll), socket programming, MySQL locking mechanisms and MVCC, CPU pipelining and instruction reordering, and design principles like the Open-Closed Principle.

Tokens
64.9K
Snippets
195
Records
373
Agent score
54%

What's inside Chanmufeng

  1. Optimize queries using the MySQL Optimizer

    The MySQL Optimizer determines the most efficient way to execute a query. Since a single SQL statement can be executed in multiple equivalent ways, the Optimizer uses a cost-based model to select the execution plan with the lowest estimated cost. Key optimizations include:

    • Deciding which table to use as the driving table in joins.
    • Selecting the most efficient index when multiple options are available.
    • Performing subquery optimization, predicate rewriting, and join elimination.

  2. Kernel-level event flow from Network Card to CPU

    The process of receiving data and notifying epoll follows a tiered interrupt model to ensure system stability:

    1. Hardware Interrupt (Hard IRQ): The network card receives data and places it in associated memory. Once complete, it triggers a hard interrupt to notify the CPU.
    2. Software Interrupt (Soft IRQ): Because hard interrupts must be handled quickly to avoid system lag, the CPU triggers a soft interrupt. The more time-consuming and complex work of processing the data and executing the ep_poll_callback is handled by the soft interrupt handler.

  3. Run Cerebro on Apple M1 using a pre-built Docker image

    Since official Cerebro Docker images may not be compatible with Apple M1 (arm64) architecture, you can use a community-built image that utilizes the arm64v8 JDK 11 runtime to monitor Elasticsearch clusters.
    # Pull the arm64 compatible image
docker pull chanmufeng/cerebro:0.9.4

# Run the container mapping port 9000
docker run -p 9000:9000 chanmufeng/cerebro:0.9.4

  4. Optimize CPU Write Performance with Store Buffers

    To prevent the CPU from idling while waiting for Invalidate Acknowledge responses from other cores during a write operation, hardware engineers introduced Store Buffers.

    Instead of waiting for the cache coherence protocol to complete, the CPU writes the updated value into a Store Buffer (a layer between the CPU and the Cache) and immediately continues executing other instructions. Once the necessary acknowledgments are received from other CPUs, the value in the Store Buffer is then committed to the actual cache line.

  5. Understand the MySQL Parser and Preprocessor flow

    When a SQL query is submitted, MySQL processes it through the Parser and Preprocessor to ensure it is valid before execution. This happens in three steps:

    1. Lexical Analysis (Tokenization): The Parser breaks the SQL string into individual tokens (words), identifying keywords (e.g., SELECT), table names, and column names.
    2. Syntax Analysis: The Parser checks if the tokens follow MySQL's grammar rules (e.g., checking for closed quotes or correct keyword spelling). It then generates a data structure called a Parse Tree.
    3. Preprocessing: The Preprocessor checks the Parse Tree for semantic errors that the parser cannot detect, such as verifying if the specified tables and columns actually exist in the database and resolving aliases to ensure there is no ambiguity.

  7. Understand the low-level flow of local I/O operations

    Local I/O operations (such as copying a file) involve multiple transitions between user space and kernel space. A typical read/write operation follows these steps:

    1. The program initiates a read() system call, triggering a context switch from user mode to kernel mode.
    2. The operating system reads data from the disk into a kernel-space buffer.
    3. The kernel copies the data from the kernel-space buffer to the user-space buffer, switching back to user mode.
    4. For writing, the process repeats: a context switch to kernel mode occurs, and data is copied from the user-space buffer to the kernel-space buffer before being written to the disk.
    public static void main(String[] args) throws Exception {
    FileInputStream in = new FileInputStream("/tmp/file1.txt");
    FileOutputStream out = new FileOutputStream("/tmp/file2.txt");

    byte[] buf = new byte[in.available()];
    in.read(buf);
    out.write(buf);
}

  8. Understand the role of Sockets in network I/O

    Network I/O is implemented via Sockets. A Socket is not the network protocol itself, but a set of program components provided by the operating system's Socket library that allows user processes to interact with the kernel's protocol stack (e.g., TCP/IP).

    TCP communication follows a logical pipeline model consisting of four main stages:

    1. Server Setup: The server creates a socket and waits for a connection.
    2. Client Connection: The client creates a socket and connects to the server.
    3. Communication: Data is exchanged between the two sockets.
    4. Termination: The connection is closed and sockets are deleted.

  9. Understand JVM Young Generation memory structure

    The JVM Young Generation is divided into three main areas: the Eden space and two Survivor spaces (Survivor From and Survivor To).

    • Eden Space: The default allocation area for new objects. It typically occupies about 80% of the Young Generation.
    • Survivor Spaces: Two smaller areas of equal size used to hold objects that survive a Minor GC. They operate in a swap mechanism: during a Minor GC, live objects from Eden and the 'From' survivor space are copied to the 'To' survivor space, and then the roles of 'From' and 'To' are swapped.

  11. Understand the motivation for Dynamic Proxy over Static Proxy

    Static proxies require a separate proxy class for every target class and every specific logic enhancement, leading to 'class explosion' when dealing with many classes and multiple types of logic. Dynamic proxies solve this by generating proxy objects at runtime, allowing a single mechanism to add common logic (e.g., permission checks, transaction management, or logging) to any method of any object without the user needing to explicitly create each proxy class.

  12. Manage data caching with the InnoDB Buffer Pool

    The Buffer Pool is a continuous region of memory allocated by InnoDB to cache pages from disk.

    • Read Process: InnoDB first checks if the required page is in the Buffer Pool. If yes, it reads from memory; otherwise, it loads the page from disk into the Buffer Pool.
    • Write Process: Updates are first written to the Buffer Pool. Pages modified in memory but not yet written to disk are called 'dirty pages'.
    • Flushing: Background threads periodically synchronize dirty pages to disk, a process known as 'flushing' (刷脏).