aYu's Personal Documentation

website·Indexed 30 days ago

http://docs.ayuisokay.com

A collection of personal technical notes covering algorithm problem solutions (including sorting, linked lists, and binary trees), development tips, interview experiences, and post-mortems of work-related issues such as Dataworks data synchronization and system failures.

Tokens
8.4K
Snippets
29
Records
37
Agent score
49%

What's inside aYu's Personal Docs

  1. Understand Quick Sort logic

    Quick Sort selects a 'pivot' element and partitions the array such that all elements smaller than the pivot are moved to its left, and all elements larger than or equal to the pivot are moved to its right. This process is applied recursively to the resulting subarrays until the entire array is sorted.
  2. Find the intersection node of two linked lists

    To find the intersection point of two singly linked lists, use two pointers starting at the heads of list A and list B. When a pointer reaches the end of its list, redirect it to the head of the opposite list. If the lists intersect, the pointers will eventually meet at the intersection node. If they do not intersect, both pointers will reach null simultaneously.
  3. Prevent data duplication in time-windowed batch processing

    When implementing T+H (incremental) data processing, ensure that time-window filters are strictly defined. A common failure occurs when a query is modified to fetch data from both 'before' and 'after' a specific timestamp (e.g., +/- 2 hours) instead of a strict forward-looking window. If combined with out-of-order execution or failed deduplication tables, this can result in massive duplicate record generation.
  4. Fix data misalignment in DataWorks offline synchronization using Name Mapping

    DataWorks offline synchronization may default to 'Position Mapping' (同行映射), which maps fields based on their order. If the field order in the ODPS table differs from the target database table, data may be misaligned or fail to load. To resolve this, manually change the mapping configuration to 'Name Mapping' (同名映射) to ensure fields are matched by their names regardless of order.
  5. Handle concurrency issues during DataWorks insert-update operations

    Be aware that concurrent insert update operations on the same primary key in a production database may not always execute sequentially. If two different batches attempt to update the same record simultaneously, the resulting record may contain a mix of field values from both batches, leading to corrupted data. To resolve this, it is recommended to reset the corrupted data to a known correct state and re-insert missing records with corrected primary keys.
  6. Troubleshoot missing data in DataWorks offline synchronization

    If a DataWorks offline synchronization node completes without errors in the primary log but no data is transferred from ODPS to the target database, check the 'detailed logs' (详细日志). The primary log may not display all errors; the detailed log provides a link to a more comprehensive log where specific failures, such as field length limit violations, are recorded.
  7. Avoid using functions on indexed columns in database queries

    When designing data extraction queries (e.g., in DataWorks), avoid placing database columns inside functions within the WHERE clause. Doing so causes index failure, which can lead to severe performance degradation or database timeouts when processing large datasets (e.g., hundreds of thousands to millions of rows).
  8. Perform a level-order traversal of a binary tree

    Level-order traversal visits nodes level by level, from left to right. This is implemented using a Queue (FIFO). The root is added to the queue, and for each level, the current size of the queue is recorded to ensure only nodes of that specific level are processed before moving to the next level. Children of processed nodes are added to the queue for the subsequent level.
    class Solution {
        public List<List<Integer>> levelOrder(TreeNode root) {
            if(root == null) return new ArrayList<>();
            // Result list
            List<List<Integer>> lists = new ArrayList<>();
            // Node queue
            Queue<TreeNode> queue = new LinkedList<>();
            // Add root node
            queue.add(root);
            while(!queue.isEmpty()){
                // Use size to ensure only current level nodes are traversed
                int size = queue.size();
                List<Integer> list = new ArrayList<>();
                while(size-- > 0){
                    TreeNode temp = queue.poll();
                    list.add(temp.val);
                    // Add left and right children to queue
                    if(temp.left != null) queue.add(temp.left);
                    if(temp.right != null) queue.add(temp.right);
                }
                lists.add(list);
            }
            return lists;
        }
    }
  9. Find the intersection node of two singly-linked lists

    To find the intersection node of two linked lists, use two pointers starting at the heads of list A and list B. When a pointer reaches the end of its list, redirect it to the head of the opposite list. If an intersection exists, the pointers will meet at the intersection node. If no intersection exists, both pointers will eventually reach null simultaneously.
    public class Solution {
        public ListNode getIntersectionNode(ListNode headA, ListNode headB) {
            if(headA == null || headB == null) return null;
            ListNode p = headA;
            ListNode q = headB;
            while(p != q) {
                p = p == null ? headB : p.next;
                q = q == null ? headA : q.next;
            }
            return p;
        }
    }
  10. Reverse a linked list from position left to right

    To reverse a specific segment of a linked list between positions left and right:

    1. Use a dummy head to handle cases where the reversal starts at the first node.
    2. Locate the node immediately preceding the reversal range (superior).
    3. Reverse the nodes within the range using a standard reversal loop.
    4. Reconnect the reversed segment: connect the original start of the segment (now the tail) to the node following the range, and connect the superior node to the new head of the reversed segment.
    class Solution {
        public ListNode reverseBetween(ListNode head, int left, int right) {
            ListNode dummyHead  = new ListNode(-1);
            dummyHead.next = head;
            ListNode superior = dummyHead;
            for(int i = 1; i < left; i++) {
                superior = superior.next;
            }
            ListNode curr = superior.next;
            ListNode prev = null;
            for(; left <= right; left++) {
                ListNode temp = curr.next;
                curr.next = prev;
                prev = curr;
                curr = temp;
            }
            superior.next.next = curr;
            superior.next = prev;
            return dummyHead.next;
        }
    }
  11. Find the middle node of a linked list

    Use the fast and slow pointer technique: the fast pointer moves two steps for every one step the slow pointer moves. When the fast pointer reaches the end, the slow pointer will be at the middle. For lists with an even number of nodes, the implementation ensures the second middle node is returned.
    /**
     * Definition for singly-linked list.
     * public class ListNode {
     *     int val;
     *     ListNode next;
     *     ListNode() {}
     *     ListNode(int val) { this.val = val; }
     *     ListNode(int val, ListNode next) { this.val = val; this.next = next; }
     * }
     */
    class Solution {
        public ListNode middleNode(ListNode head) {
            ListNode fast = head;
            ListNode slow = head;
            while (fast.next != null && fast.next.next !=null) {
                fast = fast.next.next;
                slow = slow.next;
            }
            if(fast.next == null) return slow;
            return slow.next;
        }
    }
  12. Rotate a singly-linked list to the right by k places

    To rotate a singly-linked list to the right by k positions, first determine the length of the list. Since rotating a list of length deep by deep times results in the original list, only k % deep rotations are necessary. The process involves: 1. Calculating the list length and identifying the tail node. 2. Connecting the tail node to the head to create a circular linked list. 3. Finding the new tail node at position deep - (k % deep) and the new head node immediately following it. 4. Breaking the circular connection by setting the new tail's next pointer to null.
    /**
     * Definition for singly-linked list.
     * public class ListNode {
     *     int val;
     *     ListNode next;
     *     ListNode() {}
     *     ListNode(int val) { this.val = val; }
     *     ListNode(int val, ListNode next) { this.val = val; this.next = next; }
     * }
     */
    class Solution {
        public ListNode rotateRight(ListNode head, int k) {
            // Handle cases where rotation is not needed
            if (head == null || k == 0 || head.next == null) {
                return head;
            }
            
            // Find the length of the list and the last node
            ListNode temp = head;
            int deep = 1;
            while (temp.next != null) {
                temp = temp.next;
                deep++;
            }
            
            // Calculate the actual number of rotations needed
            int n = deep - k % deep;
            if (n == deep) {
                return head;
            }
            
            // Make the list circular
            temp.next = head;
            
            // Move to the node preceding the new head
            while (n > 0) {
                temp = temp.next;
                n--;
            }
            
            // Set the new head and break the circle
            ListNode res = temp.next;
            temp.next = null;
            
            return res;
        }
    }