HiF's Technical Blog

website·Indexed 31 days ago

https://hif.icu/

A collection of technical guides and troubleshooting notes covering Java development, SpringBoot dynamic data source implementation, MySQL backup strategies (including Docker and Qiniu Cloud OSS), Git development workflows v1, and Linux server administration including Redis installation and time synchronization.

Tokens
6.2K
Snippets
23
Records
33
Agent score
53%

What's inside HiF Blog

  1. Implement dynamic data source switching in Spring Boot using AbstractRoutingDataSource

    To support multiple database connections (common in SaaS architectures where each tenant has a separate database), Spring Boot provides the AbstractRoutingDataSource class. This class acts as a router that maintains a map of all available data sources. Before executing a database connection, Spring Boot calls the determineCurrentLookupKey() method; the value returned by this method serves as the key to retrieve the specific data source from the map for the current request or thread.
  2. Automate MySQL backups for Docker containers using a Bash script

    To perform scheduled backups of MySQL databases running in Docker, you can use a bash script that iterates through all available databases (excluding system schemas) and exports them using mysqldump and gzip. The script also handles the deletion of backups older than a specified number of days to save disk space.
    #!/bin/bash
    # Container ID or Name
    container_name="73eadb6f35e2"
    # Database credentials
    mysql_name="root"
    mysql_pwd="123456"
    # Backup directories
    backup_dir="/data/mysqlBak/data"
    bakup_log="/data/mysqlBak/log"
    # Retention period in days
    backup_clean_day=5
    
    # Get list of databases, excluding system schemas
    DATABASES=$(/usr/bin/docker exec -i ${container_name} mysql -u${mysql_name} -p${mysql_pwd} -e "show databases" | grep -Ev "Database|sys|information_schema|performance_schema|mysql|brp_log")
    
    # Create backup directory if it doesn't exist
    if [ ! -d $backup_dir ]; then
        mkdir -p $backup_dir
    fi
    
    # Loop through databases and backup
    for db in $DATABASES
    do
        echo "----------${db}_$(date "+%Y_%m_%d_%H_%M_%S").sql.gz BEGIN----------"
        /usr/bin/docker exec -i ${container_name} mysqldump -u${mysql_name} -p${mysql_pwd} --default-character-set=utf8mb4 --single-transaction --source-data=2 --flush-logs --hex-blob --triggers --routines --events --databases ${db} | gzip > ${backup_dir}/${db}_$(date "+%Y_%m_%d_%H_%M_%S").sql.gz
        
        # Log the backup
        echo "create $bakup_log/${db}-$(date "+%Y_%m_%d_%H_%M_%S").dupm" >> $bakup_log/log.txt
        echo "----------${db}_$(date "+%Y_%m_%d_%H_%M_%S").sql.gz COMPLETE----------"
    done
    
    echo "done"
    
    # Delete backups older than X days
    find ${backup_dir} -type f -mtime +${backup_clean_day} -delete
  3. Resolve MySQL collation mismatch via global or table settings

    Depending on the scope of the issue, you can resolve collation mismatches using a global connection setting or by converting a specific table.
    -- Option 1: Set the default connection collation globally
    SET GLOBAL COLLATION_CONNECTION = 'utf8mb4_general_ci';
    
    -- Option 2: Convert a specific table to a consistent collation
    ALTER TABLE tableName CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
  4. Prepare environment for MySQL Docker backups

    Before running the backup script, identify the target container and prepare the storage directories for the backup files and logs.
    # 1. Find the MySQL container ID
    docker ps
    
    # 2. Create backup and log directories
    mkdir -p /data/mysqlBak/data /data/mysqlBak/log
  5. Master branch merge cycle

    The develop branch is merged into the master branch during the last few working days of every month.
  6. Install Redis using Docker on Linux

    To install Redis on a Linux server using Docker, pull the desired official image from Docker Hub and run the container with mapped ports and volumes for configuration and data persistence. The example uses Redis version 7.0.11.
    # Pull the specific Redis image
    sudo docker pull redis:7.0.11
    
    # Run the Redis container
    sudo docker run -p 6379:6379 \
      --name redis \
      -v /docker/redis/conf/redis.conf:/etc/redis/redis.conf \
      -v /docker/redis/data:/data \
      -d redis:7.0.11 \
      redis-server /etc/redis/redis.conf --appendonly yes
  7. Switch data sources dynamically at runtime

    To switch the active data source for the current thread, use DynamicDataSourceHolder.setDynamicDataSourceKey(String key). This allows the application to route queries to different databases (e.g., switching from a primary database to a specific tenant or slave database) based on a lookup key.
    @Slf4j
    @RestController
    public class DynamicController {
        @Resource
        DataSourceUtil dataSourceUtil;
        @Resource
        UrlClDao urlClDao;
    
        @GetMapping("/test")
        public Map<String, Object> dynamicDataSourceTest(){
            // 1. Query primary database
            List<UrlCl> urlCls = urlClDao.selectList(new QueryWrapper<>());
    
            // 2. Switch to a predefined slave database
            DynamicDataSourceHolder.setDynamicDataSourceKey(DataSourceConstant.DB_2);
    
            // 3. Dynamically create and add a new data source
            DataSourceInfo dataSourceInfo = new DataSourceInfo();
            dataSourceInfo.setUrl("jdbc:mysql://sql.xxxx.xxxxx.com:3306/xxxx...");
            dataSourceInfo.setUserName("xxxxx");
            dataSourceInfo.setPassword("xxxxx");
            dataSourceInfo.setKeyCode("test");
    
            DruidDataSource druidDataSource = dataSourceUtil.createDataSourceConnection(dataSourceInfo);
            if (Objects.nonNull(druidDataSource)){
                // Register the new source
                dataSourceUtil.addDefineDynamicDataSource(druidDataSource, dataSourceInfo.getKeyCode());
                
                // Set current thread to use the newly added data source
                DynamicDataSourceHolder.setDynamicDataSourceKey(dataSourceInfo.getKeyCode());
                
                // Query using the new data source
                urlCls = urlClDao.selectList(new QueryWrapper<>());
                
                // Close connection when finished
                druidDataSource.close();
            }
            return new HashMap<>();
        }
    }
  8. Git branching strategy and naming conventions

    The project uses a structured branching model. Branch names must be singular; plural forms (e.g., 'features', 'hotfixes') are prohibited.

    • master (or main): The primary branch used for major version releases.
    • develop: The main integration branch where daily development occurs.
    • feature-*: Temporary branches created from develop for specific feature development. Must be merged back into develop and deleted after verification.
    • release-*: Pre-release branches created from develop for final testing before merging into both master and develop.
    • hotfix-*: Branches created from master or develop to fix critical bugs in production. Must be merged back into both master and develop upon completion.
  9. Synchronize Linux hardware and software clocks

    Linux servers maintain two separate time sources: the hardware clock (CMOS), accessed via hwclock, and the software clock (OS), accessed via date. If system logs show incorrect times while the date command appears normal, the two clocks may be out of sync. Use the following commands to synchronize them:
    # View the current hardware clock time
    sudo hwclock
    
    # Set the software clock (OS time) to match the hardware clock
    sudo hwclock -s
    
    # Set the hardware clock to match the software clock (OS time)
    sudo hwclock -w
  10. Implement dynamic datasource switching using AbstractRoutingDataSource

    To implement dynamic datasource switching in Spring Boot, extend AbstractRoutingDataSource and override the determineCurrentLookupKey() method. This method should return a key (typically from a ThreadLocal holder) that determines which database connection the current thread should use.
    @Data
    @AllArgsConstructor
    @NoArgsConstructor
    public class DynamicDataSource extends AbstractRoutingDataSource {
        // Backup all datasource information
        private Map<Object, Object> defineTargetDataSources;
    
        /**
         * Determine which datasource the current thread uses
         */
        @Override
        protected Object determineCurrentLookupKey() {
            return DynamicDataSourceHolder.getDynamicDataSourceKey();
        }
    }
  11. Automate MySQL backup uploads to Qiniu Cloud

    Create a wrapper script to execute the qshell qupload command. It is critical to delete the qshell cache folder before uploading to ensure new files are detected and uploaded correctly.
    #!/bin/bash
    dd=`date +%Y%m%d%H%M%S`
    
    echo "----------Deleting cache folder----------"
    # Replace 'username' with your actual qshell user
    rm -rf /root/.qshell/users/username/qupload
    
    echo "----------Starting upload to Qiniu Cloud ${dd}----------"
    /data/qiniu/qshell qupload /data/qiniu/config/upload.conf
    
    echo "----------Upload successful!----------"
  12. Bug fix (hotfix) workflow

    To fix a bug in a released version:

    1. Create a hotfix branch based on develop or the most recent tag (depending on the situation).
    2. Ensure the fix passes testing in test and production environments.
    3. Merge the fix into the develop branch and generate a version tag.
    4. If other temporary feature branches exist, they must merge the updated develop code (containing the fix) into their own branches.
    5. Delete the hotfix branch after verification.
    6. Document the tag and feature details in README.md.