DOCSEARCHbyAlgolia
Submit Docs

Tetsuzawa's ISUCON and Performance Tuning Documentation

website·Indexed 31 days ago

https://www.tetsuzawa.com/

A collection of technical guides and blog posts focused on ISUCON competition preparation and system performance tuning. Includes practical tutorials for MySQL 8.0, Redis, Nginx, OpenResty, and Go (pprof, singleflight, lo), as well as methodologies for bottleneck analysis such as the USE and TSA methods.

Tokens
15.3K
Snippets
84
Records
103
Agent score
53%

What's inside Tetsuzawa Docs

  1. Overview of Amazon CloudWatch Logs pattern query

    The pattern query in Amazon CloudWatch Logs Insights uses machine learning to automatically cluster log data into patterns. This allows developers to analyze logs by grouping similar log messages together, making it easier to identify common patterns, anomalies, or recurring errors without manually writing complex filters for every variation of a log message.

  2. Overview of CloudWatch Logs Insights pattern query

    The pattern query in CloudWatch Logs Insights is a feature that leverages machine learning to automatically cluster log data into patterns. This allows developers to identify common log structures without manually writing complex filters. While the query command was released in July 2023, an updated UI was introduced in late November 2023 that allows users to analyze these patterns directly without manually executing queries.

  3. Understand Amazon CloudWatch Logs Insights pattern query

    The pattern query in Amazon CloudWatch Logs Insights uses machine learning to automatically cluster log data into patterns. This allows users to identify common log structures, view clustered log samples, and aggregate tokens without needing to manually define complex regular expressions or filters for every variation of a log message.

  4. Understand Amazon CloudWatch Logs pattern query

    The pattern query in CloudWatch Logs Insights uses machine learning to automatically cluster log data into patterns. This allows developers to identify common log structures and group similar log entries together without manually defining regular expressions or filters. In addition to the query command, AWS provides a rich UI that allows users to visualize and analyze these patterns without writing manual queries.

  5. Understand Amazon CloudWatch Logs pattern query

    The pattern query in Amazon CloudWatch Logs Insights uses machine learning to automatically cluster log data into patterns. This allows developers to identify common log structures, group similar log entries, and analyze anomalies without manually writing complex regular expressions or filters for every single log variation.

  6. Register a deploy key with write access to GitHub

    Add the generated public SSH key to the GitHub repository as a deploy key. Ensure that 'allow write access' is enabled.
    gh repo --repo tetsuzawa/isucon-xx-xxxxxx deploy-key add --allow-write ~/.ssh/isucon-xx-xxxxxx.pub

  7. Clear Redis data during application initialization

    As an alternative to modifying the Redis configuration file, you can programmatically clear existing data within the application's initialization logic to ensure a clean state after a restart.

  8. Move Varnish configuration to Git

    Move the Varnish default.vcl file to the user's home directory and create a symlink to allow Git tracking and synchronization.
    mkdir -p /home/isucon/etc
mkdir -p /home/isucon/backup/etc/varnish
sudo cp -rv /etc/varnish/default.vcl /home/isucon/backup/etc/varnish/defulat.vcl
sudo mv -v /etc/varnish/default.vcl /home/isucon/etc/varnish/default.vcl
sudo ln -s /home/isucon/etc/varnish/default.vcl /etc/varnish/default.vcl
sudo chmod 777 -R /home/isucon/etc/varnish

  9. Install OpenResty on Ubuntu 22.04 amd64

    Install OpenResty by disabling the default Nginx service and adding the official OpenResty repository.
    sudo systemctl disable nginx
sudo systemctl stop nginx
sudo apt-get -y install --no-install-recommends wget gnupg ca-certificates
wget -O - https://openresty.org/package/pubkey.gpg | sudo gpg --dearmor -o /usr/share/keyrings/openresty.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/openresty.gpg] http://openresty.org/package/ubuntu $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/openresty.list > /dev/null
sudo apt-get update
sudo apt-get -y install openresty

  10. Move Nginx configuration to Git for multi-server sync

    To manage Nginx configurations across multiple servers (a, b, c), move the configuration from /etc/nginx to a directory within the user's home folder, symlink it back, and commit it to Git.
    # On server 'a' (Initial setup)
mkdir -p ./etc
mkdir -p ./backup/etc
cp -r /etc/nginx ./backup/etc/nginx
sudo mv /etc/nginx ./etc/nginx
sudo ln -s /home/isucon/etc/nginx /etc/nginx
sudo chmod 775 -R /home/isucon/etc/nginx
sudo chmod 777 -R /var/log/nginx
sudo nginx -t

git add etc/nginx
git commit -m "add nginx conf"
git push

# On servers 'b' and 'c' (Deployment)
git pull origin main
sudo rm -rf /etc/nginx
sudo ln -s /home/isucon/etc/nginx /etc/nginx
sudo chmod 775 -R /etc/nginx
sudo chown -R root:root /etc/nginx
mkdir -p /home/isucon/log/nginx
sudo nginx -t
sudo systemctl restart nginx
sudo systemctl status nginx

  11. Performance tuning workflow phases

    Effective performance tuning to resolve system bottlenecks should follow three distinct phases to ensure improvements are based on data rather than assumptions:

    1. Current State Assessment (現状把握): Eliminate preconceptions and define exactly what needs to be improved and why.
    2. Analysis (解析): Identify the actual bottleneck using structured methodologies.
    3. Improvement (改善): Implement changes based on the analysis results.

  12. Version control MySQL 8 configuration files using Git

    To track MySQL 8 configuration files in Git without modifying the system /etc/mysql directory directly, create a local directory structure (e.g., in the user's home directory) and symlink the system configuration files into it. This allows you to git add the configuration files from a user-accessible path.
    # Create local directories for config and backups
mkdir -p ./etc
mkdir -p ./backup/etc
sudo cp -r /etc/mysql ./backup/etc/mysql

# Create the target directory structure in the user home
sudo mkdir -p /home/isucon/etc/mysql/conf.d
sudo mkdir -p /home/isucon/etc/mysql/mysql.conf.d

# Symlink system config files to the user directory
sudo ln /etc/mysql/conf.d/mysql.cnf /home/isucon/etc/mysql/conf.d/mysql.cnf
sudo ln /etc/mysql/conf.d/mysqldump.cnf /home/isucon/etc/mysql/conf.d/mysqldump.cnf
sudo ln /etc/mysql/mysql.conf.d/mysql.cnf /home/isucon/etc/mysql/mysql.conf.d/mysql.cnf
sudo ln /etc/mysql/mysql.conf.d/mysqld.cnf /home/isucon/etc/mysql/mysql.conf.d/mysqld.cnf

# Set permissions
sudo chmod 755 -R /home/isucon/etc/mysql
mkdir -p /var/log/mysql
sudo chmod 777 -R /var/log/mysql

# Commit to Git
git add etc/mysql
git commit -m "add mysql conf"
git push