KataGo Documentation

ghc::filesystem is a header-only, single-file library that provides a std::filesystem compatible interface. It is designed for projects using C++11, C++14, C++17, or C++20, allowing developers to use filesystem functionality even when a full C++17/20 standard library implementation is unavailable.

Key Characteristics:

• Namespace: All functionality resides in the ghc::filesystem namespace to avoid conflicts with std::filesystem in mixed-environment projects.
• UTF-8 Philosophy: The library follows the "UTF-8 Everywhere" philosophy. All std::string instances are interpreted as UTF-8 (similar to std::u8string), and std::u16string is treated as UTF-16.
• Compatibility: It is an almost drop-in replacement for std::filesystem, with the primary difference being the UTF-8 preference.

To train your own neural networks using KataGo, you must implement a closed self-play loop consisting of five components. This requires Python 3, PyTorch, and significant GPU power.

The 5 core components:

1. Selfplay engine (cpp/katago selfplay): Continuously plays games using the latest accepted models and writes data to a directory.
2. Shuffler (python/shuffle.py): Scans self-play data and shuffles it into .npz files.
3. Training (python/train.py): Continuously trains a neural net using the .npz files and saves models periodically.
4. Exporter (python/export_model.py): Converts PyTorch .ckpt models into the C++ compatible format.
5. Gatekeeper (cpp/katago gatekeeper) [OPTIONAL]: Tests new models against the current accepted models; if the new model passes, it is moved to the accepted models directory.

Monte-Carlo Tree Search (MCTS) can be viewed as an online policy learning algorithm. When using the PUCT formula, the cumulative visit distribution approximates the solution to an optimization problem that maximizes expected utility while staying close to a prior policy $P$ (from a neural network) using KL-divergence.

Key components of this optimization:

• Expected Utility: $\sum_{a} \pi(a) Q(a)$, where $\pi$ is the policy and $Q$ is the utility estimate.
• KL-Divergence: $D_{\text{KL}}(P || \pi)$, which regularizes the policy to stay near the prior $P$.
• Coefficient $\lambda_N$: Determines the strength of regularization, decaying as the number of visits $N$ increases, allowing the policy to deviate more from the prior as evidence accumulates.

KataGo uses a modified version of AlphaZero-style Dirichlet noise to improve exploration of 'blind-spot' moves. While standard AlphaZero replaces 25% of the root policy prior mass with uniform Dirichlet noise (alpha=0.03 per move), KataGo 'shapes' this noise.

How it works:

1. Half of the total alpha is distributed uniformly across all legal moves.
2. The other half of the alpha is concentrated on a subset of moves that have a policy prior (in logits) significantly higher than the majority of other legal moves.

This technique increases the likelihood that moves with low absolute policy priors—but which are still relatively more plausible than random moves—are selected for exploration during training.

Standard Monte-Carlo Tree Search (MCTS) treats games as branching trees, which is inefficient when different sequences of moves lead to the same state (transpositions). In games like Chess, these transpositions grow exponentially with depth.

Monte-Carlo Graph Search (MCGS) models the state space as a Directed Acyclic Graph (DAG) by sharing nodes. This allows the search to share computation across different paths that reach the same state. However, applying MCTS naively to a graph is unsound because MCTS relies on running statistics (N and Q) that are traditionally updated via a single path in a tree. To implement MCGS correctly, one must move beyond the standard 'running statistics' formulation toward an 'online policy learning' perspective to ensure information propagates correctly across the graph.

KataGo uses auxiliary value and score targets to improve neural network training. Instead of only predicting the final game outcome, the network predicts exponentially averaged future MCTS values for several time horizons (e.g., roughly the next 6, 16, and 50 turns on a 19x19 board).

This provides lower-variance feedback during training and enables advanced features like Uncertainty-Weighted MCTS Playouts and Optimistic Policy.

To improve policy learning speed, KataGo adds an auxiliary 'soft' policy head. This head attempts to predict a transformed version of the policy target: the target raised to the power of $(1/T)$ (where $T$ is a temperature, currently $T=4$) and re-normalized to sum to 1.

This forces the network to better recognize and discriminate between lower-probability moves that still contain meaningful MCTS information, rather than just focusing on the top 1-2 moves.

KataGo employs a two-part strategy to capture the benefits of Batch Normalization without the training/inference discrepancies:

1. Fixed Variance Initialization: Every layer where batch normalization would normally be inserted is initialized with a scalar multiplication $K$. $K$ is calculated such that the output of the normalization layer maintains a variance of 1, based on the idealized variance of the preceding layers.
2. One Batch Norm: The network includes exactly one batch norm layer at the end of the block trunk.
   • One set of heads (80% loss weight) passes through this batch norm layer during training to drive optimization.
   • A second set of heads (20% loss weight) skips the batch norm layer and is used for inference, ensuring no moving-average tracking is required at test-time.

KataGo weights MCTS playouts based on the neural network's predicted 'confidence'. Playouts where the network expects high error are downweighted, while highly certain playouts are upweighted.

To achieve this, the network is trained to predict the squared difference between its current short-term value/score predictions and the actual MCTS values/scores from the training data (using an exponential moving average with $\lambda \approx 5/6$ for 19x19 boards). This uncertainty estimate is then used to adjust the weight of each playout in the MCTS average and the PUCT exploration formula.