BDH (Dragon Hatchling)

BDH is a biologically inspired large language model architecture that uses a scale-free, locally interacting network of neurons. Unlike Transformers that process information token-by-token with limited internal state, BDH utilizes a larger latent reasoning space with intrinsic memory.

Key technical properties:

• Scale-free network topology: Mimics biological connectivity.
• Locally interacting neuron particles: Employs excitatory/inhibitory dynamics.
• Hebbian working memory: Based on synaptic plasticity, displaying monosemanticity.
• GPU-friendly state-space formulation: Optimized for efficient implementation.
• Interpretable activations: Sparse and positive activations.

To set up the environment for BDH (Dragon Hatchling), install the required Python dependencies using pip and the provided requirements.txt file.

pip install -r requirements.txt

After installing dependencies, you can run the training script to train the BDH model on a toy dataset using the train.py entry point.

python train.py