GuppyLM Documentation

You can load the GuppyLM Chat training dataset (60K single-turn conversations) using the Hugging Face datasets library. The dataset contains conversations between a human and Guppy, a small fish character that speaks in short, lowercase sentences about its aquatic environment.

from datasets import load_dataset
ds = load_dataset("arman-bd/guppylm-60k-generic")
print(ds["train"][0])
# {'input': 'hi guppy', 'output': 'hello. the water is nice today.', 'category': 'greeting'}

To run GuppyLM in an interactive chat mode on your local machine, install the required dependencies and use the guppylm module. Note that in interactive mode, the conversation grows until it hits the 128-token limit, which may reduce output quality.

pip install torch tokenizers
python -m guppylm chat

You can train the model from scratch using a Google Colab notebook.

1. Open the training notebook.
2. Set the runtime to T4 GPU.
3. Run all cells to download the dataset, train the tokenizer, train the model, and run tests.
4. Once complete, you can upload the resulting model to HuggingFace or download it locally.

To use GuppyLM, you need to install torch, tokenizers, and huggingface_hub. You must also download the model weights from Hugging Face (specifically the arman-bd/guppylm-9M repository) to your local directory.

# Setup + Download
!pip install -q torch tokenizers huggingface_hub
import os, shutil
if os.path.exists('/content/guppy'): shutil.rmtree('/content/guppy')
os.makedirs('/content/guppy'); os.chdir('/content/guppy')

from huggingface_hub import snapshot_download
snapshot_download(repo_id='arman-bd/guppylm-9M', local_dir='.')
print('Model downloaded.')

To set up the environment for training or running GuppyLM, install the required Python packages using pip.

!pip install -q torch tokenizers tqdm numpy datasets huggingface_hub

Training is performed using the train() function from train.py. The process involves downloading the fish conversation dataset, training a BPE tokenizer, and running the training loop with a cosine learning rate schedule. The training loop supports mixed-precision training (AMP) on CUDA devices.

from train import train
train()

You can invoke the chat interface with a single prompt and exit immediately after receiving the response using the --prompt flag.

python -m guppylm chat --prompt "tell me a joke"

To interact with the GuppyLM model, use the GuppyInference class. You must provide the path to the model checkpoint (.pt file) and the tokenizer configuration (tokenizer.json). The chat_completion method accepts a list of message dictionaries following the standard {'role': '...', 'content': '...'} format and returns a response object containing the model's generated text.

from inference import GuppyInference

engine = GuppyInference('checkpoints/best_model.pt', 'data/tokenizer.json')
r = engine.chat_completion([{'role': 'user', 'content': 'hi guppy'}])
print(r['choices'][0]['message']['content'])
# hi there. i just found a nice spot near the rock.

GuppyLM uses two main configuration classes defined in config.py: GuppyConfig for model architecture and TrainConfig for training hyperparameters.

GuppyConfig

• vocab_size: Vocabulary size (default: 4096)
• max_seq_len: Maximum sequence length (default: 128)
• d_model: Model dimension (default: 384)
• n_layers: Number of transformer layers (default: 6)
• n_heads: Number of attention heads (default: 6)
• ffn_hidden: Hidden dimension of the FFN (default: 768)
• dropout: Dropout probability (default: 0.1)
• pad_id: Padding token ID (default: 0)
• bos_id: Beginning of sequence token ID (default: 1, <|im_start|>)
• eos_id: End of sequence token ID (default: 2, <|im_end|>)