BERTopic Documentation

BERTopic follows a modular six-step pipeline to transform documents into topics:

1. Embed documents: Converts documents into numerical representations using embedding models (default: sentence-transformers).
2. Dimensionality reduction: Reduces the dimensionality of embeddings to handle the curse of dimensionality (default: UMAP).
3. Cluster Documents: Groups reduced embeddings into clusters using density-based techniques (default: HDBSCAN).
4. Bag-of-words: Combines all documents in a cluster into a single document and counts word frequencies to create a cluster-level representation.
5. Topic representation: Uses class-based TF-IDF (c-TF-IDF) to identify words that are most representative of a specific cluster compared to others.
6. Fine-tune Topic representation (Optional): Refines the c-TF-IDF keywords or generates summaries using external models (e.g., GPT, T5, KeyBERT) via the bertopic.representation module.

To use quantized Mistral models (like Zephyr) via ctransformers, first install the necessary packages:

pip install ctransformers[cuda]
pip install --upgrade git+https://github.com/huggingface/transformers

When loading the model with AutoModelForCausalLM.from_pretrained, use the gpu_layers parameter to offload layers to the GPU (set to 0 if no GPU is available). You must define a prompt template that includes [DOCUMENTS] and [KEYWORDS] tags to guide the model.

from ctransformers import AutoModelForCausalLM
from transformers import AutoTokenizer, pipeline
from bertopic.representation import TextGeneration
from bertopic import BERTopic

# Load quantized model
model = AutoModelForCausalLM.from_pretrained(
    "TheBloke/zephyr-7B-alpha-GGUF",
    model_file="zephyr-7b-alpha.Q4_K_M.gguf",
    model_type="mistral",
    gpu_layers=50,
    hf=True
)
tokenizer = AutoTokenizer.from_pretrained("HuggingFaceH4/zephyr-7b-alpha")

# Create pipeline
generator = pipeline(
    model=model, tokenizer=tokenizer,
    task='text-generation',
    max_new_tokens=50,
    repetition_penalty=1.1
)

# Define prompt with tags
prompt = """<|system|>You are a helpful, respectful and honest assistant for labeling topics..</s>
<|user|>
I have a topic that contains the following documents:
[DOCUMENTS]

The topic is described by the following keywords: '[KEYWORDS]'.

Based on the information about the topic above, please create a short label of this topic. Make sure you to only return the label and nothing more.</s>
<|assistant|>"""

# Use in BERTopic
zephyr = TextGeneration(generator, prompt=prompt)
representation_model = {"Zephyr": zephyr}
topic_model = BERTopic(representation_model=representation_model, verbose=True)

To see how topic representations change when merging topics, you can visualize the hierarchy with specific hierarchical labels. This requires first calculating the hierarchical topics using topic_model.hierarchical_topics(docs) and then passing that result to the visualization function.

When using the interactive visualization, hovering over the black circles reveals the topic representation at that specific level of the hierarchy, helping you identify logical merges or sub-topics within larger themes.

from bertopic import BERTopic
from sklearn.datasets import fetch_20newsgroups

# 1. Prepare data
docs = fetch_20newsgroups(subset='all',  remove=('headers', 'footers', 'quotes'))["data"]

# 2. Train BERTopic
topic_model = BERTopic(verbose=True)
topics, probs = topic_model.fit_transform(docs)

# 3. Calculate hierarchical topics
hierarchical_topics = topic_model.hierarchical_topics(docs)

# 4. Visualize with hierarchical labels
topic_model.visualize_hierarchy(hierarchical_topics=hierarchical_topics)

Multi-aspect topic modeling allows you to generate multiple different representations (e.g., keywords, summaries, or custom labels) for a single topic during the .fit or .fit_transform stages.

To implement this, pass a dictionary to the representation_model parameter in the BERTopic constructor.

• The main representation (used by most visualization options) must be defined using the key "Main".
• Additional aspects can be defined using any other keys. These keys can map to a single representation model or a list of models to be used as a pipeline.

from bertopic.representation import KeyBERTInspired
from bertopic.representation import PartOfSpeech
from bertopic.representation import MaximalMarginalRelevance
from bertopic import BERTopic
from sklearn.datasets import fetch_20newsgroups

# Documents to train on
docs = fetch_20newsgroups(subset='all',  remove=('headers', 'footers', 'quotes'))['data']

# The main representation of a topic
main_representation = KeyBERTInspired()

# Additional ways of representing a topic
aspect_model1 = PartOfSpeech("en_core_web_sm")
aspect_model2 = [KeyBERTInspired(top_n_words=30), MaximalMarginalRelevance(diversity=.5)]

# Add all models together to be run in a single `fit` via a dictionary
representation_model = {
   "Main": main_representation,
   "Aspect1":  aspect_model1,
   "Aspect2":  aspect_model2
}

topic_model = BERTopic(representation_model=representation_model).fit(docs)

You can perform supervised topic modeling by replacing the clustering step with a classification algorithm and skipping dimensionality reduction. This allows BERTopic to learn the relationship between existing labels and documents, while still generating c-TF-IDF topic representations for those labels. This approach also enables you to use .transform() to predict topics for new, unseen documents.

To implement this, pass an instance of BaseDimensionalityReduction to umap_model to skip dimensionality reduction, and pass a classifier (e.g., LogisticRegression) to hdbscan_model to replace the clustering step.

from bertopic import BERTopic
from bertopic.vectorizers import ClassTfidfTransformer
from bertopic.dimensionality import BaseDimensionalityReduction
from sklearn.linear_model import LogisticRegression

# 1. Prepare your documents and labels
docs = ["your document 1", "your document 2"]
y = [0, 1]

# 2. Configure models to skip dimensionality reduction and use a classifier
empty_dimensionality_model = BaseDimensionalityReduction()
clf = LogisticRegression()
ctfidf_model = ClassTfidfTransformer(reduce_frequent_words=True)

# 3. Create the supervised BERTopic instance
topic_model = BERTopic(
    umap_model=empty_dimensionality_model,
    hdbscan_model=clf,
    ctfidf_model=ctfidf_model
)

# 4. Fit the model using the labels (y)
topics, probs = topic_model.fit_transform(docs, y=y)