DOCSEARCHbyAlgolia
Submit Docs

TopMost: A Topic Modeling System Toolkit

repository·main·Indexed 43 days ago

https://github.com/bobxwu/topmost

A comprehensive toolkit for the complete lifecycle of topic modeling, including datasets, preprocessing, training, and evaluation. TopMost supports four main scenarios: basic topic modeling (e.g., LDA, NMF, BERTopic, FASTopic), hierarchical modeling (e.g., HDP, TraCo), dynamic modeling (e.g., DTM, DETM), and cross-lingual modeling (e.g., NMTM, InfoCTM). It provides high-level trainers like FASTopicTrainer and BasicTrainer, as well as evaluation metrics for topic coherence and diversity.

Tokens
11.3K
Snippets
40
Records
46
Agent score
54%

What's inside TopMost

  1. Overview of TopMost capabilities

    main

    TopMost is a comprehensive topic modeling system toolkit that manages the complete lifecycle of topic modeling tasks. It provides tools for:

    • Datasets: Access to various datasets.
    • Preprocessing: Preparing text data for modeling.
    • Models: Implementation of various topic modeling architectures.
    • Training: Workflows to train models on data.
    • Evaluations: Metrics to assess model performance.

    It supports multiple topic modeling scenarios, including:

    • Basic topic modeling
    • Dynamic topic modeling
    • Hierarchical topic modeling
    • Cross-lingual topic modeling

  2. Supported Topic Modeling Scenarios in TopMost

    main

    TopMost provides a comprehensive suite of topic modeling capabilities categorized into four main scenarios. Each scenario includes specific models, evaluation metrics, and compatible datasets:

    1. Basic Topic Modeling

    Used for standard topic extraction from a single corpus.

    • Models: LDA_, NMF_, ProdLDA_, DecTM_, ETM_, NSTM_, TSCTM_, BERTopic_, ECRTM_, FASTopic_
    • Metrics: Topic Coherence (TC), Topic Diversity (TD), Clustering, Classification
    • Datasets: 20NG, IMDB, NeurIPS, ACL, NYT, Wikitext-103

    2. Hierarchical Topic Modeling

    Used for discovering topics at multiple levels of granularity.

    • Models: HDP_, SawETM_, HyperMiner_, ProGBN_, TraCo_
    • Metrics: TC over levels, TD over levels, Clustering over levels, Classification over levels
    • Datasets: 20NG, IMDB, NeurIPS, ACL, NYT, Wikitext-103

    3. Dynamic Topic Modeling

    Used for modeling how topics evolve over time slices.

    • Models: DTM_, DETM_, CFDTM_
    • Metrics: TC over time slices, TD over time slices, Clustering, Classification
    • Datasets: NeurIPS, ACL, NYT

    4. Cross-lingual Topic Modeling

    Used for modeling topics across different languages.

    • Models: NMTM_, InfoCTM_
    • Metrics: TC (CNPMI), TD over languages, Classification (Intra and Cross-lingual)
    • Datasets: ECNews, Amazon, Review Rakuten

  3. Train a model using BasicTrainer and ProdLDA

    main

    For more granular control, you can manually compose the dataset, model, and trainer components.

    Workflow:

    1. Load a BasicDataset from a cached path.
    2. Initialize a model (e.g., ProdLDA) with the dataset's vocabulary size.
    3. Wrap the model and dataset in a BasicTrainer.
    4. Call .train() to obtain top words and training topic distributions (train_theta).
    import topmost

device = "cuda" # or "cpu"

# load a preprocessed dataset
dataset = topmost.BasicDataset("./datasets/20NG", device=device, read_labels=True)

# create a model
model = topmost.ProdLDA(dataset.vocab_size)
model = model.to(device)

# create a trainer
trainer = topmost.BasicTrainer(model, dataset)

# train the model
top_words, train_theta = trainer.train()

  4. Access TopMost tutorials

    main

    TopMost provides several Jupyter Notebook tutorials for different workflows. You can use these to learn how to preprocess datasets, train various types of topic models, and evaluate them.

    Available tutorials:

    • Quickstart: Basic introduction.
    • Preprocessing: How to preprocess datasets for use with TopMost.
    • Basic Topic Models: Training and evaluating standard topic models.
    • Hierarchical Topic Models: Training and evaluating hierarchical models.
    • Dynamic Topic Models: Training and evaluating models that capture topic evolution over time.
    • Cross-lingual Topic Models: Training and evaluating models across different languages.

  5. Test new documents with an existing trainer

    main

    To perform inference on new, unseen documents, use the Preprocess.parse method to convert text into the required format (using the existing dataset's vocabulary) and then call trainer.test().

    import torch
from topmost import Preprocess

new_docs = [
    "This is a new document about space, including words like space, satellite, launch, orbit.",
    "This is a new document about Microsoft Windows, including words like windows, files, dos."
]

preprocess = Preprocess()
# Use the existing dataset's vocab to parse new documents
new_parsed_docs, new_bow = preprocess.parse(new_docs, vocab=dataset.vocab)

# Convert bow to tensor and test
new_theta = trainer.test(torch.as_tensor(new_bow.toarray(), device=device).float())
    import torch
from topmost import Preprocess

new_docs = [
    "This is a new document about space, including words like space, satellite, launch, orbit.",
    "This is a new document about Microsoft Windows, including words like windows, files, dos."
]

preprocess = Preprocess()
new_parsed_docs, new_bow = preprocess.parse(new_docs, vocab=dataset.vocab)
new_theta = trainer.test(torch.as_tensor(new_bow.toarray(), device=device).float())

  7. Evaluate topic models

    main

    TopMost provides evaluation metrics via the topmost.eva module. You can calculate Topic Diversity (TD), Topic Coherence (TC), perform clustering, or run classification tasks.

    from topmost import eva

# topic diversity and coherence
TD = eva._diversity(top_words)
TC = eva._coherence(dataset.train_texts, dataset.vocab, top_words)

# get doc-topic distributions of testing samples
test_theta = trainer.test(dataset.test_data)

# clustering
clustering_results = eva._clustering(test_theta, dataset.test_labels)

# classification
cls_results = eva._cls(train_theta, test_theta, dataset.train_labels, dataset.test_labels)
    from topmost import eva

# topic diversity and coherence
TD = eva._diversity(top_words)
TC = eva._coherence(dataset.train_texts, dataset.vocab, top_words)

# get doc-topic distributions of testing samples
test_theta = trainer.test(dataset.test_data)
# clustering
clustering_results = eva._clustering(test_theta, dataset.test_labels)
# classification
cls_results = eva._cls(train_theta, test_theta, dataset.train_labels, dataset.test_labels)

  8. Quick Start with FASTopic

    main

    This example demonstrates how to use FASTopic to discover topic top words and document topic distributions using the 20newsgroups dataset.

    from topmost.data import RawDataset
from topmost.preprocess import Preprocess
from topmost.trainers import FASTopicTrainer
from sklearn.datasets import fetch_20newsgroups

# Load data
docs = fetch_20newsgroups(subset='all',  remove=('headers', 'footers', 'quotes'))['data']

# Setup preprocessing and dataset
preprocess = Preprocess(vocab_size=10000)
dataset = RawDataset(docs, preprocess, device="cuda")

# Initialize and train
trainer = FASTopicTrainer(dataset, verbose=True)
top_words, doc_topic_dist = trainer.train()

# Test on new documents
new_docs = [
    "This is a document about space, including words like space, satellite, launch, orbit.",
    "This is a document about Microsoft Windows, including words like windows, files, dos."
]
new_theta = trainer.test(new_docs)
print(new_theta.argmax(1))
    from topmost.data import RawDataset
from topmost.preprocess import Preprocess
from topmost.trainers import FASTopicTrainer
from sklearn.datasets import fetch_20newsgroups

docs = fetch_20newsgroups(subset='all',  remove=('headers', 'footers', 'quotes'))['data']
preprocess = Preprocess(vocab_size=10000)

dataset = RawDataset(docs, preprocess, device="cuda")

trainer = FASTopicTrainer(dataset, verbose=True)
top_words, doc_topic_dist = trainer.train()

new_docs = [
    "This is a document about space, including words like space, satellite, launch, orbit.",
    "This is a document about Microsoft Windows, including words like windows, files, dos."
]

new_theta = trainer.test(new_docs)
print(new_theta.argmax(1))

  9. Download a preprocessed dataset

    main

    You can download and cache preprocessed datasets directly using topmost.download_dataset. Supported dataset identifiers include '20NG', 'IMDB', 'NeurIPS', 'ACL', 'NYT', and 'Wikitext-103'.

    import topmost

topmost.download_dataset('20NG', cache_path='./datasets')

  11. Quick Start: Train a model with FASTopicTrainer

    main

    The FASTopicTrainer provides a high-level workflow to preprocess raw text, train a model, and perform inference on new documents. It uses FASTopic to discover top words and document-topic distributions.

    Key components:

    • RawDataset: Wraps raw text documents with a Preprocess object.
    • Preprocess: Handles text cleaning and vocabulary management (e.g., vocab_size).
    • FASTopicTrainer: Orchestrates the training process and provides a .test() method for inference.
    from topmost import RawDataset, Preprocess, FASTopicTrainer
from sklearn.datasets import fetch_20newsgroups

# Load raw data
docs = fetch_20newsgroups(subset='all',  remove=('headers', 'footers', 'quotes'))['data']

# Setup preprocessing and dataset
preprocess = Preprocess(vocab_size=10000)
dataset = RawDataset(docs, preprocess, device="cuda")

# Train the model
trainer = FASTopicTrainer(dataset, verbose=True)
top_words, doc_topic_dist = trainer.train()

# Inference on new documents
new_docs = [
    "This is a document about space, including words like space, satellite, launch, orbit.",
    "This is a document about Microsoft Windows, including words like windows, files, dos."
]
new_theta = trainer.test(new_docs)
print(new_theta.argmax(1))

  12. Train a model using BasicTrainer

    main

    To train a model from scratch, follow these steps: load a BasicDataset, initialize a model (e.g., ProdLDA), and use a BasicTrainer to execute the training.

    import topmost

device = "cuda" # or "cpu"

# load a preprocessed dataset
dataset = topmost.BasicDataset("./datasets/20NG", device=device, read_labels=True)

# create a model
model = topmost.ProdLDA(dataset.vocab_size)
model = model.to(device)

# create a trainer
trainer = topmost.BasicTrainer(model, dataset)

# train the model
top_words, train_theta = trainer.train()
    import topmost

device = "cuda" # or "cpu"

# load a preprocessed dataset
dataset = topmost.BasicDataset("./datasets/20NG", device=device, read_labels=True)
# create a model
model = topmost.ProdLDA(dataset.vocab_size)
model = model.to(device)

# create a trainer
trainer = topmost.BasicTrainer(model, dataset)

# train the model
top_words, train_theta = trainer.train()