Selective Context

You can interact with Selective Context via a Streamlit web application. Run the following command from the repository root:

streamlit run app/app.py

To reproduce the experimental results, download the dataset dumps and run the main.py script with the required arguments: dataset path, number of articles, and the Hugging Face model name/path.

# 1. Download and unzip datasets
wget https://github.com/liyucheng09/Selective_Context/releases/download/v0.1.0rc1/datasets_dumps.zip
unzip datasets_dumps.zip

# 2. Run experiments
python main.py datasets_dumps/arxiv datasets_dumps/news datasets_dump/conversation <output_path_to_save_results> <num_articles> <HF_model_name_or_path>

Install the package via PyPI and download the required spaCy language models. For English, use en_core_web_sm. If you are processing Chinese text, you must also download zh_core_web_sm.

pip install selective-context
python -m spacy download en_core_web_sm
# For Chinese support:
python -m spacy download zh_core_web_sm

When calling SelectiveContext, you can choose how the context is reduced via the reduce_level parameter. This determines the unit of text that gets replaced by a mask token if its self-information is below the threshold.

• 'sent': Masks entire sentences. If keep_leading_word is enabled (internal setting), it may preserve a few words at the start of the sentence.
• 'phrase': Masks noun phrases (using spacy's noun chunking).
• 'token': Masks individual tokens.

The reduce_level parameter in the SelectiveContext.__call__ method determines the granularity at which information is filtered. The algorithm calculates self-information and masks units that fall below a certain percentile threshold.

• 'sent': Filters entire sentences. Masked sentences are replaced by a <deleted> token.
• 'phrase': Filters at the noun-phrase level (using spacy for English or direct processing for Chinese). Masked phrases are replaced by an empty string or specific mask token.
• 'token': Filters at the individual token level. Masked tokens are replaced by an empty string.

To understand how Selective Context performs on specific subsets of your experiments, use the following patterns:

Task-wise Analysis: Groups results by the task type (e.g., summarisation, qa, reconstruction, continue_conversation). This is useful for seeing if context reduction affects certain types of reasoning more than others.

Data-wise Analysis: Groups results by the dataset source (e.g., news mapped to BBC, arxiv, or conversation mapped to ShareGPT). This helps identify if the method is sensitive to the domain of the text.

Both patterns return a Pandas DataFrame indexed by the grouping dimension and the Ratio (context reduction ratio).

def task_wise(context_type):
    # Returns DataFrame indexed by ['Task', 'Ratio']
    ...

def data_wise(context_type):
    # Returns DataFrame indexed by ['Data', 'Ratio']
    ...

After initializing a parser instance, you can iterate through manager.articles to extract conversation data. This data can be converted into a pandas DataFrame and subsequently into a HuggingFace Dataset for sharing or further processing.

Steps:

1. Extract entry_id and context from each article.
2. Create a list of dictionaries.
3. Convert to a pandas DataFrame.
4. Convert to a HuggingFace Dataset.

# 1. Extract data
covs = [{'id': article.entry_id, 'chat': article.context} for article in manager.articles]

# 2. Convert to DataFrame
import pandas as pd
df = pd.DataFrame.from_dict(covs, orient='columns')

# 3. Convert to HuggingFace Dataset
from datasets import Dataset
ds = Dataset.from_pandas(df)

# 4. (Optional) Push to HuggingFace Hub
ds.push_to_hub('liyucheng/sharegpt-500')

The SelectiveContext class allows you to compress prompts and context to increase the amount of content an LLM can process. You initialize the class with a model_type (e.g., 'gpt2') and a lang (e.g., 'en'), then call the instance on your text. It returns the compressed context and the reduced content.

from selective_context import SelectiveContext

# Initialize with a base model and language
sc = SelectiveContext(model_type='gpt2', lang='en')

# Compress text (returns compressed context and the reduced content)
context, reduced_content = sc(text)

# Compress text with a specific reduction ratio
context, reduced_content = sc(text, reduce_ratio=0.5)

Pre-processed context managers or experiment results can be loaded using the pickle module. This is useful for resuming work or inspecting metrics from previous runs.

import pickle

# Loading a NewsContextManager
with open('/path/to/NewsContextManager_sent.pkl', 'rb') as f:
    news = pickle.load(f)

# Accessing article sections
section_text = news.articles[0].sections[0]

# Loading experiment metrics
with open('/path/to/answer_summarisation_arxiv_0.35.pkl', 'rb') as f:
    d = pickle.load(f)
print(d.metrics)