FlashText Documentation

You can install the flashtext module using pip.

$ pip install flashtext

Use KeywordProcessor.extract_keywords() to find keywords in a string. You can map 'unclean' names (the terms found in text) to 'standardised' names (the returned values).

from flashtext import KeywordProcessor
keyword_processor = KeywordProcessor()
# keyword_processor.add_keyword(<unclean name>, <standardised name>)
keyword_processor.add_keyword('Big Apple', 'New York')
keyword_processor.add_keyword('Bay Area')
keywords_found = keyword_processor.extract_keywords('I love Big Apple and Bay Area.')
# ['New York', 'Bay Area']

By default, keyword extraction is case-insensitive. To enforce case sensitivity, initialize KeywordProcessor with case_sensitive=True.

from flashtext import KeywordProcessor
keyword_processor = KeywordProcessor(case_sensitive=True)
keyword_processor.add_keyword('Big Apple', 'New York')
keyword_processor.add_keyword('Bay Area')
keywords_found = keyword_processor.extract_keywords('I love big Apple and Bay Area.')
# keywords_found: ['Bay Area']

The KeywordProcessor supports several inspection methods:

• len(keyword_processor): Returns the number of terms.
• 'term' in keyword_processor: Checks if a term is present.
• keyword_processor.get_keyword(unclean_name): Returns the standardised name.
• keyword_processor['unclean'] = 'standard': Set a keyword via dictionary-like syntax.
• keyword_processor.get_all_keywords(): Returns a dictionary of all keywords.

Instead of a string, you can pass a tuple as the second argument to add_keyword(). This allows you to associate metadata with a keyword. Note that replace_keywords will not work when using this feature.

from flashtext import KeywordProcessor
kp = KeywordProcessor()
kp.add_keyword('Taj Mahal', ('Monument', 'Taj Mahal'))
kp.add_keyword('Delhi', ('Location', 'Delhi'))
keywords = kp.extract_keywords('Taj Mahal is in Delhi.')
# keywords: [('Monument', 'Taj Mahal'), ('Location', 'Delhi')]

Use KeywordProcessor.replace_keywords() to substitute identified keywords with their standardised names in a string.

from flashtext import KeywordProcessor
keyword_processor = KeywordProcessor()
keyword_processor.add_keyword('Big Apple', 'New York')
keyword_processor.add_keyword('New Delhi', 'NCR region')
new_sentence = keyword_processor.replace_keywords('I love Big Apple and new delhi.')
# 'I love New York and NCR region.'

To get the character indices where keywords were found, call extract_keywords() with the argument span_info=True. This returns a list of tuples containing (standardised_name, start_index, end_index).

from flashtext import KeywordProcessor
keyword_processor = KeywordProcessor()
keyword_processor.add_keyword('Big Apple', 'New York')
keyword_processor.add_keyword('Bay Area')
keywords_found = keyword_processor.extract_keywords('I love Big Apple and Bay Area.', span_info=True)
# keywords_found: [('New York', 7, 16), ('Bay Area', 21, 29)]

Use KeywordProcessor.extract_keywords() to find predefined keywords in a string. You can map 'unclean' names (the terms found in text) to 'standardised' names (the values returned by the extractor).

from flashtext import KeywordProcessor
keyword_processor = KeywordProcessor()
# keyword_processor.add_keyword(<unclean name>, <standardised name>)
keyword_processor.add_keyword('Big Apple', 'New York')
keyword_processor.add_keyword('Bay Area')
keywords_found = keyword_processor.extract_keywords('I love Big Apple and Bay Area.')
# keywords_found: ['New York', 'Bay Area']

Use replace_keywords to return a new string where all matched keywords are replaced by their corresponding replacement values.

keyword_processor.add_keyword('New Delhi', 'NCR region')
new_sentence = keyword_processor.replace_keywords('I love Big Apple and new delhi.')
# Output: 'I love New York and NCR region.'

The KeywordProcessor supports several dictionary-like operations:

• len(keyword_processor): Returns the number of terms.
• 'term' in keyword_processor: Checks if an unclean term is present.
• keyword_processor.get_keyword(unclean_name): Returns the standardised name.
• keyword_processor['unclean'] = 'standard': Sets a keyword.
• keyword_processor.get_all_keywords(): Returns the full mapping dictionary.

You can remove keywords using remove_keyword(unclean_name), remove_keywords_from_list(list_of_unclean_names), or remove_keywords_from_dict(dict_of_standardised_to_unclean_names).

from flashtext import KeywordProcessor
keyword_processor = KeywordProcessor()
keyword_dict = {
    "java": ["java_2e", "java programing"],
    "product management": ["PM", "product manager"],
}
keyword_processor.add_keywords_from_dict(keyword_dict)

keyword_processor.remove_keyword('java_2e')
keyword_processor.remove_keywords_from_dict({"product management": ["PM"]})
keyword_processor.remove_keywords_from_list(["java programing"])