prefixspan-py Documentation

repository·master·Indexed 19 days ago

https://github.com/chuanconggao/prefixspan-py

A unified framework for sequential pattern mining implementing PrefixSpan, BIDE (closed patterns), and FEAT (generator patterns) algorithms. It provides a Python API via the PrefixSpan class and a command-line tool, prefixspan-cli, for mining frequent or top-k patterns from integer or text sequences.

Tokens
2.1K
Snippets
6
Records
6
Agent score
15%

What's inside prefixspan-py

  1. Performance Tip: Use PyPy

    master

    For significantly better performance (up to 10x faster on average), it is strongly recommended to run prefixspan using PyPy instead of CPython.

    Note on PyPy versions:

    • For the latest PyPy3 (compatible with Python 3.6+), use the latest version of prefixspan.
    • For older PyPy3 (compatible with Python 3.5.3), you may need to install version 0.4: pip3 install prefixspan==0.4.
    # Example: Installing in a PyPy virtual environment
    # (Assuming PyPy is already installed on your system)
    pypy3 -m venv venv
    source venv/bin/activate
    pip install prefixspan
  2. Apply custom key, filter, and callback functions in PrefixSpan API

    master

    The PrefixSpan API supports advanced customization via lambda functions. In all these functions, patt is the current pattern and matches is a list of (id, position) tuples representing where the pattern was found.

    • key: Defines how to score a pattern. Default is len(matches). If you provide a custom key, it is recommended to provide a bound function for pruning efficiency.
    • filter: A function that returns True to keep a pattern or False to discard it.
    • callback: A function used to process patterns immediately. When a callback is provided, the method returns None instead of a list. This is highly recommended for large datasets to save memory.
    # Custom Key: Score patterns by the total number of matched items
    print(ps.topk(5, key=lambda patt, matches: sum(len(db[i]) for i, _ in matches)))
    
    # Custom Filter: Exclude patterns that appear in the first sequence (index 0)
    print(ps.topk(5, filter=lambda patt, matches: matches[0][0] > 0))
    
    # Custom Callback: Process patterns immediately to save memory
    coverage = [[] for i in range(len(db))]
    
    def cover(patt, matches):
        for i, _ in matches:
            coverage[i] = max(coverage[i], patt, key=len)
    
    ps.frequent(2, callback=cover)
    print(coverage)
  3. Use the PrefixSpan Python API

    master

    Use the PrefixSpan class to perform sequential pattern mining within your Python code. You can initialize the object with a database (a list of lists) and then call .frequent(threshold) or .topk(k).

    Pattern Types:

    • Frequent: Returns patterns that appear at least threshold times.
    • Top-k: Returns the k most frequent patterns.
    • Closed: Use closed=True to return only closed patterns (patterns where no super-pattern has the same frequency).
    • Generator: Use generator=True to return only generator patterns (patterns where no sub-pattern has the same frequency).
    from prefixspan import PrefixSpan
    
    db = [
        [0, 1, 2, 3, 4],
        [1, 1, 1, 3, 4],
        [2, 1, 2, 2, 0],
        [1, 1, 1, 2, 2],
    ]
    
    ps = PrefixSpan(db)
    ps.minlen = 5  # Set minimum pattern length
    ps.maxlen = 15 # Set maximum pattern length
    
    # Get frequent patterns with threshold 2
    print(ps.frequent(2))
    
    # Get top 5 frequent patterns
    print(ps.topk(5))
    
    # Get closed patterns
    print(ps.frequent(2, closed=True))
    
    # Get generator patterns
    print(ps.frequent(2, generator=True))
  4. Reference: prefixspan-cli options

    master

    The following options are available for the prefixspan-cli command:

    OptionDescription
    --textTreat each item as text instead of integer.
    --closedReturn only closed patterns.
    --generatorReturn only generator patterns.
    --key=<key>Custom key function. Must be a Python function in form of "lambda patt, matches: ...", returning an integer value.
    --bound=<bound>The upper-bound function of the respective key function. Must be $\geq$ the key function and anti-monotone.
    --filter=<filter>Custom filter function. Must be a Python function in form of "lambda patt, matches: ...", returning a boolean value.
    --minlen=<minlen>Minimum length of patterns. (Default: 1)
    --maxlen=<maxlen>Maximum length of patterns. (Default: 1000)
    Usage:
        prefixspan-cli (frequent | top-k) <threshold> [options] [<file>]
    
    Options:
        --text             Treat each item as text instead of integer.
        --closed           Return only closed patterns.
        --generator        Return only generator patterns.
        --key=<key>        Custom key function. [default: ]
                           Must be a Python function in form of "lambda patt, matches: ...", returning an integer value.
        --bound=<bound>    The upper-bound function of the respective key function. When unspecified, the same key function is used. [default: ]
                           Must be no less than the key function, i.e. bound(patt, matches) $\geq$ key(patt, matches).
                           Must be anti-monotone, i.e. for patt1 $\sqsubseteq$ patt2, bound(patt1, matches1) $\geq$ bound(patt2, matches2).
        --filter=<filter>  Custom filter function. [default: ]
                           Must be a Python function in form of "lambda patt, matches: ...", returning a boolean value.
        --minlen=<minlen>  Minimum length of patterns. [default: 1]
        --maxlen=<maxlen>  Maximum length of patterns. [default: 1000]
  5. Use the prefixspan CLI

    master

    The prefixspan-cli tool allows you to run sequential pattern mining algorithms directly from the terminal. You can mine frequent patterns or top-k patterns from a file or standard input.

    Usage: prefixspan-cli (frequent | top-k) <threshold> [options] [<file>]

    Input Formats:

    • Integer sequences: Each sequence is a line of integers separated by spaces.
    • Text sequences: Use the --text flag. Each sequence is a line of words separated by spaces (ensure stop words are removed beforehand).
    # Example: Frequent patterns with threshold 2 from a file
    prefixspan-cli frequent 2 test.dat
    
    # Example: Top-5 frequent patterns from text data
    prefixspan-cli top-k 5 --text test.txt