fpgen Browser Fingerprint Generator

repository·main·Indexed 41 days ago

https://github.com/scrapfly/fingerprint-generator

A fast Python library (v1.3.0) that uses a Bayesian generative network to create realistic, constrained browser fingerprints, including HTTP headers, TLS, GPU, and audio data. It provides a CLI for model management and a Generator class for producing fingerprints based on specific OS, browser, and hardware criteria. Includes tools like trace() for probability distribution analysis and query() to retrieve possible values from the generative model.

Tokens
5K
Snippets
27
Records
28
Agent score
46%

What's inside fpgen

  1. Filter fingerprint output with criteria

    main

    You can narrow down generated fingerprints by specifying filters for any data field. You can pass these as keyword arguments or as a dictionary.

    Keyword arguments

    fpgen.generate(os='Windows', browser='Chrome', gpu={'vendor': 'Google Inc. (Intel)'})

    Dictionary format

    fpgen.generate({
        'os': 'Windows',
        'browser': 'Chrome',
        'gpu': {'vendor': 'Google Inc. (Intel)'},
    })

    Multiple constraints

    To allow the generator to select from multiple possible values for a field, pass them as a tuple:

    fpgen.generate({
        'os': ('Windows', 'MacOS'),
        'browser': ('Firefox', 'Chrome'),
    })

    Note: If you are passing many nested constraints, run fpgen decompress to improve performance.

    fpgen.generate(
        os='Windows',
        browser='Chrome',
        gpu={'vendor': 'Google Inc. (Intel)'}
    )
  2. Apply custom filters using callables

    main

    For complex logic, you can pass callable functions (lambdas or defined functions) to filter data.

    Filtering specific nested paths

    You can use dot-separated paths to target specific fields:

    # Filter by major browser version
    fpgen.generate({'client.browser.major': lambda ver: int(ver) >= 130})

    Filtering with lambdas

    # Only allow NVIDIA GPUs (strings are lowercased before being passed to the lambda)
    fpgen.generate(gpu={'vendor': lambda vdr: 'nvidia' in vdr})
    
    # Limit window size ranges
    fpgen.generate(
      window={
        'outerWidth': lambda width: 1000 <= width <= 2000,
        'outerHeight': lambda height: 500 <= height <= 1500
      }
    )

    Filtering entire dictionaries

    You can pass a function that takes the entire sub-dictionary (e.g., window) as an argument:

    def window_filter(window):
        return 1000 <= window['outerWidth'] <= 2000 and 500 <= window['outerHeight'] <= 1500
    
    fpgen.generate(window=window_filter)
    fpgen.generate({'client.browser.major': lambda ver: int(ver) >= 130})
  3. Generate a browser fingerprint

    main

    You can generate fingerprints using the top-level fpgen.generate() function or by instantiating a fpgen.Generator object. The Generator object is useful for passing common filters downward to multiple calls.

    Simple generation

    import fpgen
    fpgen.generate(browser='Chrome', os='Windows')

    Using a Generator object for inherited filters

    gen = fpgen.Generator(browser='Chrome')
    gen.generate(os='Windows') # Generates Chrome + Windows fingerprints
  4. Query possible values with `query()`

    main

    Use fpgen.query() to retrieve a list of all possible values for a specific target node.

    # List all possible browsers
    fpgen.query('browser')
    
    # Query a nested path
    fpgen.query('navigator.maxTouchPoints')

    Note: Because the model is trained on live data, queries may occasionally return anomalous values. Values with a probability lower than 0.001% are excluded from traces and queries.

    fpgen.query('browser')
  5. Trace probability distributions with `trace()`

    main

    Use fpgen.trace() to calculate the probability distribution of a target variable given specific conditions. This helps you understand what kind of data is likely to be generated under certain constraints.

    Single target trace

    # Returns a list of TraceResult objects
    results = fpgen.trace(target='browser', os='Windows')

    Multiple target trace

    # Returns a dictionary mapping probabilities to target values
    results = fpgen.trace(
      target=('browser', 'os'),
      gpu={'vendor': 'Google Inc. (Intel)'}
    )

    Reading TraceResult objects

    Each item in a single-target trace is a TraceResult object:

    result = fpgen.trace(target='browser', os='ChromeOS')[0]
    print(result.probability) # e.g., 1.0
    print(result.value)       # e.g., 'Chrome'
    fpgen.trace(target=('browser', 'os'), gpu={'vendor': 'Google Inc. (Intel)'})
  6. Generate specific data targets

    main

    If you only need a subset of the fingerprint data, use the target parameter or the generate_target shortcut.

    Using target

    Specify a single field or a dot-separated path for nested fields:

    # Generate only HTTP headers
    fpgen.generate(target='headers')
    
    # Generate a specific nested field
    fpgen.generate(os='Windows', browser='Chrome', target='headers.user-agent')

    Using generate_target shortcut

    fpgen.generate_target('headers')

    Multiple targets

    You can provide multiple targets as a list:

    fpgen.generate(target=['headers', 'client'])
    fpgen.generate(target='headers.user-agent')
  7. Configure Generator parameters

    main

    When initializing a fpgen.Generator or calling fpgen.generate(), you can use the following parameters:

    • conditions (dict, optional): Conditions for the generated fingerprint.
    • window_bounds (WindowBounds, optional): Constrain the output window size.
    • strict (bool, optional): Whether to raise an exception if the conditions are too strict.
    • flatten (bool, optional): Whether to flatten the output dictionary.
    • target (Optional[Union[str, StrContainer]]): Only generate specific value(s).
    • **conditions_kwargs: Additional conditions for the fingerprint passed as keyword arguments.
  8. Handle restrictive constraints

    main

    When strict=True (the default), the generator will raise a RestrictiveConstraints exception if the provided conditions are logically impossible or too narrow to produce a valid fingerprint. If you want the generator to attempt to find the closest possible match by relaxing constraints, set strict=False.

    from fpgen.generator import Generator
    from fpgen.exceptions import RestrictiveConstraints
    
    generator = Generator(strict=True)
    
    try:
        # This might fail if the combination is impossible
        fp = generator.generate(os='Windows', browser='Safari')
    except RestrictiveConstraints:
        print("The constraints provided are impossible to satisfy.")
  9. Manage the fpgen model with CLI commands

    main

    The fpgen CLI allows you to manage the underlying generative model files.

    • fpgen fetch: Downloads the latest model from GitHub. This is also triggered automatically on the first import or every 5 weeks.
    • fpgen decompress: Decompresses model files to increase generation speed by up to 10-50x. Note that this requires an additional 100MB+ of storage.
    • fpgen recompress: Compresses model files after you have run decompress.
    • fpgen remove: Deletes all downloaded and extracted model files.
    # Example: Fetch the latest model
    fpgen fetch
    
    # Example: Decompress for faster generation
    fpgen decompress
  10. Generate a fingerprint with `Generator.generate()`

    main

    Use the .generate() method on a Generator instance to produce a fingerprint. You can pass new conditions that will override or supplement the instance's default conditions.

    Parameters:

    • conditions (dict, optional): Conditions for the fingerprint.
    • strict (bool, optional): Whether to raise an exception if constraints are too strict. If None, it inherits the instance's strict setting.
    • flatten (bool, optional): Whether to flatten the output. If None, it inherits the instance's flatten setting.
    • target (str or StrContainer, optional): If provided, the method will only return the values for the specified target(s).
    • **conditions_kwargs: Additional conditions.

    Returns:

    • A Dict[str, Any] representing the fingerprint (or a specific value/subset if target is used).
    from fpgen.generator import Generator
    
    generator = Generator(platform='Windows')
    
    # Generate a full fingerprint
    full_fingerprint = generator.generate(browser='Chrome')
    
    # Generate only a specific target
    chrome_version = generator.generate(target='browser_version', browser='Chrome')
  11. Trace the probability distribution of variables with trace()

    main

    Use the trace() function to compute the probability distribution(s) of a target variable (or multiple variables) given specific constraints or conditions. This is useful for inspecting how certain constraints affect the generation space of fingerprints.

    Parameters

    • target (str or StrContainer): The name of the target variable(s) you want to inspect.
    • conditions (Dict[str, Any], optional): A dictionary mapping variable names to their required values.
    • flatten (bool, optional): If True, returns a flattened dictionary of targets. Defaults to False.
    • **conditions_kwargs: Additional keyword arguments used as conditions (cannot be used simultaneously with the conditions dictionary).
    • __evidence__ (Dict[str, Set[str]], optional): Internal use for passing existing evidence.

    Return Values

    • If target is a single str: Returns a List[TraceResult] containing the possible values and their probabilities, sorted by probability in descending order.
    • If target is a StrContainer: Returns a TraceResultDict (a recursive dictionary mapping paths to TraceResult lists).
    • If flatten=True and target is a StrContainer: Returns a dictionary mapping root targets to their respective List[TraceResult].

    Errors

    • ValueError: Raised if no valid target is provided.
    • RestrictiveConstraints: Raised if the provided conditions are so restrictive that no possible values exist for the target variable.
    from fpgen.trace import trace
    
    # Example 1: Tracing a single variable
    # Returns List[TraceResult]
    results = trace("browser_name", os="Windows", chrome_version="120")
    for res in results:
        print(f"{res.value}: {res.probability * 100}%")
    
    # Example 2: Tracing multiple variables with flatten=True
    # Returns Dict[str, List[TraceResult]]
    results = trace("browser_name.version", "gpu_model", flatten=True, os="Windows")