Yauaa (Yet Another UserAgent Analyzer)

Yauaa (Yet Another UserAgent Analyzer) is a Java library designed to parse and analyze User-Agent strings and, where available, User-Agent Client Hints. It extracts a wide range of relevant attributes from these strings to provide detailed device, browser, and application information.

Key features:

• Supports Java, Scala, and Kotlin.
• Provides ready-to-use User Defined Functions (UDFs) for various processing systems (e.g., NiFi, Elasticsearch).
• Capable of detecting robots, browsers, apps, hacking tools, and more.

Yauaa is a Java library designed to parse and analyze User-Agent strings and, where available, User-Agent Client Hints. Its primary goal is to extract as many relevant device and browser attributes as possible from these strings.

Key features include:

• Support for Java, Scala, and Kotlin.
• Ready-to-use User Defined Functions (UDFs) for various data processing systems.
• Ability to extract detailed metadata from complex User-Agent headers.

Yauaa's performance varies based on the complexity and length of the UserAgent string being analyzed.

• Raw Analysis Speed: Typically ranges from 500 to 4,000 UserAgents per second (averaging ~2,000 per second or ~0.5ms per analysis).
• Cached Analysis Speed: Yauaa uses an internal LRU (Least Recently Used) cache. If a UserAgent has been seen before, retrieval speed is significantly faster, exceeding 1,000,000 per second (< 1 microsecond).
• Memory Footprint: The engine requires approximately 220MiB of RAM just for the core engine, excluding any additional caching overhead.

Yauaa may report ?? for certain version fields (e.g., OperatingSystemNameVersion: 'Windows NT ??'). This is often not a bug, but a result of intentional User-Agent reduction/manipulation by browser engines like Chromium.

When browsers remove specific information from the User-Agent header to prevent fingerprinting, Yauaa detects these documented manipulations and may overrule the literal values in the header to avoid providing false information. If the information is no longer present in the User-Agent string, Yauaa cannot invent it.

Unlike many analyzers that rely on ordered lists of regular expressions, Yauaa uses a two-step approach to handle the complexity and 'lying' nature of UserAgent strings:

1. Tree Parsing: An ANTLR4-based parser converts the UserAgent string into a structured tree. This handles >99% of real-world traffic. If a string cannot be parsed, it is typically due to manual invalidation (hacker/malformed strings).
2. Matcher Execution: A collection of matchers walks the parsed tree. A matcher triggers if specific patterns are found in the tree. Each matcher provides a value and a confidence level (ranging from 0 to 10,000,000) for specific attributes.

Conflict Resolution: If multiple matchers attempt to set the same attribute, the one with the highest confidence level wins.

Yauaa uses a performance optimization where it walks the parsed tree once and fires findings into a precomputed hashmap that points to applicable matcher actions. This design results in the following characteristics:

• Matching Speed: Relatively fast, even with hundreds of matchers.
• Startup Time: Relatively slow due to precomputation.
• Memory Footprint: High, due to the large number of matchers, the size of the precomputed hashmap, and the cache of parsed UserAgents.

When writing require or extract patterns, you use 'breadcrumb' style paths to navigate the parsed User-Agent tree.

For a User-Agent like: foo/1.0 ( one ; two three; four ) bar/2.0 (five;six seven)

Common path patterns include:

• agent.(1)product.(1).name: Targets the name of the first product.
• agent.(1)product.(1).version: Targets the version of the first product.
• agent.(1)product.(1).comments.(1).entry: Targets the first entry in the first comment block.
• agent.(1)product[1-1]: Uses index-based notation for specific segments (e.g., the first part of a product string).

Note: Most paths are automatically trimmed of whitespace during parsing.

agent.(1)product.(1)name="foo"
agent.(1)product.(1)version="1.0"
agent.(1)product.(1)comments.(1)entry="one"

Yauaa models User-Agents based on RFC-2616. The string is parsed into a tree of nodes:

• agent: The root node representing the whole string.
• product: The primary components of the string. Each product can have multiple versions and comments.
• version: The version information associated with a product.
• comments: A block of text (often in parentheses) containing one or more entries separated by semicolons (;).
• entry: Individual pieces of information within a comment block.

Nodes are indexed numerically (e.g., (1), (2)) within their parent context to allow precise path expressions.