tldts

repository·master·Indexed 20 days ago

https://github.com/remusao/tldts

A high-performance JavaScript library for extracting hostnames, domains, public suffixes, TLDs, and subdomains from URLs and hostnames. It offers multiple packages including a standard version, an ICANN-rules-only version (tldts-icann), and a probabilistic, high-speed experimental version (tldts-experimental). The library provides a comprehensive parse() method and single-purpose methods like getDomain() and getSubdomain() for optimized performance.

Tokens
9K
Snippets
39
Records
50
Agent score
72%

What's inside tldts

  1. Compare tldts features with other domain parsing libraries

    master

    When choosing a domain parsing library, tldts provides a comprehensive feature set compared to other popular options like psl, tld.js, or parse-domain.

    Key feature advantages of tldts include:

    • IDNA support: Handles Internationalized Domain Names in Applications.
    • URL support: Can parse full URLs in addition to hostnames.
    • IP support: Handles IP addresses.
    • Comprehensive API: Provides getDomain, getPublicSuffix, and handles ICANN/Private rules.
    • Self-contained: Ships its own suffix lists and has zero dependencies (unlike psl, tld.js, and others which depend on punycode).
  2. Optimize performance with tldts options

    master

    To achieve maximum parsing speed, you can use granular options to fine-tune behavior. For example, if you are certain that your inputs are already hostnames (and not full URLs), you can disable the hostname extraction step to improve performance using the { extractHostname: false } option.

    // Example of disabling hostname extraction for performance
    getDomain('example.com', { extractHostname: false });
  3. When to use tldts-experimental vs tldts

    master

    The tldts-experimental package is a faster, unstable version of the main tldts library. It uses a probabilistic data-structure instead of the optimized DAWG (direct acyclic word graph) used in the standard tldts package.

    Use tldts-experimental if you need:

    • Smaller footprint: Reduced bundle size and memory usage.
    • Instant loading: No data loading or parsing required at runtime.
    • Higher speed: Lookups are up to 1.5-2x faster.

    Trade-offs:

    • Accuracy: Because it is probabilistic, there is a possibility of unlikely false positives (similar to how bloom filters behave).
    • Stability: It is considered an experimental and unstable version.

    Recommendation: Use the default tldts package for most use cases unless your specific performance or bundle size constraints require the experimental version.

  4. Evaluate tldts performance and resource usage

    master

    The tldts library is optimized for high-performance environments. Based on micro-benchmarks:

    Performance (Operations per second)

    tldts is significantly faster than many competitors, particularly psl. For example, in getPublicSuffix operations, tldts achieves ~1,280,063 ops/s compared to psl's ~1,654 ops/s.

    Memory Usage

    tldts maintains a relatively low memory footprint. While tldts-experimental is the most efficient (~229 KB after GC), the standard tldts uses approximately 1.792 MB after Garbage Collection.

    Loading Time and Bundle Size

    • Loading Time: tldts has a fast startup time (~64.48ms mean), making it suitable for environments where quick initialization is required (e.g., mobile devices).
    • Bundle Size: The minified bundle size is approximately 95KB (37KB gzipped), which is competitive with or better than psl and tld.js.
  5. Detect IANA special-use domains

    master

    To identify reserved special-use names like localhost, *.test, or *.onion, you must enable the detectSpecialUse: true option. By default, this is false to optimize the common path. When enabled, the isSpecialUse field in the result will be populated.

    parse('http://printer.local/', { detectSpecialUse: true });
    // { ... isSpecialUse: true, publicSuffix: 'local', ... }
  6. Securely parse untrusted URLs

    master

    Because tldts uses a pragmatic, non-compliant URL parser for speed, it should not be used for security-critical decisions (like origin checks or SSRF protection) on untrusted input.

    The Safe Pattern:

    1. Use a standard, compliant platform parser (like new URL()) to extract the hostname.
    2. Pass that hostname to tldts with { extractHostname: false } to perform the public-suffix/domain split. This ensures the two parsers never disagree on where the host begins.
    const { getDomain } = require('tldts');
    
    // 1. Use platform parser for security
    const { hostname } = new URL(untrustedUrl);
    
    // 2. Use tldts only for the split, skipping hostname extraction
    const domain = getDomain(hostname, { extractHostname: false });
  7. Migrate from psl to tldts

    master

    When migrating from the psl library, note that tldts defaults to only considering the ICANN section of the Public Suffix List. To match psl's behavior (which includes the Private section), you must pass { allowPrivateDomains: true } to your tldts calls.

    Tasktldts approachNote
    Parsing hostnametldts.parse(host, { allowPrivateDomains: true })Use option for private suffixes
    Parsing URLtldts.parse(url, { allowPrivateDomains: true })tldts handles URLs directly
    Getting domaintldts.getDomain(host, { allowPrivateDomains: true })More efficient than parse
    Getting Public Suffixtldts.getPublicSuffix(host, { allowPrivateDomains: true })More efficient than parse
  8. Configure tldts options

    master

    All public API functions accept an options object to customize behavior and fine-tune performance.

    OptionTypeDefaultDescription
    extractHostnamebooleantrueIf false, inputs are treated as valid hostnames directly.
    validateHostnamebooleantrueIf false, parsing proceeds even if the hostname is invalid.
    detectIpbooleantrueEnables IP address detection.
    detectSpecialUsebooleanfalseDetects IANA special-use domains and sets isSpecialUse.
    mixedInputsbooleantrueIf false, assumes inputs are only URLs (improves speed).
    validHostsstring[] | nullnullSpecifies extra valid suffixes (e.g., for localhost).
  9. Handle localhost and custom hostnames with validHosts

    master

    By default, getDomain and getSubdomain only work with known and valid TLDs. Since localhost is a valid hostname but not a TLD, these methods will return null unless you explicitly provide it in the validHosts option.

    const tldts = require('tldts-icann');
    
    tldts.getDomain('localhost'); // returns null
    
    // Use validHosts to treat localhost as a valid domain
    tldts.getDomain('localhost', { validHosts: ['localhost'] }); // returns 'localhost'
    tldts.getSubdomain('vhost.localhost', { validHosts: ['localhost'] }); // returns 'vhost'
  10. Handle custom hostnames like localhost using validHosts

    master

    By default, getDomain and getSubdomain only work with known and valid TLDs. To work with hostnames like localhost or custom internal hostnames, you must pass them in the validHosts option array.

    const tldts = require('tldts');
    
    // Default behavior returns null for non-TLD hostnames
    tldts.getDomain('localhost'); // returns null
    
    // Use validHosts to enable parsing
    tldts.getDomain('localhost', { validHosts: ['localhost'] }); // returns 'localhost'
    tldts.getSubdomain('vhost.localhost', { validHosts: ['localhost'] }); // returns 'vhost'