DOCSEARCHbyAlgolia
Submit Docs

yarl

repository·master·Indexed 23 days ago

https://github.com/aio-libs/yarl

A high-performance URL parsing and manipulation library for Python. It provides an immutable `URL` class that handles encoding/decoding and provides easy access to URL components, including decoded and encoded properties, query parameter management, and path manipulation via the `/` operator or `joinpath()` method.

Tokens
5.1K
Snippets
18
Records
28
Agent score
70%

What's inside yarl

  1. Access decoded and encoded URL properties

    master

    The URL object provides two types of properties for accessing URL components:

    1. Decoded properties: Return the human-readable, unencoded version of the component (e.g., user, password, host, path, query_string).
    2. Encoded properties: Prefixed with raw_, these return the component in its percent-encoded format (e.g., raw_user, raw_host, raw_path).

    Use decoded properties for logic and display, and raw_ properties when you need the exact wire format or need to avoid double-unquoting issues.

  2. Why booleans are not supported in the URL query API

    master
    The yarl URL query API does not accept boolean types. This is because there is no single standard for boolean representation in URLs (e.g., true/false, yes/no, 1/0). Users should convert booleans into strings using their own preferred translation protocol before passing them to the API.

  3. How yarl handles encoding and decoding

    master

    Strings passed to the URL constructor or modification methods are automatically encoded into a canonical representation.

    • Decoding: Regular properties (like .path) return percent-decoded strings.
    • Encoding: Use raw_ properties (like .raw_path) to get the encoded strings.
    • Human Readable: Use .human_repr() to get a human-readable string representation of the URL.

  4. Install the pure-Python version of yarl

    master

    If you want to skip compilation (e.g., in environments without a C compiler), you can install the pure-Python version. Note that the pure-Python version is significantly slower than the compiled version. You can opt-in using the pure-python PEP 517 configuration setting or by setting the YARL_NO_EXTENSIONS environment variable to a non-empty value.

    $ pip install yarl --config-settings=pure-python=false

  5. Install yarl via pip

    master

    Install the yarl library using pip. Note that yarl is Python 3 only. PyPI provides binary wheels for Linux, Windows, and MacOS. On other operating systems, it will attempt to compile from source, which requires a C compiler and Python headers.

    $ pip install yarl

  7. Manage IDNA and host encoding caches

    master

    Because IDNA conversion and host encoding are computationally expensive, yarl uses a global LRU cache to store results. You can inspect and manage this cache using the following functions:

    • cache_info(): Returns a dictionary containing cache statistics for idna_encode, idna_decode, and encode_host. Each value is a CacheInfo object (similar to functools.lru_cache).
    • cache_clear(): Clears the IDNA and host encoding caches.
    • cache_configure(*, idna_encode_size=256, idna_decode_size=256, encode_host_size=512): Sets the maximum size for the respective caches. Passing None to a parameter makes that specific cache unbounded, which may improve performance but increases memory footprint.
    >>> yarl.cache_info()
{'idna_encode': CacheInfo(hits=5, misses=5, maxsize=256, currsize=5),
 'idna_decode': CacheInfo(hits=24, misses=15, maxsize=256, currsize=15),
 'encode_host': CacheInfo(hits=0, misses=0, maxsize=512, currsize=0)}

  8. Extract file names and extensions from URL paths

    master

    Use these properties to inspect the final segment of a URL path:

    • URL.name: The last part of the decoded path segments.
    • URL.raw_name: The last part of the encoded path segments.
    • URL.suffix: The file extension of the name (e.g., .txt).
    • URL.raw_suffix: The file extension of the encoded name.
    • URL.suffixes: A list of all file extensions in the name (e.g., ('.tar', '.gz') for file.tar.gz).
    • URL.raw_suffixes: A list of all encoded file extensions.
    >>> URL('http://example.com/path/to.txt').name
'to.txt'
>>> URL('http://example.com/path/to.txt').suffix
'.txt'
>>> URL('http://example.com/path/to.tar.gz').suffixes
('.tar', '.gz')

  9. Get a human-readable URL representation

    master
    By default, converting a URL object to a string via str() returns the percent-encoded version (e.g., for non-ASCII characters). To get a human-readable version of the URL where non-ASCII characters are preserved in their original form, use the .human_repr() method.

  11. Access URL query parameters

    master

    The query string can be accessed as a raw string or as a parsed collection:

    • URL.query_string: The decoded query string (e.g., a1=a&a2=b).
    • URL.raw_query_string: The encoded query string.
    • URL.query: A multidict.MultiDictProxy representing the parsed query parameters in decoded form. This allows for easy key-value access.
    >>> URL('http://example.com/path?a1=a&a2=b').query_string
'a1=a&a2=b'
>>> URL('http://example.com/path?a1=a&a2=b').query
<MultiDictProxy('a1': 'a', 'a2': 'b')>

  12. Use the URL class for parsing and accessing URL parts

    master
    Construct a URL object from a string. You can access all parts of the URL (scheme, user, password, host, port, path, query, and fragment) via properties. Regular properties return percent-decoded values, while raw_ versions return the encoded strings.