repository·main·Indexed 41 days ago
https://github.com/python-hyper/rfc3986A Python implementation of RFC 3986 providing robust URI parsing, validation, and normalization. It serves as a more capable alternative to urllib.parse, offering component-level control via URIReference and URIBuilder, as well as support for RFC 6874 zone identifiers in IPv6 addresses. The library includes tools for Internationalized Resource Identifiers (IRIs) via the IRIReference class and provides raw ABNF-derived regular expressions for custom validation.
rfc3986 library is a Python implementation of RFC 3986, providing tools for URI validation and authority parsing. Additionally, it supports RFC 6874, which extends support for zone identifiers in IPv6 addresses.The rfc3986 library provides four primary functional areas for working with URIs:
urllib.parse Compatibility: Providing an API designed to be a drop-in replacement for Python's standard urllib.parse module.Note on IRIs: The library currently does not support Internationalized Resource Identifiers (IRIs) as defined in RFC 3987.
Because rfc3986 strictly adheres to RFC 3986, it requires an authority component to be preceded by //.
If you attempt to parse a URI like a GitHub SSH URL (git@github.com:user/repo) without the leading slashes, the parser will treat the entire string as a path because it does not see the // required to trigger authority parsing.
To ensure the authority (e.g., git@github.com) is correctly identified, the URI must include the // prefix.
import rfc3986
# Strict parsing treats this as a path because '//' is missing
print(rfc3986.uri_reference('git@github.com:sigmavirus24/rfc3986'))
# URIReference(scheme=None, authority=None, path='git@github.com:sigmavirus24/rfc3986', ...)
# Adding '//' allows the authority to be parsed correctly
print(rfc3986.uri_reference('//git@github.com:sigmavirus24/rfc3986'))
# URIReference(scheme=None, authority='git@github.com:sigmavirus24', path='/rfc3986', ...)The URIBuilder class is immutable. Every time you call a method to modify the URI (like add_scheme or add_path), it returns a new instance of URIBuilder. This allows you to create a base builder and derive multiple different URIs from it without affecting the original object.
from rfc3986 import builder
# Create a base builder
github_builder = builder.URIBuilder().add_scheme('https').add_host('api.github.com')
# Derive different paths from the same base
user_uri = github_builder.add_path('/users/sigmavirus24').finalize().unsplit()
repo_uri = github_builder.add_path('/repos/sigmavirus24/rfc3986').finalize().unsplit()
print(user_uri) # https://api.github.com/users/sigmavirus24
print(repo_uri) # https://api.github.com/repos/sigmavirus24/rfc3986urllib.parse. While rfc3986 parses URIs differently than urllib.parse (which may lead to subtle differences in edge cases), it aims to implement the same API so you can seamlessly swap urlparse for rfc3986 in your existing code.The rfc3986 library provides two primary ways to parse URIs depending on your use case:
rfc3986.api.urlparse: Best suited to completely replace urllib.parse.urlparse. It returns a class that is designed to be indistinguishable from urllib.parse.ParseResult.rfc3986.api.uri_reference: Best when you are not replacing existing urllib.parse usage. It provides convenience methods for safely normalizing URIs.Both methods return objects that support immutability via the copy_with pattern.
import rfc3986
# For replacing urllib.parse
url = rfc3986.urlparse('https://github.com/sigmavirus24/rfc3986')
# For general URI reference parsing and normalization
uri = rfc3986.uri_reference('https://github.com/sigmavirus24/rfc3986')Both ParseResult (from urlparse) and URIReference (from uri_reference) objects are immutable. To change a component, use the copy_with() method to create a new instance with overridden attributes, then call unsplit() to convert it back into a string.
To remove a component entirely, pass None as the value for that component in copy_with().
Supported attributes for overriding include:
schemeuserinfoportauthoritypathqueryfragmentimport rfc3986
url = rfc3986.urlparse('https://github.com/sigmavirus24/rfc3986')
# Create a new URL with a different userinfo and port
new_url = url.copy_with(
userinfo='username:password',
port='443',
).unsplit()
print(new_url) # https://username:password@github.com:443/sigmavirus24/rfc3986
# Remove the path component
no_path_uri = rfc3986.uri_reference('https://github.com/path').copy_with(path=None).unsplit()
print(no_path_uri) # https://github.comTo validate URIs, create an instance of rfc3986.validators.Validator. You can chain configuration methods on the validator instance to define your validation rules (such as allowed schemes, hosts, or required components) and then call .validate() on a uri_reference object. If validation fails, the library raises specific exceptions like UnpermittedComponentError, PasswordForbidden, MissingComponentError, or InvalidComponentsError.
from rfc3986 import validators, uri_reference
# Create a validator with specific rules
validator = validators.Validator().allow_schemes('https').allow_hosts('github.com')
# Validate a URI reference
uri = uri_reference('https://github.com/sigmavirus24/rfc3986')
validator.validate(uri)Use the rfc3986.builder.URIBuilder class to construct valid rfc3986.uri.URIReference instances. The builder handles normalization and ensures that components are safe for real-world use, avoiding the pitfalls of manual string concatenation. Most methods on URIBuilder are chainable, allowing you to build a URI in a single expression. To get the final result, use .finalize().unsplit() or .finalize().geturl().
from rfc3986 import builder
# Building a basic URL with scheme and host
uri = builder.URIBuilder().add_scheme('https').add_host('github.com').finalize().unsplit()
print(uri)
# Output: https://github.comInstall the rfc3986 package using pip. You can use the standard pip command, or the module-specific python -m pip syntax to ensure it is installed for your current Python interpreter.
pip install rfc3986
python -m pip install rfc3986
python3.11 -m pip install rfc3986The Validator acts as a configuration object that accumulates constraints. It uses a builder pattern, where methods like allow_schemes or require_presence_of return self, allowing you to chain multiple configuration calls together.
When validate(uri) is called, the validator performs the following checks in order:
forbid_use_of_password() was called, it checks if a password exists in the userinfo component.require_presence_of are not None.check_validity_of conform to RFC 3986 regex patterns.scheme, host, and port (if configured) belong to the sets of allowed values provided via allow_* methods.If any check fails, it raises specific exceptions such as MissingComponentError, UnpermittedComponentError, PasswordForbidden, or InvalidComponentsError.
# Example of chaining configuration
validator = (validators.Validator()
.require_presence_of('scheme')
.allow_schemes('https')
.forbid_use_of_password())The URIBuilder class is used to build a URI Reference from individual components. It is recommended to instantiate it without arguments and then use its fluent API to populate fields. Each add_* method returns a new URIBuilder instance, allowing for method chaining. To get the final result, use .finalize() to get a URIReference object or .geturl() to get the URL as a string.
from rfc3986.builder import URIBuilder
# Method chaining to build a URL
url = URIBuilder().add_scheme('https').add_host('github.com').add_path('sigmavirus24/rfc3986').geturl()
print(url) # 'https://github.com/sigmavirus24/rfc3986'