DOCSEARCHbyAlgolia
Submit Docs

trurl

repository·master·Indexed 25 days ago

https://github.com/curl/trurl

A command-line tool for advanced URL parsing and manipulation. trurl allows users to programmatically modify, extract, and transform URL components such as scheme, host, port, path, query, and fragment. Key features include component extraction via --get, modification via --set, appending data with --append, query parameter management (sorting and trimming), and the ability to output parsed URL data in JSON format.

Tokens
4.5K
Snippets
15
Records
36
Agent score
84%

What's inside trurl

  2. Set up vcpkg and curl dependencies

    master

    Use vcpkg to manage curl dependencies. It is recommended to use a short path for the vcpkg repository (e.g., C:\src\vcpkg) to avoid path length issues.

    Run the following commands in your native tools command prompt to clone vcpkg and install the required curl dependencies:

    git clone https://github.com/microsoft/vcpkg.git
.\vcpkg\bootstrap-vcpkg.bat
.\vcpkg\vcpkg install curl:x86-windows-static-md
.\vcpkg\vcpkg install curl:x64-windows-static-md
.\vcpkg\vcpkg integrate install

    To update curl in the future, run git pull in the vcpkg directory and then vcpkg upgrade.

  3. Install trurl on Windows

    master
    To install on Windows, use the Cygwin installer and ensure the following packages are selected: curl, libcurl-devel, libcurl4, make, and gcc-core. After installation, you can use make as you would on Linux.

  4. Build trurl using MSBuild

    master

    Navigate to the trurl\winbuild directory in your console to perform the build. You can specify the Configuration (Debug or Release) and the Platform (x86 or x64).

    Build Commands

    TargetCommand
    x64 Debugmsbuild /m /t:Clean,Build /p:Configuration=Debug /p:Platform=x64 trurl.sln
    x64 Releasemsbuild /m /t:Clean,Build /p:Configuration=Release /p:Platform=x64 trurl.sln
    x86 Debugmsbuild /m /t:Clean,Build /p:Configuration=Debug /p:Platform=x86 trurl.sln
    x86 Releasemsbuild /m /t:Clean,Build /p:Configuration=Release /p:Platform=x86 trurl.sln
    cd trurl\winbuild
msbuild /m /t:Clean,Build /p:Configuration=Debug /p:Platform=x64 trurl.sln

  6. Install trurl on Linux

    master

    You can compile the C source using GCC. Ensure you have the development files for libcurl installed (e.g., libcurl4-openssl-dev or libcurl4-gnutls-dev).

    $ make
cc  -W -Wall -pedantic -g   -c -o trurl.o trurl.c
cc   trurl.o  -lcurl -o trurl

  7. Verify libcurl version requirements for trurl features

    master

    trurl's capabilities are strictly tied to the version of libcurl it is built against or linked with. If you encounter unexpected behavior or missing features, check your libcurl version against these requirements:

    • The URL API: Requires libcurl 7.62.0 or newer.
    • Extracting zone id: Requires libcurl 7.65.0 or newer (error code CURLUE_NO_ZONEID requires 7.81.0).
    • Allowing spaces in URLs: Requires libcurl 7.78.0 or newer.
    • Improved error messages (curl_url_strerror()): Requires libcurl 7.80.0 or newer.
    • Punycode support (CURLU_PUNYCODE): Requires libcurl 7.88.0 or newer.
    • IPv6 literal parsing (when libcurl lacks IPv6 support): Requires libcurl 8.0.0 or newer.
    • Strict host name parsing (rejecting % in host names): Requires libcurl 8.0.0 or newer.
    • Fixed URL encoding of fragments: Requires libcurl 8.1.0 or newer.
    • Rejection of bad IPv4 numerical addresses: Requires libcurl 8.1.0 or newer.
    • Prevention of illegal schemes: Requires libcurl 8.1.0 or newer.

  9. Accept spaces in the URL path

    master

    By default, spaces in a URL might be treated as delimiters or errors. Use the --accept-space flag to allow spaces in the input URL and have them automatically percent-encoded in the output.

    $ trurl "https://curl.se/this has space/index.html" --accept-space
https://curl.se/this%20has%20space/index.html

  10. Manipulate URL components with --set

    master

    Use the --set flag to replace or create specific parts of a URL, such as the host, scheme, or port.

    # Replace the hostname
$ trurl --url https://curl.se --set host=example.com
https://example.com/

# Create a URL by setting components
$ trurl --set host=example.com --set scheme=ftp
ftp://example.com/

# Change port number
$ trurl --url https://curl.se/we/../are.html --set port=8080
https://curl.se:8080/are.html

  11. Append segments to a URL

    master

    Use the --append flag to add segments to the path or query components of a URL.

    # Append a path segment
$ trurl --url https://curl.se/hello --append path=you
https://curl.se/hello/you

# Append a query segment
$ trurl --url "https://curl.se?name=hello" --append query=search=string
https://curl.se/?name=hello&search=string

  12. Manage query parameters

    master

    trurl provides several ways to manipulate query strings:

    • --qtrim <pattern>: Remove tracking tuples from the query using a glob pattern.
    • --sort-query: Sort the key/value pairs in the query component.
    • --query-separator <char>: Work with queries that use a specific separator (e.g., ;).
    # Remove tracking tuples
$ trurl "https://curl.se?search=hey&utm_source=tracker" --qtrim "utm_*"
https://curl.se/?search=hey

# Sort query pairs
$ trurl "https://example.com?b=a&c=b&a=c" --sort-query
https://example.com?a=c&b=a&c=b

# Use a semicolon separator
$ trurl "https://curl.se?search=fool;page=5" --qtrim "search" --query-separator ";"
https://curl.se?page=5