DOCSEARCHbyAlgolia
Submit Docs

Dexador Common Lisp HTTP Client

repository·master·Indexed 19 days ago

https://github.com/fukamachi/dexador

A performance-oriented HTTP client for Common Lisp featuring connection pooling and a clean API. Dexador provides high-level functions like `dex:get` and `dex:post`, supports Basic and Bearer authorization, manages cookies via `cl-cookie`, and handles redirects, proxies (HTTP and SOCKS5), and HTTP error conditions.

Tokens
2.6K
Snippets
10
Records
11
Agent score
16%

What's inside Dexador

  1. Overview of Dexador

    master
    Dexador is an HTTP client for Common Lisp designed with neat APIs and connection-pooling. It is optimized for speed, particularly when making multiple requests to the same host. Compared to Drakma, Dexador signals conditions when HTTP requests fail and does not require OpenSSL on Windows.

  2. Handle redirects and limit max-redirects

    master

    Dexador automatically follows redirects (3XX responses) for GET and HEAD requests. The fourth return value of the request function contains the final URI after all redirects.

    You can control the number of redirects using the :max-redirects option. The default is 5. If the limit is exceeded, the function returns the last response instead of raising a condition.

    ;; The 4th return value is the final URI
(dex:head "http://lisp.org")
;=> "" ; body
;   200 ; status
;   #<HASH-TABLE ...> ; response-headers
;   #<QURI.URI.HTTP:URI-HTTP http://lisp.org/index.html> ; final uri
;   NIL ; stream

;; Limiting redirects
(dex:get "http://example.com" :max-redirects 3)

  4. Configure Proxy settings

    master

    You can specify a proxy for individual requests using the :proxy keyword argument. Supported protocols include http and socks5.

    Alternatively, you can set a default proxy for all requests by setting the dex:*default-proxy* variable, which defaults to the values of the HTTPS_PROXY or HTTP_PROXY environment variables.

    ;; HTTP Proxy
(dex:get "http://lisp.org/" :proxy "http://proxy.yourcompany.com:8080/")

;; SOCKS5 Proxy
(dex:get "https://www.facebookcorewwwi.onion/" :proxy "socks5://127.0.0.1:9150")

  5. Basic Usage of Dexador

    master

    Dexador provides high-level functions for performing HTTP requests. The most common methods are dex:get and dex:post.

    (dex:get "http://lisp.org/")

(dex:post "https://example.com/login"
          :content '(("name" . "fukamachi") ("password" . "1ispa1ien")))

  6. Handle HTTP errors and status codes

    master

    Dexador signals the http-request-failed condition when a server returns a 4xx or 5xx status code.

    Common ways to handle errors:

    • Specific errors: Catch dex:http-request-bad-request for 400 errors.
    • General errors: Catch dex:http-request-failed and inspect (dex:response-status e).
    • Ignore specific errors: Use dex:ignore-and-continue with handler-bind to skip errors like 404 Not Found.
    • Retry logic: Use dex:retry-request with handler-bind to automatically retry failed requests.
    ;; Catching 400 Bad Request
(handler-case (dex:get "http://lisp.org")
  (dex:http-request-bad-request () nil)
  (dex:http-request-failed (e) (format t "Status: ~D" (dex:response-status e))))

;; Ignoring 404 Not Found
(handler-bind ((dex:http-request-not-found #'dex:ignore-and-continue))
  (dex:get "http://lisp.org/missing"))

;; Retrying with interval
(let ((retry-request (dex:retry-request 5 :interval 3)))
  (handler-bind ((dex:http-request-failed #'retry-request))
    (dex:get "http://lisp.org")))

  7. Manage cookies with cl-cookie

    master

    Dexador uses cl-cookie for cookie management. To persist cookies across multiple requests, pass a cookie-jar instance to the :cookie-jar keyword argument.

    (defvar *cookie-jar* (cl-cookie:make-cookie-jar))

;; Request 1: Server sets a cookie
(dex:head "https://mixi.jp" :cookie-jar *cookie-jar*)

;; Request 2: Cookie is automatically sent back
(dex:head "https://mixi.jp" :cookie-jar *cookie-jar*)

  8. Configure Authorization (Basic and Bearer)

    master

    Dexador supports two types of authorization. Note that you can only provide one of them per request.

    • Basic Authorization: Use :basic-auth with a cons containing (username . password).
    • Bearer Authorization: Use :bearer-auth with a string token.
    ;; Basic Auth
(dex:head "http://www.hatena.ne.jp/" :basic-auth '("nitro_idiot" . "password"))

;; Bearer Auth
(dex:head "http://www.hatena.ne.jp/" :bearer-auth "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9")

  9. Post form-data and multipart/form-data

    master

    You can send data via the :content keyword argument using an association list.

    • application/x-www-form-urlencoded: Use an association list of strings.
    • multipart/form-data: If the association list contains a pathname, Dexador automatically switches to multipart encoding.
    • Custom Content-Type: You can override the content type for a specific key by using a cons where the cdr is a :content-type override.
    ;; application/x-www-form-urlencoded
(dex:post "http://example.com/entry/create"
          :content '(("title" . "The Truth About Lisp")
                     ("body" . "In which the truth about lisp is revealed...")))

;; multipart/form-data (detected via pathname)
(dex:post "http://example.com/entry/create"
          :content '(("photo" . #P"images/2015030201.jpg")))

;; Custom content-type override
(dex:post "http://example.com/upload"
          :content '(("key" ,(make-array 5 :element-type '(unsigned-byte 8)) :content-type "application/octets")))

  10. Dexador Request API Reference

    master

    The core of Dexador is the dex:request function (and its wrappers get, post, head, put, patch, delete).

    Arguments:

    • uri (string or quri:uri): The target URL.
    • method (keyword): :GET, :HEAD, :OPTIONS, :PUT, :POST, or :DELETE. Default is :GET.
    • version (number): HTTP version (e.g., 1.1). Default is 1.1.
    • content (string, alist, or pathname): The request body.
    • headers (alist): Request headers. Setting a value to NIL prevents it from being sent.
    • basic-auth (cons): (username . password).
    • bearer-auth (string): Token for bearer auth.
    • cookie-jar (cl-cookie jar): For managing cookies.
    • connect-timeout (fixnum): Seconds to wait for connection. Default is 10.
    • read-timeout (fixnum): Seconds to wait for body read. Default is 10.
    • keep-alive (boolean): Keep connection open. Default is T.
    • use-connection-pool (boolean): Cache socket connections. Default is T.
    • max-redirects (fixnum): Limit for redirects. Default is 5.
    • verbose (boolean): If T, dumps HTTP request headers for debugging.
    • insecure (boolean): Bypass SSL certificate verification. Default is NIL.
    • proxy (string): Proxy URL.
    • want-stream (boolean): Return response body as a stream.
    • force-binary (boolean): Suppress auto-decoding of response body.

    Returns (4-value list):

    1. body: Octet vector or string (if Content-Type is text/*).
    2. status: Integer HTTP status code.
    3. response-headers: Hash table of downcased header keys.
    4. uri: The final quri:uri after redirects.
    5. stream: A usocket stream (may be NIL if :keep-alive is NIL or connection is closed).
    ;; Signature of the primary request function
(dex:request uri &key (method get) (version 1.1) content headers
             basic-auth cookie-jar (connect-timeout *default-connect-timeout*)
             (read-timeout *default-read-timeout*) (keep-alive t) (use-connection-pool t)
             (max-redirects 5) ssl-key-file ssl-cert-file ssl-key-password stream
             (verbose *verbose*) force-binary force-string want-stream proxy
             (insecure *not-verify-ssl*) ca-path)