Got HTTP Request Library

repository·main·Indexed 12 days ago

https://github.com/sindresorhus/got

A human-friendly and powerful HTTP request library for Node.js, version 15.1.0. It features native ESM support and provides a Promise API and a Stream API. Key capabilities include HTTP/2 support, built-in retries, RFC 7234 caching, pagination, and full TypeScript support. It allows for specialized client creation via got.extend() and offers advanced control over timeouts, timings, and request hooks.

Tokens
29.3K
Snippets
86
Records
129
Agent score
95%

What's inside Got

  1. Handle RequestError and its subclasses

    main
    A RequestError (code: ERR_GOT_REQUEST_ERROR) is the base error class for most request failures. All specific error types listed below inherit from RequestError and include a code property representing the specific error class (e.g., ECONNREFUSED).
  2. Customize pagination logic using the `paginate` function

    main

    The paginate function allows you to define how to find the next page. It receives an object containing { response, currentItems, allItems }.

    It must return an object representing Got options for the next request (e.g., { url: '...' } or { searchParams: { page: 2 } }). These options are merged with the previous request. To stop pagination, return false.

    Security Note: When navigating to a different origin, Got strips sensitive headers like authorization and cookie. If you need to forward them to a trusted next-page URL, you must return them explicitly in the object returned by paginate.

  3. Understand Got error metadata

    main

    All errors thrown by Got contain specific metadata that can be used for debugging or programmatic handling. Key properties available on Got error instances include:

    • code: A string identifier (e.g., ERR_NON_2XX_3XX_RESPONSE).
    • options: The Options instance used for the request.
    • request: The Got Stream instance associated with the request.
    • response (optional): The Got Response instance.
    • timings (optional): Points to response.timings.

    Note that if a root error has a code property, the error codes may differ. The original root error is preserved and propagated via the standard JavaScript cause property.

  4. Configure client-side certificates and keys

    main

    To perform mutual TLS (mTLS) or provide client credentials, use the key, certificate, and pfx options within the https object.

    • key: Private keys in PEM format. Can be a single key or an array of objects containing {pem: string | Uint8Array, passphrase: string}.
    • certificate: Certificate chains in PEM format. One chain should be provided per private key.
    • pfx: PFX or PKCS12 encoded private key and certificate chain. This is an alternative to providing key and certificate separately. Can be a single buffer or an array of objects: {buffer: string | Uint8Array, passphrase?: string}.
    • passphrase: A shared passphrase used for a single private key or a PFX.
    import got from 'got';
    import fs from 'node:fs';
    
    // Single key with certificate
    await got('https://example.com', {
    	https: {
    		key: fs.readFileSync('./client_key.pem'),
    		certificate: fs.readFileSync('./client_cert.pem')
    	}
    });
    
    // Multiple keys with different passphrases
    await got('https://example.com', {
    	https: {
    		key: [
    			{pem: fs.readFileSync('./client_key1.pem'), passphrase: 'passphrase1'},
    			{pem: fs.readFileSync('./client_key2.pem'), passphrase: 'passphrase2'},
    		],
    		certificate: [
    			fs.readFileSync('./client_cert1.pem'),
    			fs.readFileSync('./client_cert2.pem'),
    		]
    	}
    });
    
    // Multiple encrypted PFX's with different passphrases
    await got('https://example.com', {
    	https: {
    		pfx: [
    			{
    				buffer: fs.readFileSync('./key1.pfx'),
    				passphrase: 'passphrase1'
    			},
    			{
    				buffer: fs.readFileSync('./key2.pfx'),
    				passphrase: 'passphrase2'
    			}
    		]
    	}
    });
  5. Understand individual timeout event lifecycles

    main

    Each property in the timeout object governs a specific phase of the request:

    • lookup: Starts when a socket is assigned; ends when the hostname is resolved. Does not apply to IP addresses or Unix domain sockets. (Recommended max: 100ms).
    • connect: Starts when lookup completes; ends when the socket is fully connected.
    • secureConnect: Starts when connect completes; ends when the TLS handshake completes. Applies only to HTTPS.
    • socket: Starts when the socket is connected. Resets whenever new data is transferred (similar to Node.js socket.setTimeout).
    • send: Starts when the socket is connected; ends when all data has been written to the underlying OS (does not guarantee the remote end received it).
    • response: Starts when the request has been flushed; ends when headers are received.
    • request: The global timeout. Starts when the request is initiated; ends when the response's end event fires.
  6. Extend Got instances with custom options and hooks

    main

    You can extend a Got instance with configuration like headers, responseType, and context.

    If you want to pass custom options that Got doesn't natively recognize (e.g., a token), Got will throw a validation error. To handle this, use the hooks.init hook to intercept the raw options, move your custom property into options.context, and delete it from the raw options object.

    import got from 'got';
    
    const instance = got.extend({
    	prefixUrl: 'https://api.github.com',
    	headers: {
    		accept: 'application/vnd.github.v3+json'
    	},
    	responseType: 'json',
    	context: {
    		token: process.env.GITHUB_TOKEN,
    	},
    	hooks: {
    		init: [
    			(raw, options) => {
    				if ('token' in raw) {
    					options.context.token = raw.token;
    					delete raw.token;
    				}
    			}
    		]
    	}
    });
  7. Create a session with shared cookies

    main

    You can create a persistent session by extending Got with a shared CookieJar. This ensures cookies are automatically saved and reused across all requests made with that extended instance.

    import got from 'got';
    import {CookieJar} from 'tough-cookie';
    
    const cookieJar = new CookieJar();
    const sessionGot = got.extend({cookieJar});
    
    // The cookie is saved within the cookie jar of the session
    await sessionGot('http://httpbin.org/cookies/set?my_cookie=my_value');
    
    await sessionGot('http://httpbin.org/cookies', {responseType: 'json', resolveBodyOnly: true});
    //=> {cookies: {my_cookie: 'my_value'}}
    
    // The cookies only apply to the session
    await got('http://httpbin.org/cookies', {responseType: 'json', resolveBodyOnly: true});
    //=> {cookies: {}}
  8. Behavioral changes in Got compared to Request

    main

    If you are familiar with request, note the following changes in how got handles specific configurations:

    • Agent: The agent option is now an object containing http, https, and http2 properties.
    • Timeout: The timeout option is now an object, allowing you to set timeouts for specific events.
    • Search Parameters: The searchParams option is always serialized using URLSearchParams.
    • Custom Query Strings: To pass a custom query string, provide it directly within the url option (e.g., got('https://example.com/?test')) rather than using searchParams if you want to avoid automatic serialization.
    • Streams: To use streams, use the got.stream(url, options) method.
  9. Understand the `ok` property behavior

    main

    The ok property on a Response object indicates whether the request was successful. Its behavior depends on how redirects are handled:

    • Standard behavior: A request is successful if the status code of the final request is 2xx or 3xx.
    • With redirects: If following redirects, a request is successful only when the status code of the final request is 2xx.
    • 304 Responses: 304 Not Modified responses are always considered successful (ok: true).
    • Error Handling: If throwHttpErrors is true, Got will automatically throw an error when response.ok is false.
  10. Use the `hooks` API to intercept request lifecycles

    main

    Got provides a hooks option of type object<string, Function[]> that allows you to intercept various stages of a request's lifecycle. Thrown errors within hooks are automatically converted to RequestError.

    Available hook types include:

    • init: Runs before options normalization.
    • beforeRequest: Runs right before the request is made.
    • beforeRedirect: Runs when a redirect is about to occur.
    • beforeRetry: Runs before a retry attempt.
    • beforeCache: Runs before a response is cached.
    • afterResponse: Runs after a response is received.
    • beforeError: Runs before an error is thrown.
    const instance = got.extend({
    	hooks: {
    		init: [/* ... */],
    		beforeRequest: [/* ... */],
    		// ... other hooks
    	}
    });
  11. Return cached responses using the request function or beforeRequest hook

    main

    You can manually intercept requests to return cached responses (instances of IncomingMessage-like classes) using two different methods:

    1. Overriding the request function: Use got.extend to provide a custom request implementation. This is useful for complete control over the request lifecycle.
    2. Using the beforeRequest hook: Use got.extend to return a cached response within the beforeRequest hook. This is a cleaner way to inject cached data without altering the core request logic.

    Both methods require the returned object to have properties like statusCode, headers, trailers, socket, aborted, complete, httpVersion, httpVersionMinor, and httpVersionMajor.

    ```js
    import {Readable} from 'node:stream';
    import got from 'got';
    
    // Helper to create a mock response
    const getCachedResponse = (url, options) => {
    	const response = new Readable({
    		read() {
    			this.push("Hello, world!");
    			this.push(null);
    		}
    	});
    
    	response.statusCode = 200;
    	response.headers = {};
    	response.trailers = {};
    	response.socket = null;
    	response.aborted = false;
    	response.complete = true;
    	response.httpVersion = '1.1';
    	response.httpVersionMinor = 1;
    	response.httpVersionMajor = 1;
    
    	return response;
    };
    
    // Method 1: Overriding the request function
    const instanceRequest = got.extend({
    	request: (url, options, callback) => {
    		return getCachedResponse(url, options);
    	}
    });
    
    // Method 2: Using the beforeRequest hook
    const instanceHook = got.extend({
    	hooks: {
    		beforeRequest: [
    			options => {
    				return getCachedResponse(options.url, options);
    			}
    		]
    	}
    });
    ```埋
  12. Prevent duplicate requests using handlers

    main

    To prevent multiple identical requests from being sent simultaneously (request collapsing), you can use a custom handler.

    By storing the pending promise of a request in a Map keyed by the URL, subsequent calls for the same URL can return the existing promise instead of initiating a new request. This is particularly useful for preventing race conditions or redundant network traffic when multiple parts of an application request the same resource at once.

    import got from 'got';
    
    const map = new Map();
    
    const instance = got.extend({
    	handlers: [
    		(options, next) => {
    			if (options.isStream) {
    				return next(options);
    			}
    
    			const pending = map.get(options.url.href);
    			if (pending) {
    				return pending;
    			}
    
    			const promise = next(options);
    
    			map.set(options.url.href, promise);
    			promise.finally(() => {
    				map.delete(options.url.href);
    			});
    
    			return promise;
    		}
    	]
    });
    
    const [first, second] = await Promise.all([
    	instance('https://httpbin.org/anything'),
    	instance('https://httpbin.org/anything')
    ]);
    
    console.log(first === second);
    //=> true