Install open-graph-scraper
masterInstall the package via npm to start scraping Open Graph, Twitter Card, and other metadata from websites.
npm install open-graph-scraper --saverepository·master·Indexed 20 days ago
https://github.com/jshemas/opengraphscraperA Node.js module for scraping Open Graph, Twitter Card, and other metadata from websites. It supports server-side use, custom meta tag extraction, JSON-LD parsing, and the ability to scrape from either a URL or a raw HTML string. Version 6.12.0 utilizes the Fetch API for requests and provides TypeScript type definitions for results and configuration options.
Install the package via npm to start scraping Open Graph, Twitter Card, and other metadata from websites.
npm install open-graph-scraper --saveThe scraper returns one of two result types depending on whether the request succeeded or failed:
Returned when the fetch and scraping process completes successfully.
error: falsehtml: The raw HTML string of the page.response: The response object from the fetch request.result: An OgObject containing the scraped metadata.Returned when an error occurs during the process.
error: truehtml: undefinedresponse: undefinedresult: An OgObject (may contain error details).export interface SuccessResult {
error: false;
html: string;
response: object;
result: OgObject;
}
export interface ErrorResult {
error: true;
html: undefined;
response: undefined;
result: OgObject;
}The run function returns a Promise that resolves to a SuccessResult or rejects with an ErrorResult.
When scraping is successful, the returned object follows this shape:
error: falseresult: The scraped Open Graph object (ogObject).response: The HTTP response object from the fetch request.html: The raw HTML string of the page.If the scraper encounters an error, it throws an object with this shape:
error: trueresult: An object containing:success: falserequestUrl: The URL that was attempted.error: The error message string.errorDetails: The original error object.response: undefinedhtml: undefinedThe scraper returns one of two result types depending on whether the operation succeeded or failed:
Returned when the scrape is successful.
error: falsehtml: The raw HTML string of the page.response: The response object from the fetch request.result: An OgObject containing the scraped metadata.Returned when the scrape fails (e.g., network error, invalid URL).
error: truehtml: undefinedresponse: undefinedresult: An OgObject (usually containing error details).type ScrapeResult = SuccessResult | ErrorResult;Import open-graph-scraper and call it with an options object containing a url. The function returns a Promise that resolves to an object containing error (boolean), html (string), result (metadata object), and response (Fetch API response).
const ogs = require('open-graph-scraper');
const options = { url: 'http://ogp.me/' };
ogs(options)
.then((data) => {
const { error, html, result, response } = data;
console.log('error:', error); // This returns true or false. True if there was an error. The error itself is inside the result object.
console.log('html:', html); // This contains the HTML of page
console.log('result:', result); // This contains all of the Open Graph results
console.log('response:', response); // This contains response from the Fetch API
})If a site blocks the default undici user agent, you can provide a custom user agent via fetchOptions.headers.
const ogs = require("open-graph-scraper");
const userAgent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36';
ogs({ url: 'https://www.wikipedia.org/', fetchOptions: { headers: { 'user-agent': userAgent } } })
.then((data) => {
const { error, html, result, response } = data;
console.log('error:', error);
console.log('html:', html);
console.log('result:', result);
console.log('response:', response);
})Use the customMetaTags option to extract specific meta tags that are not part of the standard Open Graph or Twitter Card sets. Each object in the array requires a property (the tag name/attribute) and a fieldName (the key in the resulting result.customMetaTags object). The multiple boolean indicates if the tag appears more than once on the page.
const ogs = require('open-graph-scraper');
const options = {
url: 'https://github.com/jshemas/openGraphScraper',
customMetaTags: [{
multiple: false, // is there more than one of these tags on a page (normally this is false)
property: 'hostname', // meta tag name/property attribute
fieldName: 'hostnameMetaTag', // name of the result variable
}],
};
ogs(options)
.then((data) => {
const { result } = data;
console.log('hostnameMetaTag:', result.customMetaTags.hostnameMetaTag); // hostnameMetaTag: github.com
})Use jsonLDOptions to control how errors are handled during JSON-LD parsing.
throwOnJSONParseError: If true, the error will be thrown.logOnJSONParseError: If true, the error will be logged to the console.const ogs = require("open-graph-scraper");
const userAgent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36';
ogs({ url: 'https://www.wikipedia.org/', jsonLDOptions: { throwOnJSONParseError: true } })
.then((data) => {
const { error, html, result, response } = data;
console.log('error:', error);
console.log('html:', html);
console.log('result:', result);
console.log('response:', response);
})Instead of providing a URL, you can pass a raw HTML string via the html option. This is useful if you have already fetched the content or are using open-graph-scraper-lite in a browser environment.
const ogs = require('open-graph-scraper');
const options = {
html: `<html><head>
<link rel="icon" type="image/png" href="https://bar.com/foo.png" />
<meta charset="utf-8" />
<meta property="og:description" name="og:description" content="html description example" />
<meta property="og:image" name="og:image" content="https://www.foo.com/bar.jpg" />
<meta property="og:title" name="og:title" content="foobar" />
<meta property="og:type" name="og:type" content="website" />
</head></html>`
};
ogs(options)
.then((data) => {
const { result } = data;
console.log('result:', result);
})If using TypeScript, you can import types like SuccessResult from open-graph-scraper/types to ensure type safety for the scraper's response.
// example of how to get types
import type { SuccessResult } from 'open-graph-scraper/types';
const example: SuccessResult = {
result: { ogTitle: 'this is a title' },
error: false,
response: {},
html: '<html></html>'
}
// import example
import ogs from 'open-graph-scraper';
const options = { url: 'http://ogp.me/' };
ogs(options)
.then((data) => {
const { error, html, result, response } = data;
console.log('error:', error);
console.log('html:', html);
console.log('result:', result);
console.log('response:', response);
});The ValidatorSettings interface allows you to customize the strictness of the URL validation performed by the scraper using validator.js settings.
Common options include:
allow_protocol_relative_urls: Allows URLs starting with //.protocols: A list of allowed protocols (e.g., ['http', 'https']).require_host: If false, validation passes even if the host is missing.require_protocol: If true, the URL must include a protocol.validate_length: If false, skips string length validation.const urlValidatorSettings: ValidatorSettings = {
allow_fragments: true,
allow_protocol_relative_urls: true,
allow_query_components: true,
allow_trailing_dot: true,
allow_underscores: true,
protocols: ['http', 'https'],
require_host: true,
require_port: false,
require_protocol: true,
require_tld: true,
require_valid_protocol: true,
validate_length: true
};The JSONLDOptions interface allows you to control how the scraper handles JSON-LD data found in the HTML.
throwOnJSONParseError: If true, the scraper will throw an error if JSON-LD parsing fails.logOnJSONParseError: If true, the scraper will log errors to the console instead of throwing.const jsonLDOptions: JSONLDOptions = {
throwOnJSONParseError: false,
logOnJSONParseError: true
};