Iframely Self-Hosted API

repository·dev·Indexed 23 days ago

https://github.com/itteco/iframely

A self-hosted oEmbed/2 gateway endpoint and tool for generating responsive web embeds and extracting URL metadata. It parses URLs for rich media (YouTube, Instagram, etc.) and metadata (Open Graph tags) using domain parsers and protocol support for oEmbed, Twitter Cards, and microformats. Version 2.5.0 requires Node 14. The package provides two HTTP endpoints: an Iframely format that mimics a page head and an oEmbed format adapter.

Tokens
15.3K
Snippets
25
Records
73
Agent score
82%

What's inside iframely

  1. Overview of Iframely Self-Hosted API

    dev

    Iframely is a self-hosted version of the Iframely APIs and HTML parsers. It takes a URL and returns its metadata, including rich media embeds (layers, posts, slideshows, maps, etc.).

    Key features include:

    • Domain Parsers: Specific parsers for popular publishers like YouTube, Facebook, Instagram, Twitter, SoundCloud, Google Maps, TED, and Twitch.
    • Protocol Support: Coverage of many domains via oEmbed, Open Graph, Twitter Cards, and microformats.
    • Whitelist Management: By default, it fetches a central whitelist from Iframely, but you can replace it with your own custom whitelist file.
    • Node.js Integration: Can be used directly as a Node.js library.
  2. Understand the Iframely Link object structure

    dev

    The links array in the Iframely API contains objects representing different ways to embed or display content. Each link object provides raw data (href, rel, type, media) and, in most cases, the pre-generated responsive HTML code (html).

    Key behaviors:

    • html vs href: Most links provide an html field for responsive embeds. However, for specific embeds like Twitter or Facebook statuses, Iframely may return an html attribute without an href.
    • Missing html: Iframely does not generate an html field for image/* MIME types.
    • promo rel: If a media embed was already attached to the URL from another provider (e.g., a player is already embedded on that page), the rel array will include promo.
    {
      // SRC of embed. The main attribute
      "href": "//coub.com/embed/2pc24rpb",
    
      // functional and technical use cases.
      "rel": ["player", "autoplay", "html5"],
    
      // MIME type. Tells: "embed as iFrame"
      "type": "text/html",
    
      "media": { // Media query. Mostly responsive
        "aspect-ratio": 1.777778
      },
    
      // plus generated or native HTML code:
      "html": "<div ..></div>"
    }
  3. Understand the Iframely API response structure

    dev

    The API response provides a unified view of a web page's metadata and embeddable content.

    Key Response Fields

    • id: The short URL ID (if available) and the canonical URL.
    • url: The canonical URL of the requested content.
    • rel: An array of use cases for the primary embed (e.g., ["player", "ssl"]).
    • html: The responsive embed code for the primary media option.
    • meta: An object containing semantic attributes (e.g., title, description, author, site, keywords) in a unified naming format.
    • links: An object containing arrays of embed options grouped by their functional rel type. Common rel types include:
      • player: Embed widgets (e.g., video players).
      • thumbnail: Image previews.
      • app: Application-specific embeds.
      • image: Direct image links.
      • reader: Article reader views.
      • survey: Survey embeds.
      • icon / logo: Site branding assets.

    Note on Data Types: If a rel group contains only one element, the API wraps it as a single object instead of an array.

    {
        "id": "ACcM3Y",
        "url": "http://coub.com/view/2pc24rpb",
        "rel": ["player", "ssl"],
        "html": "<div ... </div>",
        "meta": {
            "title": "PARADISE BEACH",
            "description": "Ilya Trushin",
            "author": "Ilya Trushin",
            "site": "Coub",
            "canonical": "http://coub.com/view/2pc24rpb"
        },
        "links": {
            "player": [{
                "media": { "aspect-ratio": 1.777778 },
                "href": "//coub.com/embed/2pc24rpb",
                "rel": ["player", "ssl", "html5"],
                "type": "text/html",
                "html": "<div ... </div>"
            }],
            "thumbnail": [{
                "media": { "height": 360, "width": 640 },
                "rel": ["thumbnail"],
                "type": "image",
                "href": "http://cdn1.aka ... med_1381670134_00040.jpg"
            }]
        }
    }
  4. Differences between Self-Hosted and Cloud API

    dev

    When choosing between the self-hosted version and the Iframely Cloud API, be aware of the following limitations in the self-hosted version:

    • No Hosted iFrame Renders: The self-hosted version does not return Iframely.com-powered iFrame renders in the html field. You are responsible for rendering the media using the provided metadata.
    • Missing Cloud Features: Features like per-URL customization, predictive sizing for JS-based embeds, lazy-loading, and type-based media whitelists are exclusive to the Cloud API.
    • Limited Domain Plugins: The open-source version includes core domain parsers but does not receive new publisher updates as frequently as the Cloud version. However, you can extend the self-hosted version with your own private plugins.
  5. How to write an Iframely plugin

    dev

    Plugins are Node.js modules that extend the Iframely engine by providing methods to extract links, metadata, or custom data from a URI.

    A plugin is defined as an exported object containing specific attributes and methods. The engine automatically detects which methods to call based on the 'requirements' (parameters) defined in the function signatures.

    Core Plugin Attributes

    • re: A single RegExp or an array of RegExp objects used to test the page URI. If a domain plugin has this attribute, the engine matches the URI against it to select the specific plugin.
    • mixins: A list of other plugin identifiers (filenames without extensions) to be used alongside a domain plugin.
    • tests: An array of test configurations (URLs, feeds, or objects) used to validate the plugin.
    • lowestPriority: Boolean. If true, the getMeta results from this plugin are only used if no other plugin provides that specific metadata.
    • highestPriority: Boolean. If true, the getMeta results from this plugin will override all other plugins.

    Core Plugin Methods

    • getLink(meta, oembed): Generates a single link object.
    • getLinks(meta, oembed): Generates an array of link objects.
    • getMeta(meta): Creates unified page metadata.
    • getData(meta, oembed, ...): Generates custom data that can be consumed by other methods (getMeta, getLink, or other getData calls).
  6. Configure domain matching and wildcards in the whitelist

    dev

    The whitelist uses top-level keys for domain names. It supports wildcard entries (e.g., *.sub.domain.com). When querying a URL like http://name.sub.domain.com/slug, the system follows this priority algorithm:

    1. Check for an exact match: name.sub.domain.com
    2. Check for a subdomain wildcard: *.sub.domain.com
    3. Check for a top-level wildcard: *.domain.com

    Note on www: www.domain.com and domain.com are treated as identical in most cases. If www.domain.com is not explicitly listed, check domain.com instead of using *.domain.com to avoid potential errors.

  7. Understand the Iframely Whitelist File Format

    dev

    The Iframely whitelist file allows parsers to identify which domains can provide rich media embeds via protocols like Iframely, oEmbed, Twitter Cards, and Open Graph.

    By default, the self-hosted Iframely instance fetches updates from http://iframely.com/qa/whitelist.json. You can override this by downloading the file, customizing it, and uploading it to the /whitelist folder of your server.

    File Naming Convention: To allow the server to update the whitelist without a restart, include a timestamp in the filename: iframely-YYYY-MM-DD-HH-mm-UTC.json (e.g., iframely-2013-08-27-14-18-UTC.json).

  8. Understand the Iframely meta object structure

    dev

    Iframely normalizes various semantic standards (such as oEmbed, Open Graph, Twitter Cards, and Dublin Core) into a unified meta object. This allows you to access consistent naming keys regardless of the source platform's original metadata format. The meta object is returned as part of the API response and contains fields for general information, attribution, statistics, geographic data, and product information.

    "meta": {
    	"title": "PARADISE BEACH",  
    	"description": "Ilya Trushin",
    	"author_url": "http://coub.com/trucoubs",
    	"author": "Ilya Trushin",
    	"site": "Coub",
    	"canonical": "http://coub.com/view/2pc24rpb",
    	"keywords": "living photo, ... , media"        
    }
  9. Make an Iframely API request

    dev

    To retrieve semantics, attribution metadata, and embed links for a specific URL, send an HTTP GET request to the Iframely API endpoint.

    Parameters

    • url (Required): The URL you want to embed. This must be URL-encoded.
    • api_key (Required): Your API key.
    • key (Optional): For enhanced security, you can use the MD5 hash of your API key instead of the raw api_key.
    • group (Optional): For open-source users, add &group=true to the request to ensure the links array is grouped by rel (matching the Cloud API behavior).

    Protocol Choice

    • HTTPS: Use https://iframe.ly/api/iframely if your site uses SSL and you are making calls via JavaScript.
    • HTTP: Generally faster for server-to-server communications as it avoids additional handshakes.
  10. Handle Iframely API error responses

    dev

    Iframely uses standard HTTP status codes to indicate the success or failure of a request. A successful request returns HTTP 200. Any processing error with a URL will return an HTTP 4xx error code.

    In addition to the HTTP status code, the response body contains a JSON object specifying the error status and message:

    {
      "status": "404",
      "error": "Not Found"
    }

    Note that URLs resulting in an error do not count towards your API usage limits or billing. You can also configure your settings at iframely.com/settings to return an error if no media matches your specific requirements.

    {
      "status": "404",
      "error": "Not Found"
    }
  11. Render links Widgets

    dev

    To render widgets from the API response, iterate through the links array and use $.iframely.generateLinkElement(link, data) to create the HTML element for each link.

    To prevent scrollbars in reader rel iframes (specifically type="text/html" widgets like GitHub Gists), call $.iframely.registerIframesIn(container) after rendering. This allows the library to listen for resize messages from the iframe and adjust its size to fit the content dynamically.

    // Iterate through all links.
    data.links.forEach(function(link) {
    
        // Call generator to create html element for a link.
        var $el = $.iframely.generateLinkElement(link, data);
    
        // Add element to body.
        $('body').append($el);
    });
    
    // Call this to handle iframe resizing and prevent scrollbars
    $.iframely.registerIframesIn($('body'));
  12. Add iframely.js to Your App

    dev

    The iframely.js client library facilitates API calls to the /iframely endpoint and renders responsive embed widgets. It requires jQuery as a dependency.

    Include both jQuery and iframely.js in your page's <head> section. You should host iframely.js on your own domain (the same domain where your Iframely Gateway is hosted) for production use.