DOCSEARCHbyAlgolia
Submit Docs

Emojibase

repository·master·Indexed 20 days ago

https://github.com/milesj/emojibase

A comprehensive ecosystem of specification-compliant emoji datasets, regex patterns, and utilities based on official Unicode standards. It includes packages such as emojibase-data for localized JSON datasets, emojibase-regex for matching emojis and shortcodes, and emojibase-generator for data generation. Emojibase supports the latest emoji standards (e.g., Emoji 17, Unicode 17, and CLDR 48) and provides tools to fetch data via CDN or local installation.

Tokens
13.9K
Snippets
50
Records
68
Agent score
70%

What's inside emojibase

  1. Overview of Emojibase

    master

    Emojibase is a collection of lightweight, up-to-date, and specification-compliant emoji datasets. It provides pre-generated JSON datasets, regex patterns, and more, designed to be used in various applications.

    Key features include:

    • Support for the latest emoji standards (e.g., Emoji 17, Unicode 17, and CLDR 48).
    • Data built directly from official Unicode emoji source files.
    • Compliance with Unicode Technical Standard #51.
    • Localization based on Unicode Technical Standard #35.

  2. Introduction to Emojibase

    master

    Emojibase provides lightweight, up-to-date, and specification-compliant emoji datasets and tools. It is designed to provide pre-generated JSON datasets, regex patterns, and other emoji-related utilities for developers.

    Key features include:

    • Up-to-date support: Includes the latest Emoji 17, Unicode 17, and CLDR 48 release versions.
    • Official Data Source: Built directly from official Unicode emoji data files.
    • Standard Compliance: Based on Unicode Technical Standard #51 (UTS #51).
    • Localization: Provides localization following Unicode Technical Standard #35 (UTS #35).

  3. Handle multi-person combination skin tones (v12.0)

    master

    In Emojibase v12.0, multi-person emojis (e.g., couples) that support individual skin tone changes are handled via a skins array within the parent emoji object.

    Key behaviors to note:

    • The skins array may contain more than the standard 5 skin tone variants (up to 15 in some cases).
    • The tone property within a skin variant can be a single integer (e.g., 1) or an array of integers (e.g., [2, 1]) representing the skin tones of each person in the combination.
    • The hexcode of a skin variant may not match the parent emoji's hexcode pattern.
    {
  "label": "people holding hands",
  "hexcode": "1F9D1-200D-1F91D-200D-1F9D1",
  "skins": [
    {
      "label": "people holding hands: light skin tone",
      "hexcode": "1F9D1-1F3FB-200D-1F91D-200D-1F9D1-1F3FB",
      "tone": 1
    },
    {
      "label": "people holding hands: medium-light skin tone, light skin tone",
      "hexcode": "1F9D1-1F3FC-200D-1F91D-200D-1F9D1-1F3FB",
      "tone": [2, 1]
    }
    // ...
  ]
}

  4. Handle separate and combination genders (v11.0)

    master

    Emojibase v11.0 manages gendered emojis using two distinct patterns:

    Separate Genders

    Emojis like 'person' have specific 'man' and 'woman' variants. These variants always refer back to the neutral 'people' variant through a shared hexcode prefix.

    Combination Genders

    Single emojis that support multiple gender combinations (e.g., 'kiss: woman, man') are represented as unique emojis with their own unique hexcodes. Unlike separate genders, these combination variants do not share a hexcode with the neutral 'people' emoji.

    /* Separate Genders: Shared hexcode prefix */
{
  "label": "person in steamy room",
  "hexcode": "1F9D6"
},
{
  "label": "man in steamy room",
  "hexcode": "1F9D6-200D-2642-FE0F"
}

/* Combination Genders: Unique hexcodes */
{
  "label": "kiss",
  "hexcode": "1F48F"
},
{
  "label": "kiss: woman, man",
  "hexcode": "1F469-200D-2764-FE0F-200D-1F48B-200D-1F468"
}

  5. Understand merged variation selectors (v12.0)

    master

    To prevent data duplication, Emojibase v12.0 merges properties from emoji variation selector (FE0F) siblings into the base emoji (the non-variation selector sibling).

    When consuming the data:

    • The primary hexcode will be the version without the variation selector.
    • The emoji property will always include the emoji variation selector (FE0F).
    • The text property will always include the text variation selector (FE0E).
    {
  "label": "copyright",
  "hexcode": "00A9",
  "emoji": "©️", // 00A9-FE0F
  "text": "©︎" // 00A9-FE0E
}

  6. Understand the different types of emoji datasets

    master

    Emojibase datasets are categorized into three groups based on their purpose:

    1. Localized Data: Datasets containing emoji information with localization provided by CLDR. These return an array of emoji objects. Use data.json for full data or compact.json for a reduced set.
    2. Versioned Data: Datasets that map emoji or Unicode release versions to arrays of emoji hexcodes.
      • emojibase-data/versions/emoji.json: Grouped by emoji version.
      • emojibase-data/versions/unicode.json: Grouped by Unicode version.
    3. Metadata: Specialized datasets for specific use cases:
      • emojibase-data/meta/groups.json: Non-localized groups, subgroups, and hierarchy.
      • emojibase-data/meta/hexcodes.json: Map of hexcodes to their qualification status (fully, minimally, or unqualified).
      • emojibase-data/meta/unicode.json: Array of all emoji Unicode characters.
      • emojibase-data/meta/unicode-names.json: Map of hexcodes to official Unicode names.
    // Localized data example
import emojis from 'emojibase-data/<locale>/data.json';
import compactEmojis from 'emojibase-data/<locale>/compact.json';
import groupsSubgroups from 'emojibase-data/<locale>/messages.json';

// Versioned data example
import unicodeVersions from 'emojibase-data/versions/unicode.json';

// Metadata example
import { groups, subgroups, hierarchy } from 'emojibase-data/meta/groups.json';

  7. Important caveats when using shortcodes

    master

    When implementing shortcodes in your application, be aware of these limitations:

    • Non-standardization: Shortcode names and terminology vary significantly between presets.
    • Missing Data: Some emojis may not have shortcodes defined in certain presets.
    • Conflicts: Using multiple presets simultaneously may result in duplicate or conflicting shortcodes.
    • Version Lag: Presets may not immediately support the latest emoji or Unicode specifications.

  8. How Emojibase translations work

    master

    Emojibase uses GNU gettext .po files for localization. Translations are organized into two primary types of files:

    1. messages.po: Contains localized strings for emoji groups, sub-groups, keywords, and other emoji-related messages.
    2. shortcodes.po: Contains translations for Emojibase preset shortcodes for non-modification emoji characters.

    Note: Do not edit .po files manually. Use a dedicated tool like Poedit to make changes, then commit the files and submit a pull request.

  9. How emoticons are managed in Emojibase

    master

    Emojibase manages emoticons (e.g., :), =], :}) using a centralized set of naming guidelines and a permutation generation system. Instead of defining every possible variation manually, a single emoticon pattern is defined per emoji. The system then generates the various permutations (varying eyes like : or =, and mouths like ), ], or }) automatically. This approach ensures data integrity, reduces maintenance overhead, and keeps the overall data footprint small.

    Note: Currently, only western-styled emoticons are supported.

  10. Understand Emojibase shortcode presets

    master

    Shortcodes (e.g., :cat:) are community-driven words used to represent emojis. Because there is no official Unicode standard for shortcodes, Emojibase provides several presets. Each preset is a distinct collection of shortcodes mapping to emoji hexcodes, tailored for specific platforms or standards.

    When choosing a preset, consider:

    • Platform compatibility: Use github, slack, discord, or joypixels to match specific services.
    • Standardization: Use emojibase for a curated, effort toward an
    // 😁 beaming face with smiling eyes
:beaming_face_with_smiling_eyes: // English (cldr)
:ni_yatto_warau: // Japanese (cldr)
:미소_짓는_눈으로_웃는_얼굴: // Korean (cldr-native)

  11. Generate data files for a new locale

    master

    Once translations are complete, run the generator to create the data files. This process may take some time and outputs significant log information to the console. Monitor the logs for any warnings or errors.

    If successful, a packages/data/<locale> directory will be created. You must then manually create the following TypeScript declaration files (you can copy these .d.ts files from an existing locale like English en):

    • packages/data/<locale>/compact.json.d.ts
    • packages/data/<locale>/data.json.d.ts
    • packages/data/<locale>/messages.json.d.ts
    • packages/data/<locale>/shortcodes/<preset>.json.d.ts
    yarn run generate