ex_cldr

repository·main·Indexed 19 days ago

https://github.com/elixir-cldr/cldr

An Elixir library providing access to the Unicode Common Locale Data Repository (CLDR) to simplify locale-specific formatting and parsing for numbers, currencies, calendars, units, and dates. It utilizes a backend module system to encapsulate configuration and a provider system to include specific functionality via specialized packages such as ex_cldr_numbers, ex_cldr_dates_times, and ex_cldr_units.

Tokens
13.3K
Snippets
54
Records
69
Agent score
66%

What's inside ex_cldr

  1. How providers work in ex_cldr

    main

    Because CLDR data is massive, ex_cldr uses a provider system to allow you to include only the functionality you need (e.g., just numbers, or just dates).

    • To include specific functionality: List the provider modules in the :providers key of your backend module.
    • To include everything installed: If :providers is nil (the default), ex_cldr will attempt to configure all known providers that are present in your deps.
    • To include nothing: Set :providers to [].
    defmodule MyApp.Cldr do
      use Cldr,
        locales: ["en", "zh"],
        default_locale: "en",
        providers: [Cldr.Number, Cldr.List]
    end
  2. Use Locale extensions (U extension)

    main

    The U extension allows you to request specific treatments for CLDR data formats within a locale string. Supported keys vary by library, but ex_cldr_numbers (v2.10+) supports:

    • cf: currency format
    • cu: currency
    • nu: number system

    Example of a complex locale requesting a specific timezone and accounting currency format: en-AU-u-tz-ausyd-cf-account

    # Validating a locale with extensions
    MyApp.Cldr.validate_locale "en-AU-u-tz-ausyd-cf-account"
    # Returns %Cldr.LanguageTag{} with extensions populated
  3. Understand Language Tags and Locale strings

    main

    ex_cldr uses IETF standard locale strings (RFC5646) using hyphens (-) as separators, rather than the POSIX underscore (_) convention.

    Conventions:

    • Language codes: lower-case
    • Territory codes: UPPER-CASE
    • Script names: Capital-case
    • Other subtags: lower-case

    Key Functions:

    • Cldr.known_locale_names(backend): Returns locales known to your specific configuration.
    • Cldr.all_locale_names(): Returns all locales available in the CLDR data repository.
    • Locales are always handled as binary strings in the API, but parsed internally into %Cldr.LanguageTag{} structs.
  4. Define a Cldr backend module

    main

    To use ex_cldr, you must define a module to host your configuration (such as supported locales and the default locale) and serve as your public API. This module is referred to as a backend module. Using a backend module allows different parts of an application to have isolated configurations without interference.

    defmodule MyApp.Cldr do
      @moduledoc """
      Define a backend module that will host our
      Cldr configuration and public API.
    
      Most function calls in Cldr will be calls
      to functions on this module.
      """
      use Cldr,
        locales: ["en", "fr", "zh", "th"],
        default_locale: "en"
    
    end
  5. Configure Maven with a GitHub token

    main

    The CLDR toolchain requires a GitHub token with the read:packages scope to function. You must add this token to your Maven settings.xml file.

    1. Generate a token at https://github.com/settings/tokens with the read:packages scope.
    2. Modify or create $HOME/.m2/settings.xml.
    3. Add a <server> stanza inside the <servers> section using the ID githubicu.
    <settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
     <servers>
      <server>
       <id>githubicu</id>
       <username>your github user id</username>
       <password>your access token</password>
      </server>
     </servers>
    </settings>
  6. Install and download locales

    main

    You can install ex_cldr via GitHub or Hex. The number of locales installed depends on the installation method:

    • GitHub: Cloning the repository installs all 571 locales into your application dependencies.
    • Hex: Only the locales en, en-001, and und are installed by default. Additional locales are downloaded during application compilation when you configure them.
  7. Install ex_cldr and a JSON library

    main

    Add ex_cldr and a compatible JSON library to your mix dependencies. A compatible library must implement encode!/1 and decode!/1. :jason is recommended.

    Note on OTP 27 and Elixir 1.18+: If you are running on OTP 27 or Elixir 1.18 or later, no :json_library configuration is required or recommended, as ex_cldr will automatically use the native JSON libraries included in these versions.

    defp deps do
      [
        {:ex_cldr, "~> 2.37"},
        # Poison or any other compatible json library
        # that implements `encode!/1` and `decode!/1`
        # :jason is recommended
        {:jason, "~> 1.0"}
        # {:poison, "~> 2.1 or ~> 3.0"}
      ]
    end
  8. Configure a Cldr backend module

    main

    To use ex_cldr, you must create a backend module by invoking use Cldr. This module encapsulates CLDR data at compile time to maximize runtime performance. The functions generated in this module form the primary recommended API for the library.

    It is recommended that the module invoking use Cldr does not define any other functions. Additional modules may be generated with names prefixed by your backend module's name, depending on the :providers configured.

    defmodule MyApp.Cldr do
      use Cldr,
        default_locale: "en",
        locales: ["fr", "en", "bs", "si", "ak", "th"],
        add_fallback_locales: false,
        gettext: MyApp.Gettext,
        data_dir: "./priv/cldr",
        otp_app: :my_app,
        precompile_number_formats: ["¤¤#,##0.##"],
        precompile_transliterations: [{:latn, :arab}, {:thai, :latn}],
        providers: [Cldr.Number],
        generate_docs: true,
        force_locale_download: false
    end
  9. Update ex_cldr content with new CLDR data

    main

    Once the JSON data has been generated in the $CLDR_PRODUCTION directory, you must consolidate it into the ex_cldr format.

    Steps:

    1. Update your local ex_cldr repository via git pull.
    2. Run mix cldr.consolidate to consolidate locales into the ex_cldr format.
    3. Regenerate the language_tags.ebin file. Use the DEV=true environment variable to disable stale locale tests during this process.
    cd $EX_CLDR
    git pull
    mix cldr.consolidate
    DEV=true mix cldr.generate_language_tags
  10. Configure Gettext pluralization with CLDR

    main

    To use CLDR plural rules with gettext, create a custom plural forms module that uses Cldr.Gettext.Plural. This module must be passed to the plural_forms option in your Gettext configuration.

    Note: Cldr.Gettext.Plural may return different plural indices than the default Gettext engine, which can cause issues if mixing engines.

    # 1. Define the plural module using your Cldr backend
    defmodule MyApp.Gettext.Plural do
      use Cldr.Gettext.Plural, cldr_backend: MyApp.Cldr
    end
    
    # 2. Configure Gettext to use it
    defmodule MyApp.Gettext do
      use Gettext, plural_forms: MyApp.Gettext.Plural
    end
  11. Initial setup for ex_cldr development

    main

    To develop or maintain ex_cldr, you must set up a local environment that includes the Unicode CLDR repository and the necessary toolchain for data generation.

    Prerequisites:

    • git-lfs: Must be installed before cloning the repositories to handle large objects.
    • Java: At least version 9.0 is required.
    • Maven: Required for the CLDR toolchain.

    Setup Steps:

    1. Install git-lfs via Homebrew (on MacOS).
    2. Clone the Unicode CLDR repository to a local directory.
    3. Install maven via Homebrew (on MacOS).
    4. Configure Maven with a GitHub token (see 'Configure Maven with a GitHub token' for details).
    5. Clone the ex_cldr repository.
    # Install git-lfs
    brew install git-lfs
    git lfs install
    
    # Clone CLDR repo
    export CLDR_REPO="$HOME/development/cldr_repo"
    git clone https://github.com/unicode-org/cldr $CLDR_REPO
    
    # Install Maven
    brew install maven
    
    # Clone ex_cldr repo
    export EX_CLDR="$HOME/Development/ex_cldr"
    git clone https://github.com/elixir-cldr/cldr $EX_CLDR
  12. Migrate from Cldr 1.x to 2.x

    main

    To upgrade from version 1.x to 2.x, follow these steps:

    1. Create a backend module: Follow the configuration instructions to define a custom backend module (e.g., MyApp.Cldr).
    2. Clean up global configuration: Remove any duplicated global configuration in config.exs. Only :default_locale and :json_library are supported in global configuration.
    3. Update Plugs: Update any plugs to use your new configured backend.
    4. Update API calls: Change calls from Cldr.some_function to your backend module, e.g., MyApp.Cldr.some_function. Alternatively, use an alias: alias MyApp.Cldr, as: Cldr.