misaki

Misaki uses a set of 49 phonemes for English, designed as input tokens for neural networks. The phoneme set is divided into shared phonemes (used by both American and British English), American-only phonemes, and British-only phonemes.

Shared Phonemes (41)

• Stress Marks: ˈ (Primary stress), ˌ (Secondary stress).
• IPA Consonants: bdfhjklmnpstvwz (standard), ɡ (hard g), ŋ (ng), ɹ (r), ʃ (sh), ʒ (zh), ð (soft th), θ (hard th).
• Consonant Clusters: ʤ (j/dg), ʧ (ch).
• IPA Vowels: ə (schwa), i, u, ɑ, ɔ, ɛ, ɜ, ɪ, ʊ, ʌ.
• Diphthong Vowels: A (eh/eɪ), I (eye/aɪ), W (ow/aʊ), Y (oy/ɔɪ).
• Custom Vowel: ᵊ (small/muted schwa).

American-only (4)

• Vowels: æ (ash), O (oh/oʊ), ᵻ (intermediate between ə and ɪ).
• Consonant: ɾ (flap/between t and d).

British-only (4)

• Vowels: a (ash), Q (oh/əʊ), ɒ (on).
• Other: ː (vowel extender).

The output of JAG2P consists of two parts:

1. Phoneme and Pitch String: A concatenated string where phonemes are followed by pitch markers. Pitch markers use:

   • _ for low pitch (accent 0)
   • ^ for high pitch (accent 3)
   • - for rising/falling pitch (accent 1 or 2)
   • j for unknown tokens or whitespace.

2. MToken List: A list of MToken objects. Each token contains:

   • text: The original surface text.
   • tag: The part-of-speech tag.
   • whitespace: String indicating trailing whitespace.
   • phonemes: The phonetic representation of the token.
   • _: An underscore object containing detailed metadata:
     • pron: The raw pronunciation string.
     • acc: The accent nucleus position.
     • mora_size: Number of moras in the word.
     • chain_flag: Boolean indicating if the word is part of a pitch-accent chain.
     • moras: List of moras.
     • accents: List of accent levels per mora.
     • pitch: The pitch contour string for this specific token.

The G2P engine follows a specific lifecycle to ensure accurate phoneme generation:

1. Preprocessing: Cleans text and extracts features (like stress or numeric flags) from Markdown-style links.
2. Tokenization: Uses spaCy to split text into initial tokens.
3. Folding: Merges adjacent tokens that belong together (e.g., parts of a single word).
4. Retokenization: Refines token boundaries, handles sub-tokenization (splitting words into components), and identifies currencies or punctuation.
5. Lexicon Lookup: Iterates through tokens (often in reverse) to find phonemes in the Lexicon. It uses TokenContext to handle phonetic variations based on surrounding vowels or specific words (like 'to' or 'the').
6. Fallback: If the lexicon fails, the FallbackNetwork (BART model) is invoked.
7. Resolution: Adjusts stress patterns across tokens to ensure natural prosody.
8. Finalization: Merges all tokens into a single phoneme string.

To convert phonemes generated by espeak (via phonemizer) into Misaki format, use the from_espeak function. This function handles the mapping of espeak symbols to Misaki's specific token set and accounts for regional differences (British vs. American).

Note: The conversion logic relies on a specific mapping of espeak strings to Misaki tokens. When using phonemizer.backend.EspeakBackend, ensure with_stress=True and tie='^' are set to match the expected input format.

import re
import phonemizer

FROM_ESPEAKS = sorted({'̃':'','a^ɪ':'I','a^ʊ':'W','d^ʒ':'ʤ','e':'A','e^ɪ':'A','r':'ɹ','t^ʃ':'ʧ','x':'k','ç':'k','ɐ':'ə','ɔ^ɪ':'Y','ə^l':'ᵊl','ɚ':'əɹ','ɬ':'l','ʔ':'t','ʔn':'tᵊn','ʔˌn\u0329':'tᵊn','ʲ':'','ʲO':'jO','ʲQ':'jQ'}.items(), key=lambda kv: -len(kv[0]))

def from_espeak(ps, british):
    for old, new in FROM_ESPEAKS:
        ps = ps.replace(old, new)
    ps = re.sub(r'(\S)\u0329', r'ᵊ\1', ps).replace(chr(809), '')
    if british:
        ps = ps.replace('e^ə', 'ɛː')
        ps = ps.replace('iə', 'ɪə')
        ps = ps.replace('ə^ʊ', 'Q')
    else:
        ps = ps.replace('o^ʊ', 'O')
        ps = ps.replace('ɜːɹ', 'əɹ')
        ps = ps.replace('ɜː', 'əɹ')
        ps = ps.replace('ɪə', 'iə')
        ps = ps.replace('ː', '')
    return ps.replace('^', '')

# Usage Example
british = False
espeak = phonemizer.backend.EspeakBackend(
    language=f"en-{'gb' if british else 'us'}",
    preserve_punctuation=True, with_stress=True, tie='^'
)
text = 'merchantship'
espeak_ps = espeak.phonemize([text])
espeak_ps = espeak_ps[0].strip() if espeak_ps else ''
ps = from_espeak(espeak_ps, british)
print(ps) # Output for US: mˈɜɹʧəntʃˌɪp

The zh_normalization module provides support for converting non-standard Chinese text (numbers, symbols, and special formats) into their spoken/normalized forms. This is useful for text-to-speech (TTS) frontends to ensure numbers and symbols are read correctly.

Supported normalization types include:

• Serial numbers: Converts digit sequences into spoken numbers (e.g., 27149 $\rightarrow$ 二七一四九).
• Cardinal numbers: Converts decimals and integers into spoken Chinese (e.g., 324.75 $\rightarrow$ 三百二十四点七五).
• Numeric ranges: Converts ranges using ~ or - into spoken ranges (e.g., 12~23 $\rightarrow$ 十二到二十三).
• Dates: Converts year/month/day formats into spoken dates (e.g., 1995年3月1日 $\rightarrow$ 一九九五年三月一日).
• Time: Converts clock times into spoken time (e.g., 12:05 $\rightarrow$ 十二点零五分).
• Temperature: Converts temperature readings (e.g., -10°C $\rightarrow$ 零下十度).
• Fractions: Converts slash-separated numbers into spoken fractions (e.g., 7/12 $\rightarrow$ 十二分之七).
• Percentages: Converts % symbols into spoken percentages (e.g., 62% $\rightarrow$ 百分之六十二).
• Money: Converts currency amounts into spoken formats (e.g., 34.5元 $\rightarrow$ 三十四点五元).
• Telephone numbers: Converts landline and mobile formats into spoken digit sequences (e.g., +86 185... $\rightarrow$ 八六一八五...).

|NSW type|raw|normalized|
|:--|:-|:-|
|serial number|电影中梁朝伟扮演的陈永仁的编号27149|电影中梁朝伟扮演的陈永仁的编号二七一四九|
|cardinal|这块黄金重达324.75克<br>我们班的最高总分为583分|这块黄金重达三百二十四点七五克<br>我们班的最高总分为五百八十三分|
|numeric range |12\~23<br>-1.5\~2|十二到二十三<br>负一点五到二|
|date|她出生于86年8月18日，她弟弟出生于1995年3月1日|她出生于八六年八月十八日， 她弟弟出生于一九九五年三月一日|
|time|等会请在12:05请通知我|等会请在十二点零五分请通知我|
|temperature|今天的最低气温达到-10°C|今天的最低气温达到零下十度|
|fraction|现场有7/12的观众投出了赞成票|现场有十二分之七的观众投出了赞成票|
|percentage|明天有62％的概率降雨|明天有百分之六十二的概率降雨|
|money|随便来几个价格12块5，34.5元，20.1万|随便来几个价格十二块五，三十四点五元，二十点一万|
|telephone|这是固话0421-33441122<br>这是手机+86 18544139121|这是固话零四二一三三四四一一二二<br>这是手机八六一八五四四一三九一二一|