PanPhon: IPA and Phonological Feature Tools

The diacritic_definitions.yml file is used to control the Diacritic Application Tool by defining how diacritics and modifiers are applied to IPA segments based on phonological features. The file is organized into two main sections: diacritics and combinations.

Diacritics

The diacritics section is a list of rules for applying modifiers to bases. Each rule is a map containing:

• marker: The Unicode diacritic or modifier.
• name: The name of the series derived from applying the diacritic/modifier.
• postion: The position of the diacritic relative to the base (pre or post).
• conditions: A list of conditions (associative arrays of feature specifications) that must be met for the diacritic to be applied.
• exclude: A sequence of segments to be excluded from application, even if they match the conditions.
• content: A map of feature specifications that will be set if the diacritic is applied.

Combinations

The combinations section allows you to group existing diacritic rules. Each rule is a map containing:

• name: The name of the combined category.
• combines: A sequence of names referencing rules defined in the diacritics section.

A schema for validation is provided in diacritic_definitions_schema.yml.

The generate_ipa_all.py script is a CLI tool used to generate a comprehensive list of IPA segments by applying diacritics and combinations to a set of base IPA segments. It uses a CSV file of base segments, a YAML file defining diacritics and combinations, and a YAML file defining the sort order for the output.

Input File Requirements

1. Base IPA Segments (bases): A CSV file where the first column is ipa (the segment string) and subsequent columns are feature names and their values.
2. Diacritic Definitions (--dia): A YAML file containing:
   • diacritics: A list of diacritic objects with name, marker, position (pre or post), conditions (feature requirements), exclude (forms to ignore), and content (feature overrides).
   • combinations: A list of named sequences of diacritics defined by name and combines (a list of diacritic names).
3. Sort Order (--sort-order): A YAML file defining the order in which segments should be sorted in the output. It uses a list of objects with name (feature name) and reverse (boolean).

Output

A CSV file containing all generated segments (bases + modified segments) with the same columns as the input base file.

python panphon/bin/generate_ipa_all.py <bases_csv> <output_csv> --dia <diacritics_yml> --sort-order <sort_order_yml>

The panphon.distance.Distance class provides several methods for computing the distance or similarity between Unicode IPA strings based on their articulatory features.

Available methods:

• levenshtein_distance: Python implementation of Levenshtein distance.
• fast_levenshtein_distance: Faster C implementation of Levenshtein distance.
• dogol_prime_distance: Fast Levenshtein distance after collapsing segments into Dogolpolsky's equivalence classes.
• feature_edit_distance: Edit distance where each feature-edit has cost 1/22 (unspecified to specified costs 1/44).
• hamming_feature_edit_distance: Edit distance where feature-edit costs are 1/22 (including unspecified to specified), and insertions/substitutions cost 1.
• weighted_feature_edit_distance: Edit distance where feature-edit costs are weighted by class and subjective variability.

import panphon.distance

dst = panphon.distance.Distance()
# Example using dogol_prime_distance
dist = dst.dogol_prime_distance(u'pops', u'bobz') # 0

The panphon.distance.Distance class provides several methods for computing the distance between Unicode IPA strings, ranging from standard Levenshtein distance to articulatory feature-based metrics.

Available Methods

• levenshtein_distance(s1, s2): Standard Python implementation of Levenshtein distance.
• fast_levenshtein_distance(s1, s2): A high-performance C implementation of Levenshtein distance.
• dolgo_prime_distance(s1, s2): Fast Levenshtein distance using enhanced Dolgopolsky equivalence classes.
• feature_edit_distance(s1, s2): Edit distance where feature-edit costs are fixed (1/22 per edit; 1/44 for unspecified to specified).
• hamming_feature_edit_distance(s1, s2): Edit distance using Hamming distance for substitutions (cost 1/22 per feature edit; insertions/substitutions cost 1).
• weighted_feature_edit_distance(s1, s2): Edit distance where feature-edit costs are weighted by class and variability.

import panphon.distance

dst = panphon.distance.Distance()

# Example: Dolgo prime distance
dist = dst.dolgo_prime_distance(u'pops', u'bobz')
print(dist)

The panphon.panphon.FeatureTable class provides an API to access the segment-feature relationships encoded in the IPA database. You can use it to check if a segment matches specific features, retrieve individual segments from a string, or get the full set of features for every segment in a word.

Key methods:

• ftr_match(feature_set, segment): Returns True if the segment matches the provided set of features (e.g., set([(u'+', u'syl')])).
• segs(word): Returns a list of Unicode segments for a given word.
• word_fts(word): Returns a list of sets, where each set contains the features for the corresponding segment in the word.

import panphon.panphon as panphon

ft = panphon.FeatureTable()

# Check if a segment matches a feature set
matches = ft.ftr_match(set([(u'+', u'syl')]), u'a') # True

# Get segments from a word
segments = ft.segs(u'pʲãk') # [u'p\u02b2', u'a\u0303', u'k']

# Get feature sets for each segment in a word
feature_sets = ft.word_fts(u'pʲãk')

The panphon.segment.Segment class represents an analyzed IPA segment. It provides a dictionary-like interface for feature manipulation and supports set-like operations for matching and intersection.

Construction

Construct a Segment by providing a list of feature names (to define order) and either a dictionary of values or a formatted string.

• Values are integers: -1 (for -), 0 (for 0), or 1 (for +).
• ftstr format: [+feature1, -feature2]

Operations

• Access/Update: Use segment['feature_name'] to get or set values.
• Match (match or >=): Checks if the segment has a superset of the provided features.
• Intersection (intersection or &): Returns a new Segment containing only the shared features.
• Vectors: Use .numeric() for integer lists or .strings() for ['+', '-', '0'] lists.

from panphon.segment import Segment

# Construction
# Method 1: Using a dictionary
seg = Segment(['syl', 'son', 'cont'], {'syl': -1, 'son': -1, 'cont': 1})

# Method 2: Using a feature string
seg = Segment(['syl', 'son', 'cont'], ftstr='[-syl, -son, +cont]')

# Querying and Updating
print(seg['syl'])      # -1
seg['son'] = 1          # Update feature
seg.update({'cont': -1}) # Bulk update

# Set Operations
# Check if segment matches a pattern
print(seg >= {'son': 1, 'cont': -1}) 

# Find intersection of features
shared = seg & {'syl': -1, 'son': 1}

# Get vector representations
print(seg.numeric())   # [-1, 1, -1]
print(seg.strings())   # ['-', '+', '-']