DOCSEARCHbyAlgolia
Submit Docs

Open Scriptures Hebrew Bible (OSHB) Morphology

repository·master·Indexed 42 days ago

https://github.com/openscriptures/morphhb

A linguistically analyzed version of the Hebrew Bible featuring lemma and morphology attributes in OSIS XML format. The project includes tools for morphology parsing, a JavaScript parser (MorphologyParser.js), and utilities to convert XML data to JSON via Perl or Python. It also provides OSHB Read for text inspection and OshbVerse for visualizing cantillation structure.

Tokens
3K
Snippets
3
Records
15
Agent score
54%

What's inside morphhb

  1. Overview of OSHB Read

    master

    OSHB Read is an application for reading the Open Scriptures Hebrew Bible directly from the source text. It provides access to the latest available version, including completed morphology from the first pass.

    Key features include:

    • Book and Chapter Navigation: Select a book and then a chapter to view the entire text.
    • Verse Selection: Quickly zero in on specific passages; the verse selector highlights and scrolls the selected verse into view.
    • Word Segmentation: Words are divided into prefixes, main words, and suffixes.
    • Morphology Inspection: Hovering over a word part shows a popup with the lemma (compatible with the OSHB Hebrew Lexicon), the morphology code, and the word's accent. Clicking a part of the word displays a readable rendering of the morphology code in a box below the text.

  2. Overview of the Open Scriptures Hebrew Bible (Miqra according to the Mesorah)

    master

    The Open Scriptures Hebrew Bible (OSIS version) provides a structured markup of the Hebrew Bible text compiled by Seth (Avi) Kadish. This version preserves the traditional Hebrew Bible structure and is organized into three primary divisions:

    1. The Torah: The five books of Moses, marked up in sections corresponding to weekly Torah readings. Chapter and verse divisions are implemented as milestones.
    2. The Prophets: Includes Joshua, Judges, Samuel (1 & 2), Kings (1 & 2), Isaiah, Jeremiah, Ezekiel, and the Twelve (the minor prophets).
    3. The Writings: Includes the poetic books (Psalms, Proverbs, Job), the Scrolls (Song of Songs, Ruth, Lamentations, Ecclesiastes, Esther), and Daniel, Ezra (including Nehemiah), and Chronicles (1 & 2).

    Data Structure Notes:

    • Titles: Hebrew titles are preserved within <title> elements.
    • Numbering: Hebrew chapter and verse numbers are maintained using the n attribute within the opening <chapter> and <verse> tags.

  3. General Parsing Principles

    master

    When parsing the OSHB text, follow these core principles:

    1. Gender Ambiguity: Nouns and cardinal numbers that can be both feminine and masculine should be parsed as both, unless context dictates a specific gender.
    2. Gentilic Nouns: All names of peoples (tribes, nations, clans, etc.) must be parsed as Noun-gentilic (with the rest of the morphology following the specific word).
    3. Morphological Symmetry: The number of morphological parts must match the number of word parts (the number of slashes must be identical).
    4. Completeness: Complete all parsing information for each word. Exceptions: Proper names (only need Np), prepositions without a definite article, and paragogic/directional suffixes.
    5. Person Markers: Do not include an x in the middle of a parsing except for non-personal pronouns (demonstrative, interrogative, indefinite, and relative) which do not have a person.

  4. Avoid NFC normalization for Hebrew text

    master
    When working with OSHB data, you should avoid NFC normalization. Following the guidance in the SBL Hebrew User Manual, normalization can cause issues with character sequences that are semantically identical but encoded differently. To maintain data integrity, use the encoding provided without applying NFC normalization.

  5. Understand the Cantillation Structure attributes

    master

    The OpenScriptures Hebrew Bible includes structure attributes for words with disjunctive accents. These attributes are stored in the n attribute:

    • Major divisions: Indicated by single numbers (e.g., 1, 2).
    • Lower divisions: Indicated by decimal numbers following the major division (e.g., 1.1, 1.2).

    This hierarchical numbering allows for a nested representation of the verse's cantillation breakdown.

  6. Understand Open Scriptures Hebrew Bible (OSHB) word tag attributes

    master

    The OSHB project provides Hebrew Bible text marked up in OSIS XML. Each word tag in the XML contains three specific attributes used for linguistic analysis and data association:

    • lemma: The lemma of the word (e.g., c/m/6529).
    • morph: The morphological information (e.g., HC/R/Ncmsc).
    • id: A unique, immutable identifier for every word (e.g., 018xz). The first two digits represent the KJV book number, and the last three are random. This ID is used to associate data with specific words and to facilitate textual criticism by aligning words across different codices.

  7. Transform data to JSON using Python and Docker

    master

    A Python script is available to transform the XML data into JSON. It supports arguments similar to the Perl script, with the addition of --splitByBook, which creates a separate JSON file for each book.

    To run the transformation, build and execute the script within a Docker container using the provided Dockerfile.

    docker build . -t local/morphhb && docker run -it -v `pwd`:/var/app local/morphhb

  8. Visualize verse structure with OshbVerse

    master

    The OshbVerse demonstration provides two methods for visualizing cantillation divisions in code. Both methods support two visual modes: a "binary tree" form (default, using colors to discern accent roles) and a "clean" form.

    Layout Options

    1. Horizontal Layout (Recommended)

      • Markup Component: VerseMarkupHorizontal.js
      • CSS Requirement: PassageHorizontal.css
      • Note: This view is more suggestive of the true breakdown and is the default setup.

    2. Vertical Layout

      • Markup Component: VerseMarkupVertical.js
      • CSS Requirement: PassageVertical.css
      • Note: This follows the layout of the older demo application.

  9. Phase 1 Parsing Workflow

    master

    The parsing process involves completing chapters of the Hebrew Bible that have been partially auto-parsed. When using the parsing tool, words are categorized by color/highlighting:

    • Green (Verified): These are already confirmed. Skip these.
    • Brown (Done): These are guessed by the auto-parser but not yet verified. Review these for errors (e.g., incorrect gender or construct status) and correct them. Correcting a brown word will mark it as verified during the next auto-parsing run.
    • No Highlighting (Unparsed): These words have no parsing. Parse them to the best of your ability, skipping only if you are unsure of the morphology.

  10. Advanced Parsing and Verification for Editors

    master

    If you have editor status in the parsing widget, you can verify existing parsings.

    Handling Conflicts and Verification

    • Qere/Ketiv: These often cause conflicts in the etcbc. Double-check them manually; a conflict does not always mean an error.
    • Grammatical Specificity: Some words are confirmed but not verified because the etcbc lacks specific granularity (e.g., distinguishing jussives/cohortatives from imperfects, or perfect from sequential-imperfect). Verify these manually if they look correct.
    • Removed Parsings: If parsing was removed, it is usually due to a mismatch in the number of parts or an invalid gender-both assignment. Add the correct parsing and verify.
    • Construct/Absolute Conflicts: Many noun conflicts arise from incorrect construct/absolute status. Correct and verify.

    Using the Parsing Widget

    1. Click the Edit button to reveal the parsing codes underneath the words.
    2. To verify: Select the correct parsing from the list and click the verify button.
    3. If the required parsing is not in the list, you must first apply the new parsing to add it to the list.

  12. Generate JSON output using the Perl script

    master

    The morphhbXML-to-JSON.pl Perl script converts the XML data into a lightweight JSON format. The resulting JSON structure is organized as follows:

    • Root Object: Keys are book names.
    • Book Value: An array of chapters.
    • Chapter Value: An array of verses.
    • Verse Value: An array of words.
    • Word Value: An array containing [ 'wordString', 'lemma', 'morphology' ].

    Script Options

    • --stripPointing: Removes all non-letter characters from the Hebrew words.
    • --removeLemmaTypes: Removes additional type letters from Strong's numbers that are not in the dictionary.
    • --prefixLemmasWithH: Adds an H prefix to lemmas to differentiate Hebrew Strong's numbers from Greek ones.
    • --remapVerses: Maps Hebrew versification to English versification.

    Note: A pre-built JavaScript module of this JSON data is available on npm as morphhb.

    perl morphhbXML-to-JSON.pl --stripPointing --removeLemmaTypes --prefixLemmasWithH --remapVerses