So, I've managed to realize a long-standing idea of mine: developing a program algorithm that divides Hebrew text into prosodic segments in accordance with the Masoretic accent marks (te'amim).

A working version has been published online here: https://hebraica-digitalia.org

The original goal was to present Hebrew poetry in "poetic form", i.e. as a set of lines. The starting hypothesis was that the te'amim marks poetic units of text. The task was simply to figure out exactly how they do this (I don't mean the simplest cases, like Atnah and Sof pasuq). For the Hebrew text in UTF-8 encoding, I used the BHSA project (Biblia Hebraica Stuttgartensia Amstelodamensis, a linguistically annotated database of the Tanakh, prepared by the Eep Talstra Centre for Bible and Computer at Vrije Universiteit Amsterdam, based on the Codex Leningradensis and distributed via the Text-Fabric toolkit). Fortunately, this database also contains the cantillation marks.

But before I could write the Python algorithm for dividing the text into cola according to the Masoretic te'amim, a lot of preparation was needed. In many cases, the Unicode characters in the range U+0591–U+05AE cannot yet be considered te'amim marks on their own, since they only become so in a particular context. For example, the character U+05A3 denotes the conjunctive accent Munah, but that same character combined with U+05C0 Paseq becomes a different sign, Legarmeh — and this one is disjunctive! The combination of U+059D [Geresh muqdam] and U+0597 [Revia] is not two accents but one — Revia mugrash — and so on. There are many such cases.

Unfortunately, there are also outright erroneous attributions in Unicode. For instance, the character U+0598֘ is named HEBREW ACCENT ZARQA, which is a clear mistake, since the disjunctive accent Zarqa is actually U+05AE in Unicode, while U+0598 is the conjunctive Zinnorit (this anomaly is officially documented by the Unicode Consortium itself in Technical Note #27, "Known Anomalies in Unicode Character Names," and has deliberately not been fixed for the sake of backward compatibility).

The correct interpretation of an accent is based not on the word but on the token — that is, a word fused into a single form with the article, a preposition, the waw, or the interrogative particle. Words joined by a maqqef also count as tokens. In the vast and fascinating world of BHSA there is no "token" object, so I had to create one myself. I used the same node-and-slot philosophy that underlies Text-Fabric. The new otype "token" was assigned node numbers from 1,446,832 to 1,709,345, making a total of 262,514 nodes. For each node, the constituent word-slots were defined. The main features of a token are tokenQ_utf8_lm and tokenQ_accents_lm (the Q indicates that the qere variant is used instead of the ktiv).

Practice has shown that the idea of displaying Hebrew text as segments defined by the te'amim is fruitful not only for poetry but for prose as well, although prose has its own nuances of colometry.

My algorithm is built on the classical understanding of the Hebrew accents as a hierarchical structure: the position of a boundary is determined by the rank of the accent within the chains of disjunctive accents.

Here is a very brief list of authoritative works on Hebrew accentology:

• William Wickes, A Treatise on the Accentuation of the Three So-called Poetical Books of the Old Testament, Psalms, Proverbs, and Job. Oxford: Clarendon Press, 1881.

• William Wickes, A Treatise on the Accentuation of the Twenty-One So-called Prose Books of the Old Testament. Oxford: Clarendon Press, 1887.

• Mordechai Breuer, Ta'amei ha-Mikra be-21 Sefarim u-ve-Sifrei Emet. Jerusalem, 1982.

• Sung Jin Park, The Fundamentals of Hebrew Accents: Divisions and Exegetical Roles beyond Syntax. Cambridge: Cambridge University Press, 2020.

• Joshua R. Jacobson, Chanting the Hebrew Bible: The Art of Cantillation, 2nd expanded edition. Philadelphia: The Jewish Publication Society, 2017.

The proposed algorithm is not set in stone. It still requires further refinement for the many "special" or "disputed" cases, many of which remain outside my field of view for now.

The next logical step is to compare Masoretic prosody with the syntactic division of the Hebrew text proposed by modern scholars. BHSA contains an extraordinarily rich syntactic annotation of the Hebrew text, which makes this project a truly unique and invaluable resource. What is needed is a clear visual representation of the syntactic data from BHSA — sentences, clauses, phrases, and subphrases (I prefer the term "group") alongside with their attributes, so that it becomes possible to compare this data with the Masoretic te'amim side by side.

Of course, visual comparison alone isn't enough for statistical research, so a syntactic search mechanism also needs to be implemented (ideally with a user-friendly interface). If we're developing a search tool, it's crucial to include support for regular expression searches, a feature that is almost non-existent in digital Hebrew studies today. So, the Colometric Analyzer, a narrowly focused tool, turns out to stimulate a whole host of further ideas and, hopefully, will become a catalyst for a new platform for the study of the Tanakh and Biblical Hebrew.

Sources:

• 26078 zarqa zinor
• UTN #27: Known anomalies in Unicode Character Names
• A treatise on the accentuation of the three so-called poetical books... | Internet Archive
• A treatise on the accentuation of the twenty-one so-called prose books... | Internet Archive
• טעמי המקרא בכ"א ספרים ובספרי אמ"ת | The National Library of Israel
• The Fundamentals of Hebrew Accents | Cambridge University Press
• Chanting the Hebrew Bible, Second, Expanded Edition | JSTOR