Teaching a Machine to Say Namárië

In my very first Quenya post (so proud of myself!), I listed QuenyaTTS among the tools worth knowing: a text-to-speech system for Quenya built by Annie Zhou for her 2024 master's thesis at the University of Groningen.

I found this project quite interesting and thought that perhaps I could improve it by trying a few different techniques. First, I mined some issues of Parma Eldalamberon (published by the Elvish Linguistic Fellowship) for pronunciation rules that Annie's code might be missing. Second, I tried to extract the phonetics from a recording of Tolkien himself reciting Namárië, published by the Romanian Tolkien Society.

This post is about what changed, and about what happened when I put the synthetic voice next to Tolkien's.

How Annie's system works

A text-to-speech system like this one works in two steps:

1. From text to sounds. A set of rules turns written Quenya into phonetic symbols (IPA). Elen síla lúmenn' omentielvo becomes ˈɛlɛn sˈiːla lˈuːmɛnn ɔmɛntiˈɛlvɔ: which sounds to make and where the stress falls.
2. From sounds to audio. A neural model, IMS-Toucan from the University of Stuttgart, turns those symbols into a voice.

What changed in Annie's code

I used Claude to find gaps in how Annie's code converts Quenya into the International Phonetic Alphabet. The rules were rewritten and checked against Appendix E of The Lord of the Rings, Eldamo, several published pronunciation guides, and Tolkien's own notes in Parma Eldalamberon. Some examples: hr and hl are read as plain r and l, w becomes v only at the start of a word, archaic spellings such as þ are read with their Third Age sound, and stress follows the Appendix E rule (never the last syllable; the second-to-last if it is heavy, otherwise the third-to-last). Every rule now has a test.

Annie's original project contained a modified copy of an older Toucan. Now the Quenya-TTS repository holds only the Quenya parts, and Toucan comes in unchanged, at a fixed version. Quenya is given to Toucan as phonetic symbols, and since Toucan has never heard Quenya, it borrows the settings of Finnish, Tolkien's own inspiration for the language.

Toucan starts speaking at the very first moment of the file, so the first syllable came out clipped. The tool now adds a short silent pause before the first sound. A side effect was that the model filled that pause with a little noise, heard as an extra k or p before the first word, so that pause is now muted.

Comparing with Tolkien

Tolkien recorded himself reciting Namárië. I wrote a small script that compares any recording with our output, line by line:

• A speech recogniser for many languages (wav2vec2 XLSR) writes down which sounds it hears in both recordings. Comparing that with what our rules ask for gives a phone error rate: the lower, the closer to the rules.
• Both melodies are drawn on top of each other, in semitones.

There are two caveats. The recogniser is not perfect; for example, it hears many open e sounds as closed ones. And Tolkien's tape is old and noisy. So the numbers are a guide, not a grade.

Once Tolkien's recording was correctly cut into the lines of Namárië, I ran the script three times, each round fixing what the previous one showed:

Tolkien's own recording scores 33% against our rules, but listening to it, you can tell that the recogniser misses some of his phonemes. Manual corrections could probably improve that rate, but the purpose of this exercise was to find big pronunciation gaps to fix in Annie's code.

What I learned

The rules agree with Tolkien. Where the recogniser hears Tolkien clearly, he says what the rules predict. The one interesting difference is ly, as in ilyë and hiruvalyë: the rules use a single sound, but in Tolkien's voice the recogniser hears two phonemes, l followed by y. The sources leave both readings open (in Tengwar, ly is one letter with two dots below), so I kept the single sound.

The remaining errors are probably the model's. I don't have access to Annie's trained model, and the few Quenya recordings available aren't enough to train a new one (more on that below). The main remaining issues are:

• the model can't make the soft l, so ilyë comes out as "ihe";
• a final r is weak and gets heard as l (Valimar, tellumar), where it should be trilled;
• the d of undulávë drifts toward b, between two rounded u.

Not enough voice

To see how far existing recordings could take the model, I fine-tuned Toucan on Kaggle with about eight minutes of Quenya readings, for private testing only. After 5,000 training steps, the model sounded better on the sentences it had heard and worse on the ones it hadn't: a textbook case of overfitting. Better rules can't fix that. What the model needs is hours of clean Quenya, ideally from a single voice.

What comes next

My own voice first. I wrote a recording script of 2,500 lines, about 90 minutes of speech split into 25 sessions: first the lines of lesson 1 of Ma Quetilyë Quenya?, then attested phrases, then sentences built from words attested in Eldamo. I record it at home with a small tool that can also play the synthetic version of each line. The same recogniser I used on Tolkien now checks my own takes, and it is humbling: it catches me on nw, ny, au and an initial qu more often than I would like.

Then volunteers. A small website is almost ready, where volunteers can read sessions of the same script in their browser. A reviewer listens to each session and accepts it, or asks for single lines to be read again. A voice is used for training only once it has enough accepted audio.

I changed my mind about one thing since I first wrote this post: the recordings will not be published under an open licence. They are used only to train the model. Every volunteer can download their own recordings and do whatever they like with them, and can export or erase their data at any time, as the GDPR requires. The website, the storage and the training all run on servers in the European Union, in Luxembourg and Portugal.

A voice that is nobody's. A model trained on someone's voice will sound like them. So the model I publish will speak with a blended voice that belongs to no single volunteer. Every audio file it produces will carry an inaudible watermark marking it as synthetic, as the EU AI Act asks, and its licence will forbid using it to imitate a real person.

If you would like to lend your voice, see the volunteer voices issue. I will write a follow-up post once the model has learned to speak with my voice.

Please let me know if you have any feedback.

Namárië | See you | Bis geschwënn | Até mais | À bientôt