LOTRO: Project Questa Update

Project Questa Logo

I began work on Project Questa on the 27th of September, because I couldn’t get Khazad Voice TTS to work on my PC. Since then I’ve spent several days a week working on my AI narrator utility and have made steady progress. Last time I wrote about the project the production pipeline ran as follows. DXCam, OpenCV, Windows OCR and Microsoft Speech API (SAPI). Over time, I determined that Tesseract was a better OCR and that Kokoro TTS produced superior narration. However, in the last week I found that constant fine tuning across different aspects of the production process was not really producing any significant improvements. The main problems were capturing text from dialogue windows that required scrolling, as well as overall latency in producing narration. A delay over 5 seconds is too long in my assessment. Hence I decided to fundamentally change the overall approach to dialogue capture.

Previously, all builds required OCR to read the dialogue accurately enough for Kokoro to narrate it. Occasionally, the capital letter “I” would be mistaken for the number “1” and rogue characters would appear at the end of sentences. This was due to the stylised font theat LOTRO uses. Narration delivery times would also vary and I had trouble trying to identify why this was happening because often there will be no discernable difference between the dialogue bestowal windows and how they presented text. Hence, on the latest build I decided to switch to using OCR to cross reference a percentage of captured text against a database of LOTRO quest dialogue extracted from the game. This data is held in a simple 94MB text file. This revision allows Questa to retrieve the original game text and send it to Kokoro. The response time is faster and the improved process removes a major source of narration errors.

There are a few minor issues at present. I encountered a PCM queue drain during a longer passage of text. One narration, dialogue from a Bounty Quest, took 6.19 seconds to start despite an exceptionally fast database lookup. I need to determine why audio generation or buffering occasionally interrupts delivery. Ideally, Kokoro should begin speaking promptly while preparing sufficient subsequent audio to prevent gaps. It is important that I preserve the more natural, expressive paragraph delivery that is currently being produced, rather than take the path of least resistance by dividing everything into isolated sentences for convenience. But I am confident that these issues can be rectified. At present Questa is in its seventh revision. Once I have dealt with the above issues, I will be able to focus on the next major stage of development.

Questa at present determines quest bestowal dialogue by comparing captured text with a database. I need to determine if that data, held in a simple text file, contains any information regarding the identity of the NPC speaking. If that is the case, then that data can then be used to determine which voice Kokoro uses for the narration. If that data is not held or easily accessible, then I will explore the viability of using screen-based recognition using names, portraits or other visual information. At present, Kokoro has 8 default voices that have UK based accents. I am currently using one named “George” for testing purposes, because it has a senior tone to it and a degree of gravitas. In the long term, I would like Questa users to be able to add custom voices of their own choosing when using the utility.

Aragorn Quest Bestowal Dialogue
George Kokoro

At present I cannot realistically put a timeline or an ETA on when Project Questa will be completed. This is a one man project that I’m undertaking in between my regular writing schedule and real world obligations. However, I do feel that good progress has been made over the last 4 weeks and the recent revision from complete OCR of captured text, to dialogue retrieval from a database was a major turning point. I also do not see this project as being in competition with any similar undertakings that are currently in development. Everything used in to create Project Questa is open source and I shall make the final build available on GitHub upon completion. Ultimately I just want to see a reliable and robust, text-to-speech utility developed and made available to the LOTRO community. I have seen similar things available for other MMORPGs such as WoW, packaged as a straightforward add-on. I see no reason why LOTRO shouldn’t have the same.

Roger Edwards
Writer & editor of Contains Moderate Peril. A website about gaming, genre movies & cult TV. Co-host of the Burton & Scrooge podcast.
http://containsmoderateperil.com
Next
Next

Remote Controls