MoroccoAI’s Post

Last Thursday 𝗠𝗼𝗿𝗼𝗰𝗰𝗼𝗔𝗜 had the pleasure of hosting an insightful webinar on a recent publication at 𝗘𝗠𝗡𝗟𝗣 𝟮𝟬𝟮𝟰 titled “Machine Translation Hallucination Detection for Low- and High-Resource Languages Using Large Language Models,” delivered by Kenza Ily Benkirane Benkirane, AI Lead at Vivanti in London, UK. During the webinar, Kenza began by addressing the issue of hallucinations in machine translation systems, particularly for low-resource languages, and highlighted their impact on critical domains such as healthcare and law. Kenza then presented the methodology of her study, which utilized the Halomi dataset from Meta and employed the Matthews Correlation Coefficient (MCC) as an evaluation metric. She explored two main approaches for hallucination detection: the first involved using Large Language Models (LLMs), where she evaluated eight different models, and the second focused on embedding spaces, where she tested three embedding-based methods. She then presented the results of her study, where both LLMs and embedding spaces significantly outperformed state-of-the-art methods for high-resource languages, even without explicit computations. However, performance dropped notably for low-resource languages, particularly when translating from non-centric English low-resource languages to a language like English. 🎥 If you missed the webinar or would like to rewatch it, check the link to the recording in the comment section below. A big thanks to Kenza Ily Benkirane for sharing her expertise and to all attendees forthe rich discussions ! #MoroccoAI #MachineTranslation #EMNLP2024