[GALA Valencia 2024] Quality 360°: Uber’s Use Case on Applied LLMs to Translation Evaluation
Translation quality is shifting to new scores and metrics with the adoption of LLMs. Still, there is much manual effort in the industry, both at the buyer- and vendor-side, to assess the quality of produced content using localization quality models (LISA, SAE J2450, MQM, etc.) and correlating the results. What is the impact of LLMs in translation quality? Will the same quality models apply? How can Quality Managers keep up with localization volumes doubling every year?
Truth is that we may start looking to LLMs not only as tools to serve content, but to help us evaluate said content through automated quality evaluation. Uber’s use case on applying analytical quality assessments through LLMs covers AI application to quality operations in order to automate processes, limit human dependency, improve efficiencies and reduce TATs and costs, in order for Quality Managers to focus on strategy, root cause analysis and localization continuous improvements.
Host organization: Globalization and Localization Association
Event Speakers
Marta Nieto Cayuela
Uber
Based in Barcelona and with a background in Translation and Linguistics, Marta started in the Localization industry 10+ years ago. In 2013, she joined the vendor side where she hold different roles that allowed her to understand the process and requirements in order to work now towards continuous improvement and programs at scale. For the last five years, she has dedicated herself to Language Quality Management supporting the creation and definition of frameworks. In 2023, she moved to the client side to fully grasp e2e localization processes and currently owns AI quality frameworks at Uber, as well as the EMEA user experience.
James Lin
Uber
With over two decades of expertise in internationalization and localization, James excels at spearheading international development and managing technical localization operations at Uber. His specialization lies in building and transforming cross-functional teams and developing products for global markets. Being a knowledgeable expert in globalization, CMS integration, machine translation, end-to-end custom workflows, localization transformation, and vendor management, James currently leads as Head of Localization Operations at Uber, optimizing translation pipelines via vendor and MT-agnostic models.