Service Quality Specialist, GenAI Manual Evaluations

Service Quality Specialist, GenAI Manual Evaluations Department: Community Operations Team: Support Operations Location: Phoenix, United States Type: Full-Time **About The Role** The GenAI Manual Evaluations team serve as Uber’s human-in-the-loop quality, safety, and risk gate for AI deployments that will engage with our users. The team plays a critical role in ensuring AI meets a human bar and that our products can scale with consistent conformance, as well as prevent downstream impact by surfacing CX, policy, and financial risks before customer impact. The GenAI Manual Evaluations team is responsible for conducting end-to-end reviews of customer and agent interactions with Uber’s AI-powered support tools. We assess the full experience—from product behavior and agent responses to language quality, translations, policy accuracy, and knowledge base alignment. Our goal is to establish a scalable, continuous review process that ensures the quality of AI-driven support solutions and enables the seamless integration of new features. By identifying experience gaps and areas for improvement, we deliver actionable insights that directly inform product, engineering, and operations roadmaps. **What You’ll Do:** - Execute rubric-based evaluations on real interactions between Uber’s users (earners, riders, merchants) and our GenAI support products. - Assess the accuracy, relevance, and overall customer experience of the responses provided by AI. - Identify defects, edge cases, and gaps in the support experience and document findings into relevant and actionable insights for cross functional teams to action. - Support the calibration and continuous improvement of various AI programs as we continue to scale to a larger audience. - Become an SME of multiple tools & programs to provide comprehensive evaluations of varying interaction types. - Communicate quickly and effectively to internal Uber members as needed - Become an expert on processes and may help with sophisticated problem solving - Follow process guides to complete tasks and provide support to various crucial initiatives - Work with your Team Lead and the Manual Evals Prog