Manual Evaluations Program Leader

Manual Evaluations Program Leader Department: Community Operations Team: Strategy & Program Operations Location: San Francisco, United States Type: Full-Time **About the Role** The Manual Evaluations Program Leader will own the end-to-end strategy, design and execution of human evaluations for Uber’s GenAI-powered products, including conversational AI, voice AI, agent workflows and auto-evaluation systems. This role sits within the Global Digital Experience team, the operational arm of Uber’s customer support tech organisation, and is a critical driver of quality, safety, and performance across Uber’s next-generation AI solutions. This leader will build and scale Uber’s Manual Evaluation framework: defining methodologies, creating evaluation rubrics, ensuring annotation quality, and generating the insights that shape model tuning, product improvements, and release decisions. They will partner closely with Product, Engineering, Data Science and Product Ops to translate evaluation outcomes into clear technical and operational actions. The role includes both strategic leadership and operational execution. The Program Leader will directly manage a team of three and indirectly oversee a distributed network of evaluators across global business sites. They will be responsible for setting the quality bar for evaluations, ensuring consistent delivery at scale, and driving continuous improvement of the evaluation pipeline. The ideal candidate brings strong technical literacy in GenAI systems, exceptional program design and operational skills, and the ability to lead high-impact cross-functional initiatives. They are comfortable navigating ambiguity, building strong partnerships across Uber and influencing product direction through rigorous evaluation insights. This is a rare opportunity to play a leading role in one of Uber’s most transformative technology programs and help shape the future of Uber’s AI-driven experiences. **What the Candidate Will Do** 1. **Own the end-to-end strategy, design, and execution** of Manual Evaluations for Uber’s GenAI-powered products (chatbots, voice AI, automated workflows, autoeval systems)