Principal Machine Learning Engineer, Evaluation

Job Requisition ID # 26WD96327 26WD96327, Principal Machine Learning Engineer, Evaluation French translation to follow!/Traduction française à suivre! Position Overview The work we do at Autodesk touches nearly every person on the planet. By creating software tools for making buildings, machines, and even the latest movies, we influence and empower some of the most creative people in the world. As a Principal Machine Learning Engineer focused on evaluating ML models for CAD and BIM, you will ensure ML-powered experiences meet high standards for quality, reliability, and customer impact across Autodesk products. You will work hands-on with ML-enabled Autodesk software to design evaluation datasets, metrics, and protocols that reflect real user workflows. You will use those signals to guide release readiness and continuous improvement in production. You will communicate results through clear evaluation summaries and recommendations, partnering with researchers, engineers, and product teams to close the gap between offline performance and real customer outcomes. You will report to a manager in the Model Delivery team within Autodesk Research. This role is based in proximity to our North American Autodesk offices, with a preference for Toronto or Vancouver. We support both in-person, hybrid, and remote work. Responsibilities Design and run evaluation protocols (datasets, metrics, statistical analysis) that reflect real CAD/BIM user workflows and production conditions Build repeatable evaluation tooling and automation to support model development, regression testing, and release readiness decisions Curate, process, and analyze data from multiple sources (including production derived samples) to assess model behavior and customer outcomes Validate end-to-end ML powered product experiences and translate product requirements into measurable evaluation criteria Communicate findings and recommendations clearly to researchers, engineers, product teams, and leadership Document model quality issues discovered in production or during validation (accuracy, robustness, latency, failure modes) and help prioritize follow-up investigations Minimum Qualifications BS or MS in Mechanical Engineering, Architecture, Computer Engineering, Computer Science, Applied Math, Statistics, or equivalent industry experience 4 years of professional experience in ML model evaluation, ML enabled QA, or applied ML engineering, including designing evaluation datasets, metrics, and protocols Strong software engineering skills for building repeatable evaluation systems, including Python, data pipelines and analysis, testable and maintainable code, version control, and cloud based workflows (for example AWS or Azure) Strong written communication skills for documenting evaluation methods, results, and recommendations Preferred Qualifications Familiarity with design, manufacturing, or AEC workflows, including hands-on experience with CAD/BIM tools such as Fusion, AutoCAD, or Revit Experience with geometry or design data representations, including 2D and 3D Familiarity with ML frameworks and tooling (for example PyTorch, Ray, or similar) ______________________________________________________________________________________________________________ 26WD96327, Ingénieur principal en apprentissage automatique, Évaluation Présentation du poste Le travail que nous accomplissons chez Autodesk touche pratiquement chaque personne sur la planète. En créant des outils logiciels destinés à la conception de bâtiments, de machines et même des films les plus récents, nous influençons et donnons les moyens d’agir à certaines des personnes les plus créatives au monde. En tant qu'ingénieur principal en apprentissage automatique spécialisé dans l'évaluation des modèles d'apprentissage automatique pour la CAO et le BIM, vous veillerez à ce que les expériences basées sur l'apprentissage automatique répondent à des normes élevées en matière de qualité, de fiabilité ... (truncated, view full listing at source)