Senior Analyst, Model Risk Management

<p>Toast creates technology to help restaurants and local businesses succeed in a digital world, helping business owners operate, increase sales, engage customers, and keep employees happy. Toast is driven by building the restaurant platform that helps restaurants adapt, take control, and get back to what they do best: building the businesses they love. Now, more than ever, the Toast team is committed to our customers. We’re taking steps to help restaurants navigate these unprecedented times with technology, resources, and community. Our focus is on building the restaurant platform that helps restaurants adapt, take control, and get back to what they do best: building the businesses they love. And because our technology is purpose-built for restaurants, by restaurant people, restaurants can trust thatwe’ll deliver on their needs for today while investing in experiences that will power their restaurant of the future.</p> <p> </p> <p><strong>Bready* to make a change?</strong></p> <p>As SeniorAnalyst in our Model Risk Management team, you will help manage the Model Risk Program including completing model validation reviews, maintaining the model risk inventory, and monitoring model performance particularly for Fraud and Gen AI Models. You will work closely with the Data Science Team, architects, engineers and product managers to assess the risk of model design, implementation, and use across product lines.<br><br></p> <p><strong>A day in the life (Responsibilities) </strong></p> <ul> <li>Support the implementation and day-to-day execution of the Second Line of Defense (2LOD) Model Risk Management (MRM) program for high-risk models, with particular focus on Fraud detection models (Transaction Fraud Merchant Fraud) and Generative AI / LLM-based systems deployed across Toast.</li> <li>Assist in maintaining and enhancing the Model Risk Management framework, including policies, procedures, validation standards, governance documentation, templates, and best practices aligned with evolving regulatory and industry expectations.</li> <li>Enforce model lifecycle standards across development, implementation, use, monitoring, recalibration, change management, governance, and decommissioning, ensuring appropriate controls for traditional ML models as well as GenAI systems (e.g., RAG architectures, copilots, AI-assisted decision tools).</li> <li>Contribute to the development, risk-tiering, and ongoing maintenance of a comprehensive model inventory, including assessment of model impact, intrinsic risk (complexity and methodology), reliance on model outputs, and emerging AI-specific risks.</li> <li>Perform independent model validation reviews under the guidance of senior leadership, covering conceptual soundness, data integrity, model methodology, performance metrics (e.g., AUC, precision/recall, calibration), stability, bias/fairness risk, explainability, and monitoring frameworks. Produce validation reports and track issue remediation plans through closure.</li> <li>For Fraud models, evaluate class imbalance handling, threshold optimization, cost-sensitive performance metrics, operational overlays, rule-based controls, and portfolio-level impact analyses.</li> <li>For Generative AI systems, validate systems and evaluate risks related to hallucination, prompt injection, adversarial vulnerabilities, data privacy and leakage, model explainability limitations, bias, guardrails, output monitoring, jailbreak testing, regression testing, and human-in-the-loop controls.</li> <li>Partner with Data Science, Data Engineering, Product/Engineering, Information Security, Legal/Compliance, Finance, Credit Risk and Business teams to obtain documentation, perform effective challenge, conduct validation and oversee performance monitoring</li> <li>Prepare reports and executive materials summarizing model risk issues, validation findings, monitoring insights, and remediation status for leadership review, risk committees, audit committees, and internal audit engag ... (truncated, view full listing at source)