Staff Software Engineer - Reliability

About Us Temporal is an open source programming model that can simplify code, make applications more reliable, and help developers focus on the important things like delivering features faster. We are on a mission to be the reliable foundation of every developer’s toolbox, and are building the team that will make that happen. Our values guide us —they are present in how we show up, make decisions, and work together to make an impact. We’re curious, driven, collaborative, genuine and humble. Temporal is growing and we are looking for those who share our values, challenge 'standard' thinking, and want to influence our future. If you have a passion for improving the developer experience, building world-class open-source software and communities, and want to be a part of our amazing team, we'd love to hear from you! Summary Join our Reliability team as a Staff Software Engineer, where you’ll own the reliability of operating Temporal Cloud end to end. You will help define and measure reliability expectations, harden systems through gamedays and chaos testing, and build the tooling and practices that make reliability visible and continuously improving across services and operational processes. We’re looking for someone who thrives in ambiguity, enjoys turning reliability goals into concrete engineering work, and can lead cross-team efforts that make systems more resilient at scale. What You’ll Do Own reliability outcomes for operating Temporal Cloud end to end, partnering across engineering, infrastructure, and product to drive measurable improvements. Define, implement, and evolve reliability targets and associated practices, including alerting thresholds, operational readiness criteria, and escalation paths. Plan and run gamedays to validate incident response, operational procedures, and cross-team coordination under realistic failure scenarios. Build and scale a chaos testing program that exercises failure modes safely and drives remediation work that reduces real risk. Define and maintain a reliability scorecard across services and key operational processes, and use it to prioritize reliability investments. Lead load testing and performance testing efforts, including test design, tooling, and analysis of bottlenecks and capacity constraints. Improve observability standards (metrics, logs, traces, dashboards) so reliability signals are consistent, actionable, and easy to audit. Drive post-incident learning and corrective actions, ensuring fixes are durable and reduce recurrence risk over time. Make system-level tradeoffs across reliability, performance, cost, and velocity, and document decisions clearly for long-term maintainability. Mentor other engineers and raise the bar on reliability engineering practices across teams. What You’ll Bring Strong computer science fundamentals, especially in distributed systems, concurrency, and performance. Demonstrated ability to design and build complex systems that operate reliably under high load and partial failure. Experience driving reliability improvements across multiple services, not just within a single codebase. Hands-on experience with at least one of: gamedays, chaos testing, load testing, or building reliability scorecards. Strong judgment in ambiguous situations, including the ability to prioritize reliability work based on risk and impact. Excellent communication skills, including the ability to align multiple stakeholders on reliability goals, plans, and tradeoffs. A collaborative mindset and a track record of mentoring and leveling up engineering practices. Nice to Haves Experience operating multi-tenant systems and designing protections against noisy-neighbor behaviors. Deep expertise in observability (metrics design, tracing strategy, dashboard standards) and alert hygiene. Experience building internal platforms or tooling that enables other teams to meet reliability standards. Familiarity with workflow orchestration systems or durable execution ... (truncated, view full listing at source)