Staff Software Engineer, Observability

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. Our novel wafer-scale architecture provides the AI compute power of dozens of GPUs on a single chip, with the programming simplicity of a single device. This approach allows Cerebras to deliver industry-leading training and inference speeds and empowers machine learning users to effortlessly run large-scale ML applications, without the hassle of managing hundreds of GPUs or TPUs. Cerebras' current customers include top model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership with Cerebras , to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference. Thanks to the groundbreaking wafer-scale architecture, Cerebras Inference offers the fastest Generative AI inference solution in the world, over 10 times faster than GPU-based hyperscale cloud inference services. This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation. About The Role We are seeking a talented Platform Software Engineer to join the team building the Cerebras Inference Platform. You will be instrumental in designing, developing, and operating the core backend services and APIs that power the Inference platform. You'll build the software that allows customers to seamlessly deploy, manage, and serve inference workloads on dedicated Cerebras hardware. Responsibilities Set Technical Direction: Own the long-term architecture and roadmap for the observability platform across all pillars (metrics, logs, traces, and events). Drive adoption across the engineering organization through RFCs, design reviews, and close partnership with stakeholders. Build for Cerebras-Scale: Design telemetry pipelines that handle high-cardinality, high-frequency data from thousands of Wafer-Scale Engines, large Kubernetes clusters, and the storage and networking fabric connecting them. Solve ingestion and query performance problems that break conventional approaches. Drive Reliability Across the Organization: Partner with Engineering, SRE, Hardware, and ML Infrastructure teams to define SLOs, build alerting strategies, and create tooling that accelerates root cause analysis and reduces MTTR. Make it easy for any engineer to understand system health and diagnose failures. Bridge Hardware and Software Observability: Work with engineering teams accross the stack to surface critical hardware health signals into a unified observability layer alongside software metrics. Shape Developer Experience: Design and implement instrumentation libraries and standards (OpenTelemetry) that make rich observability the default for every service, not an afterthought. Establish conventions that scale across teams. Mentor and Grow Engineers: Raise the technical bar across the organization. Mentor senior engineers, influence engineering practices across team boundaries, and foster a culture of operational excellence. Qualifications 8+ years of software engineering experience, with 4+ years building or operating observability/monitoring platforms at significant scale (millions of active time series, petabytes of log data). Deep expertise in the open-source observability ecosystem. You have operated, tuned, and extended systems like Prometheus, Thanos/Cortex/Mimir, Elasticsearch/ClickHouse, or Loki and understand their internals, failure modes, and cost characteristics. Experience with OpenTelemetry for instrumentation across a polyglot services environment. Proficiency in Go preferred, with strong experience in Python. You write performant, concurrent systems code, not just glue scripts. Strong distributed systems and Kubernetes expertise: sharding, replication, consistency models, monitoring K8s itself and the workloads running on it, and the tradeoffs involved in building reliable ... (truncated, view full listing at source)