Senior+ Software Engineer - Cloud Availability Platform Engineering (Observability)

Crusoe's mission is to accelerate the abundance of energy and intelligence. We’re crafting the engine that powers a world where people can create ambitiously with AI — without sacrificing scale, speed, or sustainability.Be a part of the AI revolution with sustainable technology at Crusoe. Here, you'll drive meaningful innovation, make a tangible impact, and join a team that’s setting the pace for responsible, transformative cloud infrastructure.About the Role:We are looking for a highly skilled engineer with deep expertise in building and operating observability platforms at scale. You will design, develop, and run Crusoe’s next-generation observability stack, enabling engineers to understand the internal state of distributed systems through metrics, logs, and traces. Your work will ensure reliability, performance, and actionable insights across Crusoe’s global infrastructure and cloud platform.What You’ll Be Working On:Designing and operating scalable observability systems (metrics, logging, tracing) across multi-datacenter Kubernetes environmentsArchitecting end-to-end telemetry pipelines, including ingestion, storage, querying, and visualizationExtending monitoring and alerting with Prometheus, Alertmanager, Thanos/Cortex, Grafana, and OpenTelemetryBuilding scalable log collection and processing pipelines with Fluent Bit, Vector, Loki, or ELK/Opensearch stacksImplementing distributed tracing platforms (Tempo, Jaeger, OpenTelemetry) and integrating with service meshes, load balancers, and APIsDefining and driving adoption of SLOs, SLIs, and error budgets across services and teamsAutomating provisioning and scaling of observability infrastructure with Kubernetes, Terraform, and custom tooling (Go, Python)Ensuring reliability and cost efficiency of telemetry pipelines while supporting high-volume workloads (AI/ML, HPC clusters, GPU infrastructure)Embedding security best practices into observability platforms, including RBAC, TLS, secret management, and multi-tenant access controlsPartnering with engineering teams to embed observability into applications, services, and infrastructureMentoring engineers and shaping Crusoe’s observability strategy and technical roadmapWhat You’ll Bring to the Team:7+ years of experience in infrastructure or platform engineering, with a focus on observability and monitoring systemsDeep expertise with metrics systems (Prometheus, Thanos, Mimir, Cortex), logging pipelines (Fluent Bit, Vector, Loki, ELK/Opensearch), and tracing platforms (Jaeger, Tempo, OpenTelemetry)Strong programming skills in Go or Python for automation, operators, and custom integrationsExperience running observability platforms on Kubernetes and operating them at scale across multi-datacenter environmentsProven ability to design, optimize, and scale telemetry pipelines handling high cardinality and high throughput dataSolid understanding of distributed systems, performance engineering, and debugging complex workloadsStrong collaboration skills and the ability to influence engineering teams to adopt observability best practicesBonus Points:Contributions to open source observability projects (Prometheus, OpenTelemetry, Grafana, Loki, etc.)Experience supporting AI/ML or GPU-heavy environments with high observability demandsKnowledge of event-driven or streaming systems (Kafka, NATS, Pulsar) used in telemetry pipelinesExperience implementing cost optimization strategies for large-scale observability platformsBackground in incident response, chaos engineering, and reliability practicesBenefits:Industry competitive payRestricted Stock Units in a fast growing, well-funded technology companyHealth insurance package options that include HDHP and PPO, vision, and dental for you and your dependentsEmployer contributions to HSA accountsPaid Parental LeavePaid life insurance, short-term and long-term disabilityTeladoc401(k) with a 100% match up to 4% of salaryGenerous paid time off and holiday scheduleCell phone reimbursementTuition reimburs ... (truncated, view full listing at source)