Software Engineer - Apache Spark

Business Area: Engineering Seniority Level: Associate Job Description: At Cloudera, we empower people to transform complex data into clear and actionable insights. With as much data under management as the hyperscalers, we're the preferred data partner for the top companies in almost every industry. Powered by the relentless innovation of the open source community, Cloudera advances digital transformation for the world’s largest enterprises. The Data Platform Pillar is the bedrock of Cloudera’s technology, where we design and build the core components that let our customers store, manage, and process data with unmatched scalability, security, and performance. Are you ready to build the future of big data? Cloudera is searching for a talented Software Engineer to join the Apache Spark Team. You will work on our next-generation, enterprise-grade system, designed to solve data challenges at a massive scale—supporting Spark clusters running on thousands of nodes for the world’s largest companies. This is a unique opportunity to learn from and work alongside Spark committers and a high-impact, distributed team. You will contribute to the open-source community as an active participant in Apache Spark while developing features for Cloudera’s Data Engineering Experience. If you are a problem-solver eager to master distributed data processing, come build with us. As a Software Engineer, you will: Contribute to Scalable Solutions: Implement and test features for Cloudera’s Data Engineering Experience, assisting in the development of systems that operate at a massive scale on thousands of production nodes. Support Open-Source Innovation: Work with senior engineers to contribute to Apache Spark, learning how to shape the future of distributed data processing in the open-source community. Build with Modern Stacks: Develop high-performance features using Scala, Java, and Python on modern data platforms. Deepen Technical Mastery: Gain and apply knowledge in core distributed data processing concepts, including SQL Planners, Apache Parquet, and Iceberg. Address Large-Scale Challenges: Assist with developing and debugging features in massive distributed systems, learning about scaling from hundreds to thousands of nodes. Ensure System Integrity: Conduct thorough root cause analysis, debug system-level issues, and resolve failures to maintain high system quality. Enhance Engineering Velocity: Improve internal infrastructure and tooling to streamline development, testing, and deployment processes. Collaborate Effectively: Work closely within a distributed team and with stakeholders to drive product delivery. We’re excited about you if you have: Professional Experience: 2-3+ years of experience in professional software development. Contribution & Execution: Proven experience implementing features or addressing issues, and delivering high-quality code. Core Languages: Strong proficiency in Java, Scala, or another JVM-based language. Engineering Excellence: Passion for clean coding, attention to detail, and a focus on software quality and maintainability. Communication: Strong oral and written communication skills for effective collaboration within a distributed team. Learning & Problem-Solving: Demonstrated ability to research, learn quickly, and solve technical problems with guidance. Growth Mindset: An open-minded approach with a desire to learn new technologies and an unwavering passion for building exceptional products. You may also have: Spark & Ecosystem: Experience with using/developing Apache Spark, Apache Iceberg, or other related technologies. Distributed Systems Understanding: Experience with large-scale, distributed systems development. SQL Knowledge: Experience with or interest in SQL Planners and Optimizers. Open-Source Contributions: Prior experience or interest as a contributor to open-source projects. Why this role matters: You will tackle complex distributed systems challenges, crafting the foundational software for the control an ... (truncated, view full listing at source)