Senior Data Engineer

POSITION SUMMARY: Design, develop, and maintain robust, scalable, and secure data pipelines for ingesting, transforming, and validating clinical real-world data (RWD) from various sources. Implement software solutions using programming languages (e.g., Python, SQL) and cloud technologies (e.g., AWS) to support clinical data management and analysis workflows. Work closely with the clinical data management specialist and other engineers to ensure software and data are of high-quality, scalable, and compliant with industry standards. PRIMARY RESPONSIBILITIES: Communication and Team Collaboration: Collaborate with software/quality engineers/analysts and product managers (technical and non-technical) to make key design and technical implementation decisions. Partner with stakeholders and users to define requirements and deliver reliable, extensible, operations-friendly applications, services, solutions, data pipelines, or reports. Communicate testing progress, risks, and quality metrics effectively to project stakeholders and senior management. Data Engineering Expertise: Design, build, and optimize production-grade ETL/ELT data pipelines and software components based on product requirements, and where applicable innovative ideas based on domain expertise. Data Analysis and Debugging: Analyze and fix defects identified within applications, services, solutions, data pipelines, or reports.Along with creating unit tests and coordinate testing within the Clinical Data and Software engineering team, and all relevant cross-functional teams. Data Management: Write SQL, Python, ELT code to query raw and structured clinical, and genomic data for various clinical data management activities and to provide data as a service. Data-as-a-Service: Create data marts, including performing data transformation and de-identification for internal and external data projects, research initiatives, quality monitoring programs, and meeting regulator submission requirements. Cloud Computing: Develop and maintain scalable data pipelines, storage solutions, and processing workflows using AWS services (e.g., S3, Redshift). Workflow Orchestration: Experience with Apache Airflow for scheduling and managing complex data workflows. Review design alternatives and architectural proposals and provide relevant and timely recommendations and suggestions to leadership. Work Scoping and Estimation: Accurately scope and estimate work using sound engineering best practices. Software Development Lifecycle (SDLC): Apply version control (Git), CI/CD pipelines, and agile development methodologies. Process Improvement: Perform code reviews and provide relevant and timely recommendations and suggestions for solutioning to the leadership team, and when appropriate, independently implement solutions. New Hire Onboarding/Training: Support staff recruitment and onboarding. Mentorship: Provide strong leadership and support to new hires and cross-functional team members, fostering an environment of professional development and growth. Conferences and Certifications: Maintenance of all relevant clinical or technical licensures. Attend conferences relevant to role and clinical field. Other duties and responsibilities to be performed as assigned QUALIFICATIONS: Educational Background: BS degree in computer science or comparable program. Technical Skills and Background: 4-5+ years of overall software development experience, preferably building complex data or process management applications SQL databases (Postgres, Snowflake, Redshift, Athena), SQL, and ETL Excel/Macros, Quicksight and Tableau or similar applications AWS services like Redshift and S3 Software engineering best practices including agile methodologies, code reviews, source management, build processes, testing, and operations Data Standards: Proficient with data standards (e.g., CDISC, FHIR) and regulatory requirements (e.g., HIPAA, GDPR) for data management and DEID. Database Systems: In-depth know ... (truncated, view full listing at source)