Quality Engineer (Data Engineering)

<p>We are seeking a highly motivated and skilled Quality Engineer to join our Data Engineering team in Hyderabad, India. This individual will play a critical role in ensuring the accuracy, reliability, and integrity of our central data lake and reporting products. The ideal candidate will be passionate about data quality, possess a strong understanding of data engineering principles, and be adept at developing and implementing robust quality controls and testing strategies for data pipelines and reporting solutions.</p> <p> </p> <p><span style="text-decoration: underline;"><strong>Key Responsibilities</strong></span></p> <ul> <li><strong>Design and Execute Quality Strategy:</strong> Develop and implement comprehensive test strategies and quality control processes specifically for data ingestion, transformation, storage, and consumption within the data lake and reporting ecosystem.</li> <li><strong>Data Quality Testing:</strong> Create, execute, and maintain automated and manual data quality tests (including unit, integration, and end-to-end tests) to validate data accuracy, completeness, consistency, and conformity with business rules.</li> <li><strong>Pipeline Validation:</strong> Validate ETL/ELT data pipelines to ensure data is processed correctly and efficiently, focusing on error handling, performance, and scalability.</li> <li><strong>Reporting and Dashboard Verification:</strong> Thoroughly test and validate the data displayed in business intelligence (BI) reports and dashboards against source data to guarantee accuracy and reliability for business decision-making.</li> <li><strong>Quality Gates Monitoring:</strong> Implement quality gates within the CI/CD pipeline and establish data quality monitoring dashboards to proactively identify and alert on data anomalies and pipeline failures.</li> <li><strong>Collaboration:</strong> Work closely with Data Engineers, Data Scientists, and Product Managers to understand data requirements, define quality metrics, and integrate quality assurance throughout the data development lifecycle.</li> <li><strong>Documentation:</strong> Document test plans, test cases, and quality control processes, as well as track and report on data quality metrics and issues.</li> </ul> <p><span style="text-decoration: underline;"><strong>Required Qualifications</strong></span></p> <ul> <li>5+ years of experience in Quality Assurance or Quality Engineering, with a focus on data-centric testing or data warehousing environments.</li> <li>Strong proficiency in SQL for data querying and validation.</li> <li>Experience in developing test automation frameworks and scripts for data pipelines using tools/languages like Python, PHP, or Javascript.</li> <li>Solid understanding of cloud-based data warehousing and data lake concepts (e.g., AWS S3/Glue, Azure Data Lake/Synapse, Google Cloud Storage/BigQuery).</li> <li>Familiarity with ETL/ELT tools and data pipeline orchestration (e.g., Apache Airflow, dbt, Talend).</li> <li>Experience with BI tools (e.g., Tableau, Looker, Power BI) for report validation and testing.</li> </ul> <p><span style="text-decoration: underline;"><strong>Preferred Qualifications</strong></span></p> <ul> <li>Experience with big data technologies.</li> <li>Knowledge of data governance, metadata management, and master data management principles.</li> <li>Familiarity with performance testing and optimization techniques for large-scale data systems.</li> <li>Experience working in an Agile/Scrum development environment.</li> <li>Technical leadership for steering testing efforts.</li> </ul>