Azure Data Engineer - 8th May Friday - Virtual Interview

Job description

Required Technical Skill Set\\*\\* • Azure DataBricks/Spark (Mandatory) • Azure Data Factory (Mandatory • Strong level SQL/TSQL • Strong level Spark • Strong CI/CD experience • Microsoft Azure data services skills, specifically: • Azure Synapse/Azure SQL DW (Useful) • Azure DataBricks/Spark (Mandatory) • Azure Data Factory (Mandatory) • Azure Analysis Services (Useful) • Azure ML (Nice to have) • Azure Cosmos DB (Useful) • Azure EventHubs (Useful) • PowerBI (Useful) Responsible for both developing and supporting data products alongside existing data teams • Deliver outcomes and as well as support products in production (80/20 split) • Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory (ADF), Azure Synapse, and DBT. • Implement data transformation, data modeling, and data quality best practices. • Develop and optimize SQL-based transformations in Azure Synapse. • Ensure data integrity, security, and governance in compliance with best practices. • Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements. • Work towards automating and streamlining data engineering workflows. • Optimize cloud data infrastructure for performance and cost efficiency. • Support and maintain CI/CD pipelines for data engineering processes.