Azure Senior Data Engineer - S

Job description

**Key Responsibilities** - Design, develop, and maintain scalable and secure data platforms on Microsoft Azure. - Build and optimize ETL/ELT pipelines for ingesting, transforming, and processing large volumes of data. - Design and implement Azure-based data lake, data warehouse, and data integration solutions. - Develop batch and real-time data processing pipelines. - Integrate data from multiple structured, semi-structured, and unstructured data sources. - Build and maintain scalable data models to support analytics and reporting needs. - Ensure data quality, governance, security, and compliance standards are implemented. - Optimize data pipeline performance, reliability, and cost efficiency. - Collaborate with Data Scientists, Analysts, Architects, and business stakeholders to understand data requirements. - Implement monitoring, logging, and alerting for data pipelines and platforms. - Troubleshoot and resolve production data issues and bottlenecks. - Support cloud migration and modernization initiatives. - Develop technical documentation, architecture diagrams, and operational runbooks. - Mentor junior engineers and promote best practices in data engineering. **Required Skills & Qualifications** - Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field. - 6+ years of experience in Data Engineering with at least 3+ years of Azure cloud experience. - Strong programming skills in: - Python - SQL - PySpark - Scala (preferred) - Hands-on experience with Azure Data Services: - Azure Data Factory (ADF) - Azure Databricks - Azure Synapse Analytics - Azure Data Lake Storage (ADLS Gen2) - Azure SQL Database - Azure Cosmos DB - Azure Event Hubs - Azure Stream Analytics - Azure Functions - Azure Key Vault - Azure Monitor - Strong expertise in: - ETL/ELT Development - Data Modeling - Data Warehousing - Data Lake Architecture - Dimensional Modeling - Data Governance - Experience with Apache Spark and distributed data processing frameworks. - Strong knowledge of database technologies including SQL and NoSQL databases. - Experience with version control systems such as Git. - Excellent analytical, troubleshooting, and problem-solving skills.