Azure Data Bricks Sr Developer

Job description

Job Description - -------------- **Job Role - Senior Azure Databricks Developer** **Experience -7 Years** **Location - Gurgaon** Key Responsibilities * Design and develop scalable data ingestion, transformation, and processing pipelines using **Azure Databricks (PySpark/Scala)**. * Build and maintain **ETL/ELT workflows** for structured and unstructured data. * Implement and manage **Delta Lake** architecture and optimize performance of Spark jobs. * Integrate Azure Databricks with: * Azure Data Factory (ADF) * Azure Data Lake Storage Gen2 (ADLS) * Azure Synapse Analytics * Azure SQL Database * Event Hub/Kafka * Develop reusable frameworks, notebooks, and data engineering best practices. * Optimize Spark workloads through partitioning, caching, indexing, and cluster tuning. * Implement data quality checks, monitoring, and governance standards. * Collaborate with business stakeholders, architects, data scientists, and BI teams. * Support CI/CD implementation using Azure DevOps or GitHub Actions. * Troubleshoot production issues and provide performance improvements. Required Skills Azure \& Databricks * Azure Databricks * PySpark / Apache Spark * Delta Lake * Unity Catalog * Databricks Workflows * Databricks SQL Data Engineering * ETL/ELT Development * Data Warehousing Concepts * Data Lakehouse Architecture * Data Modeling Azure Services * Azure Data Factory (ADF) * Azure Data Lake Storage Gen2 (ADLS) * Azure Synapse Analytics * Azure Key Vault * Azure Functions Programming * Python (Advanced) * SQL (Advanced) * Scala (Preferred) Version Control \& DevOps * Git/GitHub * Azure DevOps * CI/CD Pipelines