Databricks_Subcon

Job description

Area(s) of responsibility **About The Role** We are seeking a Senior Data Platform Engineer with extensive hands-on experience to build and scale enterprise-grade data pipelines on Azure and Databricks. This role requires deep technical expertise, strong ownership, and the ability to independently deliver complex, production-ready data solutions while setting engineering best practices. What You'll Do - Build scalable pipelines using Databricks (DLT, Autoloader, Workflows) and Azure services - Extend metadata-driven ingestion frameworks (declarative, reusable) - Develop modular PySpark code and libraries (WHL packaging) - Implement Delta Lake patterns (CDC, SCD Type 2, merges) - Optimize SQL workloads for performance and cost - Orchestrate pipelines via Synapse or UC4 - Apply data modeling (Kimball, 3NF, lakehouse) - Implement CI/CD with Azure DevOps - Enforce governance via Unity Catalog - Mentor engineers and contribute to engineering standards Required Technical Skills & Proficiency Category Technology / Skill Required Proficiency Cloud Platform Azure (ADLS Gen2, Key Vault, Azure SQL, Entra ID) Advanced (Deep hands-on, production-scale design & troubleshooting)