Azure Databricks Engineer/Lead/ Architect

Job description

🚀 Hiring: Azure Databricks Data Engineer| HCLTech 🚀 We are seeking a highly skilled **Azure Databricks Data Engineer** to build and optimize modern data platforms leveraging the **Lakehouse architecture** . If you are passionate about designing scalable data solutions, implementing robust ETL/ELT pipelines, and enabling data-driven insights through Databricks, we'd love to hear from you. 📍 Location: Chennai | Bengaluru | Hyderabad| Noida | Pune 💼 Experience: 7 – 15 Years What We're Looking For Key Responsibilities - Design and implement **Data Lakehouse architectures** using the **Medallion Framework (Bronze, Silver, Gold layers)** . - Develop high-performance data pipelines using **PySpark notebooks** and Databricks. - Build and manage **Delta Live Tables (DLT)** pipelines for reliable and efficient data processing. - Implement automated data ingestion using **Autoloader** for batch and streaming workloads. - Ingest and process data from diverse sources including files, databases, APIs, and streaming platforms. - Perform complex **ETL/ELT transformations** to support analytics and reporting requirements. - Ensure data governance, security, and access control using **Unity Catalog** . - Monitor, optimize, and troubleshoot pipelines for performance, scalability, and cost efficiency. - Implement robust **data quality** checks, validations, and monitoring frameworks. - Develop and optimize advanced SQL queries for large-scale data workloads. Mandatory Skills ✅ Azure Databricks ✅ Data Lakehouse / Medallion Architecture (Bronze-Silver-Gold) ✅ PySpark Development ✅ Delta Live Tables (DLT) ✅ Databricks Autoloader ✅ Unity Catalog & Data Governance ✅ ETL / ELT Pipeline Development ✅ Advanced SQL ✅ Data Quality & Validation Frameworks ✅ Data Ingestion (Files, Databases, Streaming Sources) ✅ Pipeline Performance Tuning & Optimization 📩 **Interested candidates can share their updated resumes or refer suitable professionals from their network.** **Keywords:** Azure Databricks | Data Engineer | PySpark | Delta Live Tables | DLT | Autoloader | Unity Catalog | Data Governance | Data Lakehouse | Medallion Architecture | ETL | ELT | Advanced SQL | Data Quality | Delta Lake