Databricks Engineer (Insurance Domain)

Job description

Share your resume over **Aarushi.Shukla@Coforge.Com** Location: **Hyderabad || Greater Noida** **Years of Experience**: 6-7 years **Job Summary:** We are seeking a highly skilled Senior Lead Engineer with extensive experience in the insurance domain and a strong proficiency in Databricks. The ideal candidate will work on complex projects, ensuring successful delivery while leveraging Databricks for data analytics and processing. **Responsibilities:** - Lead end-to-end design and delivery of enterprise data solutions on Databricks. - Design and implement Medallion Architecture (Bronze, Silver, Gold layers). - Build scalable ETL/ELT pipelines using **PySpark** and **Spark SQL**. - Develop ingestion frameworks using **Databricks Auto Loader** for batch and streaming workloads. - Design and implement **Delta Lake** based solutions including CDC, MERGE, Time Travel, and Schema Evolution. - Develop data quality, validation, reconciliation, and monitoring frameworks. - Implement orchestration using **Databricks Workflows** and external schedulers. - Define data models, transformation logic, and integration patterns for insurance data platforms. - Implement security and governance using **Unity Catalog**, RBAC, and data access controls. **Mandatory Skills:** - Strong expertise in Databricks, including data engineering and analytics. In-depth knowledge of the insurance domain, including industry standards and regulations. - Proven experience in project management methodologies (Agile, Waterfall, etc.). - Excellent leadership and team management skills. Strong analytical and problem-solving abilities. - Exceptional communication and interpersonal skills. **Strong expertise in:** - Databricks Lakehouse Platform - PySpark - Spark SQL - Delta Lake - Auto Loader - Databricks Workflows - Unity Catalog - Delta Live Tables (DLT)