Data Engineer

Job description

Jon Description: We are seeking an experienced Azure Databricks Engineer with 6+ years of experience in Data Engineering and strong expertise in Azure cloud technologies. The ideal candidate should have hands-on experience building scalable data pipelines using Azure Databricks, PySpark, and Azure Data Factory, along with a solid understanding of US Healthcare data models, claims, provider, member, and clinical datasets. The candidate will be responsible for designing, developing, and optimizing enterprise-scale data solutions while ensuring compliance with healthcare data standards and governance requirements. Mandatory Skills: Azure Databricks, PySpark, Python, SQL, Azure Data Factory, Azure Synapse, ADLS Gen2, Delta Lake, US Healthcare Domain. Key Responsibilities • Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks and Azure Data Factory. • Build and optimize data ingestion, transformation, and processing frameworks using PySpark and Spark SQL. • Develop and maintain data solutions using Azure Data Lake Storage (ADLS Gen2), Azure Synapse Analytics, and Azure SQL. • Implement Lakehouse architecture using Delta Lake and Databricks best practices. • Work with large healthcare datasets including Claims, Membership, Provider, Clinical, EHR/EMR, HL7, and FHIR data. • Ensure data quality, governance, security, and compliance with healthcare regulations. • Perform performance tuning and optimization of Databricks clusters, notebooks, and Spark jobs. • Collaborate with Data Architects, Business Analysts, Data Scientists, and Healthcare SMEs. • Develop reusable frameworks and CI/CD pipelines using Azure DevOps. • Support production deployments, monitoring, troubleshooting, and incident resolution. Required Skills: • 6+ years of experience in Data Engineering. • 6+ years of hands-on experience with Azure Databricks. • Strong expertise in: • Azure Databricks • PySpark • Spark SQL • Python • SQL • Azure Data Factory (ADF) • Azure Data Lake Storage (ADLS) • Azure Synapse Analytics • Delta Lake • Experience with Data Warehousing concepts, Data Modeling (Star/Snowflake Schema), and ETL/ELT development. • Knowledge of Medallion Architecture (Bronze, Silver, Gold layers). • Experience with Git, Azure DevOps, and CI/CD implementation. • Strong understanding of performance tuning and optimization techniques in Databricks. US Healthcare Experience (Mandatory) • Minimum 2+ years of experience working in the US Healthcare domain. • Strong understanding of: • Claims Data • Member Data • Provider Data • Clinical Data • EHR/EMR Systems • HL7/FHIR Standards • HIPAA Compliance • PHI/PII Data Handling • Experience supporting healthcare analytics and reporting initiatives. Preferred Skills • Experience with Azure Event Hub or Kafka. • Knowledge of MLflow and Databricks Workflows. • Experience with Power BI. • Exposure to Microsoft Purview or other Data Governance tools. • Azure Data Engineer or Databricks Certification preferred. Educational Qualification • Bachelor's or master's degree in computer science, Information Technology, Engineering, or a related field. Key Competencies • Strong analytical and problem-solving skills. • Excellent communication and stakeholder management skills. • Ability to work independently in a fast-paced environment. • Experience collaborating with global teams and business stakeholders. Note: 📢 Face-to-Face Interview Drive – 5-Sep-2026 | Bangalore 📅 Interview Date: 5-Sep-2026 ⏰ Interview Timing: 10:00 AM – 12:00 PM 📍 Location: Bangalore – AMR Tech Park 🏢 Work Mode: Hybrid – 2 days WFO per week 🕑 Shift Timing: 2:00 PM – 11:00 PM 📌 Notice Period: Immediate to 15 Days 📩 Interested candidates, please share your updated profile to: Shaik.ahamed@sagility.com Thanks & regards, Shaik Thanveer Ahamed