Data Engineer – Azure Databricks & PySpark

Job description

Data Engineer – Azure Databricks & PySpark (US Healthcare – Mandatory) **Experience:** 6+ Years **Job Summary** We are looking for an experienced **Data Engineer** with strong expertise in **Azure Databricks, PySpark, Oracle SQL, and ETL/ELT development** . The ideal candidate **must have hands-on experience in the US Healthcare domain** , working with payer datasets such as **Claims, Membership, Provider, and Eligibility** . **Mandatory Skills** - 5+ years of experience in Data Engineering/ETL development. - **US Healthcare (Payer) domain experience is Mandatory.** - Strong SQL expertise with **Oracle (Mandatory)** . - Hands-on experience with **Azure Databricks** and **PySpark** . - Experience in designing and developing ETL/ELT pipelines. - Working knowledge of **Azure Synapse Analytics** . - Strong Python programming skills. - Experience in data validation, debugging, and performance tuning. - Experience with healthcare datasets such as **Claims, Membership, Provider, and Eligibility** . **Key Responsibilities** - Design, develop, and maintain ETL/ELT pipelines using Azure Databricks. - Develop scalable data transformation solutions using PySpark. - Write and optimize complex SQL queries in Oracle, Azure SQL, and Synapse. - Perform data validation, reconciliation, and performance optimization. - Support cloud migration initiatives from Oracle to Azure. - Build high-quality datasets for reporting and analytics (Power BI support). - Collaborate with Product Owners, Data Scientists, BI teams, and business stakeholders. - Ensure data quality, governance, and compliance with enterprise standards. - Participate in Agile ceremonies, sprint planning, and documentation. **Regards,** Chetan Gurudev chetan.gurudev@sagility.com