Lead Data Engineer - Databricks- (7 to 10 Years)

Job description

MSA: Lead Data Engineer - Databricks - Special Mission : Build modern data platforms that underpin bank’s ambitions for next phase of growth. Main Responsibilities : Design and implement scalable Lakehouse architecture using Databricks, Delta Lake, and Cloud Storage. Develop robust ETL/ELT pipelines using PySpark, Spark SQL, and Databricks Notebooks. Integrate Databricks with Cloud Services (AWS/Azure) including storage, IAM, and event services. Implement Unity Catalog, Data Lineage, Access Governance, and Auditing Standards. Define and enforce data quality, observability, and FinOps monitoring. Profile : 7–10 years of IT experience with min 4 yrs exp in designing, building, and optimizing large-scale data processing solutions on the Databricks Platform Skills: DBX, PySpark, Spark SQL, Delta Lake, SQL Warehouse, Cloud Platforms (AWS/Azure), CI/CD, Data Governance Preferred: DBX Certification, MLFlow Functional background: Experience in Banking and Financial Services - particularly within Corporate Functions domain Methodology: Agile Behavioural Skills: Ability to work in a cross-cultural team with multi geographical locations. Ability to communicate (verbal and written) effectively with counterparts in SG Paris Profile Main Responsibilities : Design and implement scalable Lakehouse architecture using Databricks, Delta Lake, and Cloud Storage. Develop robust ETL/ELT pipelines using PySpark, Spark SQL, and Databricks Notebooks. Integrate Databricks with Cloud Services (AWS/Azure) including storage, IAM, and event services. Implement Unity Catalog, Data Lineage, Access Governance, and Auditing Standards. Define and enforce data quality, observability, and FinOps monitoring. Profile : 7–10 years of IT experience with min 4 yrs exp in designing, building, and optimizing large-scale data processing solutions on the Databricks Platform Skills: DBX, PySpark, Spark SQL, Delta Lake, SQL Warehouse, Cloud Platforms (AWS/Azure), CI/CD, Data Governance Preferred: DBX Certification, MLFlow Functional background: Experience in Banking and Financial Services - particularly within Corporate Functions domain Methodology: Agile