Databricks

Job description

**⃣ Sr. Data Engineers – Databricks Pipeline Developer** **Role Overview** Build and maintain data pipelines in Databricks for ingestion, cleansing, enrichment, and storage. **Key Responsibilities** - Develop PySpark pipelines for - API ingestion - FTP file ingestion - Data cleansing - Enrichment transformations - Implement incremental & batch loads - Develop Bronze → Silver → Gold transformations - Translate legacy SQL stored procedures into Spark jobs - Optimize Delta Lake tables - Implement validation checks - Support testing & reconciliation **Required Skills** - Strong PySpark / Spark SQL - Strong SQL (read & interpret legacy procedures) - Experience with API integration - Experience handling flat files (FTP ingestion) - Experience with Delta Lake - Knowledge of partitioning & optimization