Databricks Engineer

Job description

**Role & responsibilities** - 3+ years of hands-on experience working with Databricks, Apache Spark, or related big data platforms - Proficiency in programming languages such as Python, Scala, or SQL for data transformation and pipeline development - Hands-on experience with Spark SQL, Delta Lake, Lakeflow Spark Declarative Pipelines and Unity Catalog - Strong understanding of ETL/ELT workflows, data modeling, and data warehousing concepts - Experience with cloud platforms (Azure, AWS, or GCP) - Familiarity with CI/CD practices and version control tools such as Git - Knowledge of data governance, data quality, and security best practices - Excellent problem-solving, analytical, and communication skills. - Ability to work collaboratively in an agile team environment - Databricks certifications are preferred **Preferred candidate profile** - Implement data ingestion, transformation, and integration processes from various structured and unstructured data sources - Develop, maintain, and optimize scalable data pipelines and workflows on the Databricks platform - Collaborate with analysts and business stakeholders to understand data requirements and deliver solutions - Ensure data reliability, quality, and security across all data engineering processes - Troubleshoot, debug, and resolve data pipeline issues and performance bottlenecks - Document data engineering workflows and technical designs - Monitor and maintain data pipeline health, proactively addressing any potential issues - Stay up-to-date with emerging data engineering technologies and best practices, especially within the Databricks ecosystem