Databricks Engineer - Immediate Joiners Only

Job description

Job Title: Databricks Engineer - 5+ Experience Job Summary We are looking for a highly skilled Databricks Engineer to design, build, optimize, and maintain large-scale data processing pipelines and analytics workloads on the Databricks platform. This role requires strong hands-on experience with Databricks Jobs, Delta Tables, Pipelines, Spark SQL, AWS integrations, Genie workflows, and Databricks features such as Unity Catalog, Delta Lake optimizations, Workflow orchestration, and advanced cluster management. The ideal candidate has both data engineering and cloud engineering expertise. Key Responsibilities Databricks Platform Engineering • Build, configure, and maintain Databricks Jobs, Workflows, Pipelines, and Delta Tables. • Develop scalable high-performance ETL pipelines using PySpark, SQL, and Databricks notebooks. • Apply optimization techniques - SQL queries, Delta Lake tables, partitioning strategies, liquid clustering. • Ensure workflows scale efficiently and maintain reliability. • Manage cluster performance, job debugging, resource auto-scaling. • Document workflows, data models • Build, publish and maintain Databricks Dashboards • Apply the Databricks Backup and restore methods based on requirements. Security, Governance & Compliance • Implement Unity Catalog, data lineage, access controls, and governance best practices. • Manage secrets, tokens, and secure integrations. • Ensure compliance with enterprise security and data governance standards. AWS & Databricks Integration • Integrate Databricks with AWS services (S3, EC2, IAM). • Manage roles, secure data lake permissions. • Optimize cluster configuration for cost, performance, and scalability on AWS. Required Skills Technical Skills - Working with Genie, debugging skills, CI/CD experience and good understanding of networking and security practices. Soft Skills - Problem solving, Ownership mindset and communication Databricks certification is preferable Nice-to-Have Skills AI and Machine Learning capabilities in Databricks.