Senior Principal Consultant Databricks Tech Lead

Job description

**Role & responsibilities** Key Responsibilities: 1. Technical Leadership: Provide technical guidance and mentorship to data engineering teams fostering a culture of innovation and continuous improvement. 2. Solution Architecture: Design and architect scalable high-performance data solutions on Databricks leveraging components like Delta Lake Databricks Workflows Databricks SQL Databricks Machine Learning and MLflow to meet client requirements. 3. Client Engagement: Serve as the primary technical point of contact for clients understanding their business needs and translating them into effective data solutions. 4. Project Management: Lead and manage end-to-end project delivery ensuring projects are completed on time within budget and to the highest quality standards. 5. Data Modeling: Develop and maintain comprehensive data models that support analytics reporting and machine learning use cases. 6. Performance Optimization: Continuously monitor and optimize data pipelines and systems within the Databricks environment to ensure optimal performance and cost-efficiency. 7. Technology Evangelism: Stay abreast of the latest Databricks technologies and industry trends and advocate for their adoption within the organization. 8. Team Collaboration: Foster strong collaboration with data scientists analysts and other stakeholders to ensure alignment and successful project outcomes. Qualifications we seek in you! Minimum Qualifications - Bachelor's or Master's degree in Computer Science Engineering or a related technical field - experience in data engineering with experience in technical leadership role - Deep expertise in Databricks technologies and data services - Proven track record in architecting and delivering complex data solutions on Databricks - Strong leadership communication and client-facing skills - Experience in data modeling ETL processes and data warehousing - Excellent problem-solving analytical and decision-making skills Technical Skills and Expertise: - Databricks Components: Deep understanding of core Databricks components for data engineering including Delta Lake Databricks Workflows Databricks Jobs Databricks SQL Databricks Machine Learning MLflow Unity Catalog Databricks Repos and Databricks Notebooks - Programming Languages: Proficiency in Python SQL PySpark and Spark for data processing transformation and analysis - Data Warehousing and ETL: Expertise in designing and implementing data warehousing solutions ETL processes and data modeling techniques within the Databricks Lakehouse architecture - Machine Learning: Familiarity with Databricks Machine Learning and MLflow for model development tracking and deployment - Big Data Technologies: Expertise in big data frameworks like Apache Spark for handling largescale data processing within the Databricks environment