Data bricks engineer

Job description

Role & responsibilities • Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and PySpark. • Develop data processing solutions using Python and Spark for large-scale data transformation. • Write complex SQL queries for data extraction, transformation, validation, and reporting. • Build and optimize data workflows to improve performance, scalability, and reliability. • Work with structured and semi-structured data from multiple data sources. • Collaborate with data analysts, business stakeholders, and development teams to understand data requirements. • Perform data quality checks, troubleshooting, and root cause analysis for data issues. • Optimize Spark jobs and SQL queries for improved performance. • Follow coding standards, documentation, and best practices for data engineering. • Participate in code reviews and support production deployments. Required Skills • Minimum 3 years of experience in Data Engineering. • Strong hands-on experience with Databricks. • Good experience in PySpark and Apache Spark. • Strong programming skills in Python. • Proficiency in SQL, including complex queries, joins, window functions, and performance tuning. • Experience in developing ETL/ELT pipelines. • Knowledge of data warehousing concepts and data modeling. • Experience with Git or other version control systems. • Strong analytical, problem-solving, and debugging skills. Preferred candidate profile