Associate Data Engineer

Job description

**Responsibilities-** - Develop, maintain, and optimize ETL/ELT pipelines using **Databricks, PySpark, and SQL**. - Create, schedule, monitor, and troubleshoot **Databricks Jobs, Workflows, and Pipelines**. - Design and implement scalable data processing solutions for large datasets. - Perform data cleansing, transformation, validation, and quality checks. - Collaborate with business and analytics teams to understand requirements and deliver data solutions. - Support production deployments, monitoring, and issue resolution. - Follow best practices for code versioning, documentation, and performance optimization. **Educational Qualifications** - Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field. **Competencies Required:** - 2–3 years of experience in Data Engineering. - Strong proficiency in **SQL** and **PySpark**. - Hands-on experience with **Azure Databricks**. - Experience in creating and managing **Databricks Jobs, Workflows, and Pipelines**. - Good understanding of ETL/ELT processes and data warehousing concepts. - Familiarity with Git/version control and basic CI/CD practices. - Strong analytical and problem-solving skills. **Experience -** - Experience with **Azure Data Factory (ADF), ADLS, Synapse, or other Azure services**. - Exposure to Delta Lake and Lakehouse architecture. - Knowledge of performance tuning and optimization of Spark workloads.