Azure Databricks-Data Engineer

Job description

Position: Data Bricks-Data Engineer Experience: 2 to 5 years Location: Hyderabad Employment Type: Full-Time Roles & Responsibilities • Design, develop, and maintain scalable data pipelines using Databricks. • Build and optimize ETL/ELT processes for large-scale data ingestion and transformation. • Develop Spark and PySpark applications for batch and real-time processing. • Implement Lakehouse architecture using Delta Lake. • Create and maintain data models, tables, and data warehouses. • Optimize Databricks jobs, Spark performance, and query execution. • Integrate data from multiple cloud and on-premises sources. • Ensure data quality, governance, and security standards. • Work with Unity Catalog for data governance and access management. • Automate deployments using CI/CD and Infrastructure as Code (Terraform). • Monitor and troubleshoot production data pipelines. • Collaborate with Data Scientists, BI Developers, and Business Stakeholders. • Support cloud migration and data modernization initiatives.