Walk-in || Databricks - In person Drive - 20th Jun (Sat) - Noida - (4+ years)

Job description

- **Pipeline Development:** Design, build, and optimize scalable ETL/ELT data pipelines using PySpark, Spark SQL, and Databricks Workflows. - **Lakehouse Architecture:** Implement medallion architectures (Bronze, Silver, Gold layers) leveraging Delta Lake for ACID transactions and data versioning. - **Performance Tuning:** Optimize Spark workloads for cost and efficiency using caching, partitioning, and Z-ordering. - **Data Governance:** Enforce security and compliance using Databricks Unity Catalog for fine-grained access control, data masking, and lineage tracking. - **DevOps Integration:** Implement CI/CD best practices using Git, automated testing frameworks, and Infrastructure-as-Code tools like Terraform. **Core Required Skills** - **Programming Languages:** Advanced proficiency in **Python** and **SQL**. - **Data Processing:** Mastery of **PySpark / Spark SQL** for production-grade data manipulation. - **Databricks Stack:** Deep experience with **Delta Lake**, **Unity Catalog**, and **Databricks Workflows/Jobs**. - **Cloud Platforms:** Strong hands-on experience integrating Databricks within cloud ecosystems like **AWS** (S3, Glue) or **Azure** (Azure Data Factory, Azure Synapse). - **DevOps:** Working knowledge of **Git**, CI/CD pipelines, and basic containerization tools. **Preferred Qualifications** - **Certifications:** Databricks Certified Data Engineer (Associate or Professional) or Databricks Certified Developer.