Data bricks Lead Data Engineer – R01565550

Job description

# **Data bricks Lead Data Engineer - R01565550** **Lead Data Engineer** ###### **Primary Skills** - ETL Fundamentals, SQL, SQL (Basic + Advanced), Python, Data Warehousing, Modern Data Platform Fundamentals, Data Modelling Fundamentals, PLSQL, T-SQL, Stored Procedures ###### **Specialization** - Databricks Engineering: Data Engineer ###### **Job requirements** **Job Description – Data Engineer (Databricks)** **Company:** Brillio Technologies **Role:** Data Engineer – Databricks **Experience:** 5–10 Years **Location:** India (Hybrid/3 days) **Employment Type:** Full-time **About Brillio** Brillio is a leading digital transformation partner delivering innovative solutions across Data, Cloud, AI, and Digital Engineering. We help global enterprises unlock value through cutting-edge technology and data-driven insights. **Role Overview** We are looking for a **hands-on Data Engineer with strong Databricks expertise** to design, build, and scale modern data platforms. The ideal candidate will have experience in **PySpark, Delta Lake, and cloud-native data engineering** and will work closely with analytics, product, and business teams to deliver high-performance data pipelines. **Key Responsibilities** - Design, develop, and maintain **scalable ETL/ELT data pipelines** using Databricks and PySpark - Build and optimize **data lakehouse architectures (Bronze–Silver–Gold layers)** - Develop data ingestion frameworks from multiple sources (batch & real-time) - Implement **Delta Lake features** including schema evolution, ACID transactions, and optimization - Create and manage **Databricks notebooks, workflows, and jobs** for production pipelines - Ensure **data quality, governance, and lineage** across pipelines - Optimize Spark jobs for performance (partitioning, caching, tuning) - Collaborate with cross-functional teams (Data Scientists, Analysts, Architects) - Support production deployments, monitoring, and troubleshooting of data pipelines - Contribute to migration of legacy data platforms to modern cloud-based architecture **Technical Skills (Must-Have)** - Strong experience in: - - **Databricks (Workflows, Notebooks, Jobs)** - **PySpark / Spark SQL** - **SQL & Data Warehousing concepts** - Hands-on experience with: - - **Delta Lake / Lakehouse architecture** - **Data pipeline development & orchestration** - Good understanding of: - - **Data modeling, ETL/ELT processes, SCD concepts** **Cloud & Tools** - Experience in at least one: - - **AWS (S3, Glue, EMR) / Azure (ADF, ADLS, Synapse)** - Workflow tools: - - Airflow / Databricks Workflows - Version control: - - Git / CI-CD pipelines **Good to Have** - Experience with: - - **Streaming frameworks (Kafka / Spark Streaming)** - **DBT / Snowflake / Redshift / Fabric** - Exposure to: - - **Unity Catalog, Autoloader, Delta Live Tables** **Experience Required** - **5–10 years of experience** in Data Engineering - Minimum **2–4 years of hands-on Databricks experience** - Proven experience building **enterprise-scale data platforms and pipelines** **Soft Skills** - Strong problem-solving and analytical skills - Ability to work in a **fast-paced, agile environment** - Good communication and stakeholder management **Why Join Brillio** - Work on **cutting-edge data & AI programs** - Opportunity to build **large-scale data platforms** - Collaborative and innovation-driven culture