Data bricks Tech Architect

Job description

**Databricks Technical Architect** **Experience : 10- 20 yrs** **Location: PAN India** Required skills: 10 + Years of deep expertise in Databricks Lakehouse, Delta Lake, Unity Catalog and • Strong proficiency in Python, PySpark, Scala SQLand experience with relevant libraries and frameworks (e.g., NumPy, pandas, Fast API). • Design, build, optimize, and operate scalable data pipelines and lakehouse architectures on Databricks, enabling analytics, reporting, and AI/ML use cases in an enterprise environment. - **Lakehouse Architecture:** Design scalable and secure Databricks solutions using Delta Lake patterns like Medallion Architecture. - **Platform Modernization:** Migrate legacy data systems to Databricks Lakehouse on cloud platforms. - **Domain-Oriented Design:** Develop data models and enable cross-platform access with Data Mesh and Lakehouse Federation. - **Technology Roadmap:** Define roadmaps and assess new Databricks and AI features for modernization. - **Spark Optimization:** Optimize PySpark/Spark SQL jobs for better performance and cost efficiency. - **ETL/ELT Pipelines:** Build robust streaming and batch pipelines with Databricks tools. - **CI/CD and DevOps:** Use IaC and DevOps tools to automate deployments. - **Unity Catalog:** Manage centralized governance, access control, and data lineage. - **Compliance and Security:** Set data security standards to meet regulatory requirements. **Data Visualization:** Knowledge of data visualization tools and libraries like Matplotlib, Power BI or Tableau for presenting insights derived from data analysis.