L

Data Architect – Data Engineering Manager

LatentView Analytics · Bengaluru, Karnataka, India

8–15 yrs experiencefull_timePosted 2w ago
Apply now →

Job description

Role Overview We are looking for a Data Architect to lead the design and evolution of our enterprise data platform. You will own end-to-end architecture decisions across ingestion, processing, storage, sharing, and governance — working primarily on Databricks and Azure, with cross-platform responsibilities spanning Snowflake and data governance tooling. Key Responsibilities - Architect and govern a multi-layer data platform (Landing → Bronze → Silver → Gold → Platinum) on Databricks and ADLS - Design and enforce data governance frameworks using **Databricks Unity Catalog** — including access control, data lineage, tagging, and metadata management - Evaluate and recommend governance and cataloguing tools (e.g. Alation, Purview, Collibra) and lead implementation - Design secure data sharing architectures for internal and external consumers across Databricks and Snowflake — including Delta Sharing, Iceberg/UniForm catalog integration, and direct write patterns - Define and maintain data models across medallion layers; enforce modelling standards (dimensional, data vault, or domain-driven as appropriate) - Own orchestration design — workflow dependencies, SLA management, and failure handling (Databricks Workflows, ADF, or equivalent) - Partner with platform and DevOps teams on CI/CD pipelines for data assets (e.g. Databricks Asset Bundles, Azure DevOps) - Evaluate emerging tools and technologies, produce options assessments with clear rationale, phasing, and trade-offs - Translate business requirements into scalable, governed, and cost-efficient platform architectures. Must-Have Skills - Databricks: deep expertise including Unity Catalog, Delta Lake, Workflows, Asset Bundles, and compute governance - Azure: strong working knowledge of ADLS Gen2, Azure DevOps, Key Vault, Entra ID / Service Principals, and Azure networking fundamentals - Data modelling: proficient across conceptual, logical, and physical modelling; experience with medallion/lakehouse patterns - Orchestration: hands-on experience designing and managing complex pipelines and dependency chains - Data sharing: understanding of Delta Sharing, Iceberg, and cross-platform data access patterns Good-to-Have Skills - Snowflake: Working knowledge of Snowflake architecture, catalog integrations, and external table patterns; familiarity with Synapse or Redshift is a plus - Governance tooling: hands-on experience with Unity Catalog/Alation/Collibra/Microsoft Purview, or equivalent - Security & compliance: experience applying RBAC, column/row-level security, and data classification at scale - Python / SQL / YAML: comfortable reading and writing automation, pipeline config, and transformation logic We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, sex, gender, gender expression, sexual orientation, age, marital status, veteran status, or disability status.