Assistant Manager - Data Engineering

Job description

Designation : Assistant Manager - Data Engineering Level : L3 Location : Chennai, Tamil Nadu , India Experience : 7 to 10 years Job Role : We are looking for a Lead Data Engineer to drive the design, development, and scaling of our enterprise modern data platform. In this role, you will be the technical authority for our core data stack—leveraging Azure as our cloud backbone, Snowflake for data warehousing, and dbt (data build tool) for transformations and data modeling.  You will manage a team of data engineers, direct data architecture strategies, and establish rigorous software engineering practices (CI/CD, automated testing, version control) across our data pipelines. Responsibilities : 1. Data Platform Architecture & Strategy Design and execute a scalable, cost-efficient ELT architecture using Azure Data Factory/Event Hubs, Snowflake, and dbt Cloud/Core. Establish medallion architecture standards (Bronze/Silver/Gold layers) or Kimball dimensional modeling across raw, transformed, and analytics-ready marts. Drive platform performance tuning, capacity planning, resource monitoring, and Snowflake warehouse cost optimization (FinOps). 2. Core Pipeline Engineering & Transformation Develop modular, reusable, and tested dbt models using advanced features (macros, packages, custom materializations, state-based incremental loads). Configure ingestion pipelines from Azure Data Lake Storage (ADLS Gen2) into Snowflake via Azure Event Grid, Snowpipe, or Auto-ingest tasks. Extend ELT capabilities using Python/Snowpark for unstructured data processing and complex transformations. 3. Data Engineering Best Practices & DevOps Implement continuous integration and delivery (CI/CD) pipelines using Azure DevOps / GitHub Actions for dbt code deployment and Snowflake zero-downtime schema migrations. Establish automated data quality testing, lineage, and documentation via dbt tests, Great Expectations, or integrated observability platforms (e.g., Monte Carlo). Enforce security and governance standards: Role-Based Access Control (RBAC) in Snowflake, dynamic data masking, row-level security, and PII protection. 4. Leadership & Stakeholder Collaboration Lead and mentor a team of 4–8 Data Engineers through code reviews, architectural blueprints, and agile execution. Collaborate with Data Architects, Analytics Engineers, BI Leads, and Business Stakeholders to translate analytics needs into robust data platform features. Required Qualifications Technical Must-Haves: 8+ years of total experience in Data Engineering, Data Warehousing, or Analytics Engineering. 3+ years of hands-on expertise building and managing Snowflake (Clustering, Tasks, Streams, Snowpipe, Zero-Copy Clone, Resource Monitors). 3+ years of expert-level dbt experience (productionizing dbt Cloud/Core, macro writing, custom test packages, semantic layer configuration). 3+ years working with Microsoft Azure services (ADLS Gen2, Azure Data Factory, Azure Key Vault, Azure Active Directory / Entra ID, Azure DevOps). Advanced SQL proficiency (window functions, query profiling, complex optimizations) and strong Python/PySpark development skills. Demonstrated experience with CI/CD tools (Git, GitHub, Azure DevOps) and infrastructure automation. Required skills :  Azure, ADF, snowflake, DBT, fivetran, SQL, Python,  Pyspark, ETL