Data Engineer (SQL & ETL)

Job description

About WebEngage WebEngage is an enterprise-grade customer engagement and retention platform that helps global brands across industries such as e-commerce, fintech, travel, edtech, gaming, media, and consumer apps. and turn data into measurable revenue impact. Trusted by 800+ brands globally, we have strong presence in India, UAE, KSA, SEA, Europe and beyond. WebEngage powers intelligent, real-time engagement across the entire customer lifecycle. • We are built for scale. • We are built for complexity. • We are built for outcomes. At our core, WebEngage is a full-stack retention operating system that combines: • A powerful Customer Data Platform (CDP) • Real-time behavioral segmentation and intelligence • Omnichannel journey orchestration • AI-driven personalization and recommendations • Deep analytics, experimentation, and revenue attribution • WebEngage BLACK: our AI-native layer that brings Agentic capabilities to engagement. Learn more about us at www.webengage.com About The Role WebEngage’s Data Engineering team is the backbone of our analytics, personalisation, and machine-learning capabilities. As a Data Engineer, you will own the end-to-end lifecycle of data — from ingesting raw event streams and third-party feeds to building well-modelled, highly reliable datasets that power dashboards, predictive models, and customer journey orchestration. This is a high-impact, hands-on role where you will work at the intersection of product, analytics, and backend engineering. You will be expected to think beyond just moving data — designing for data quality, pipeline observability, cost efficiency, and long-term maintainability from day one. Responsibilities Pipeline Development & Reliability • Design, build, and maintain production-grade ETL/ELT pipelines that ingest data from APIs, databases, event streams (Kafka/Pub-Sub), and flat files into the central data warehouse. • Implement idempotent, incremental load patterns with built-in retry logic, dead-letter queues, and SLA-based alerting to ensure zero-data-loss pipelines. • Own pipeline observability — set up data freshness checks, row-count validations, schema drift detection, and anomaly alerts using tools like Great Expectations or dbt tests. Data Modelling & Warehouse Design • Translate business requirements into clean dimensional models (star/snowflake schemas) and maintain a well-documented data catalogue. • Design slowly changing dimensions (SCD Type 1/2), bridge tables, and fact tables optimised for analytical query patterns. • Enforce partitioning, clustering, and materialised view strategies to keep warehouse costs under control while maintaining sub-second query performance. Code Quality & Engineering Best Practices • Write clean, modular, well-tested Python and SQL code. Follow DRY principles, use version control (Git), and participate in peer code reviews. • Build reusable transformation frameworks using dbt or equivalent tooling, with proper documentation and testing at every layer (staging → intermediate → mart). • Containerise data services with Docker and automate deployments via CI/CD pipelines (GitHub Actions / GitLab CI). BI, Visualization & Analytics • Build interactive dashboards and analytical tools using Streamlit, enabling stakeholders to explore metrics, run ad-hoc analyses, and make data-driven decisions without engineering dependency. • Design and maintain BI layers — semantic models, KPI definitions, and pre-aggregated mart tables that serve as the single source of truth for reporting across teams. • Translate raw data into compelling visual narratives using libraries like Plotly, Matplotlib, or Altair; present findings to both technical and non-technical audiences. Collaboration & Communication • Partner with product managers, analysts, and data scientists to understand data needs and proactively identify gaps in current data coverage. • Document data lineage, transformation logic, SLAs, and known limitations in a shared knowledge base to enable self-service analytics. • Contribute to internal engineering guilds, knowledge-sharing sessions, and post-incident reviews for pipeline failures. Requirements • Strong SQL skills with expertise in complex queries, performance optimization, and cost-efficient design on cloud data warehouses (BigQuery, Redshift). • Strong Python scripting for data ingestion, transformation, and validation, with hands-on experience in Pandas, SQLAlchemy, APIs, and automation. • ETL/ELT- End-to-end ownership of data pipelines, including ingestion, transformation, and loading, with understanding of incremental loads, and backfills. • Data Modelling- Ability to design dimensional and transactional data models (star/snowflake, SCDs) and translate business needs into optimized table structures. Good To Have • Airflow, dbt, Docker, CI/CD • GCP/AWS, data warehousing concepts • BI tools / Streamlit / visualization Qualifications • Bachelor’s degree in Computer Science, Engineering, Mathematics, Statistics, or a related quantitative field (or equivalent practical experience). • 1–3 years of professional experience in data engineering, analytics engineering, or a backend role with significant data pipeline work. • Strong understanding of data warehouse architecture — know when to use wide denormalised tables vs. normalised models, and the trade-offs of each. • Familiarity with version control workflows (Git branching strategies, pull requests, code reviews) and agile development practices. • A data quality mindset — you instinctively validate assumptions, add assertions to pipelines, and treat silent data failures as critical incidents. Life At WebEngage • We take transparency very seriously. Along with a full view of team goals, get a top-level view across the board with our monthly & quarterly town hall meetings. • A highly inclusive work culture that promotes a relaxed, creative and productive envi