Senior Data Engineer

Job description

**Senior Data Engineer** **Location:** Gurgaon, Sector 32 **Experience:** 3-6 Years **Job Summary** We are seeking a highly skilled **Senior Data Engineer** with strong expertise in **Python, Kafka, Spark, AWS, and distributed data systems**. The ideal candidate will be responsible for building scalable batch and streaming data pipelines, designing cloud-native data platforms, and enabling reliable data solutions for analytics and business applications. **Key Responsibilities** - Design, build, and maintain scalable batch and real-time data pipelines. - Develop streaming solutions using **Kafka** and **Spark Structured Streaming**. - Implement **CDC (Change Data Capture)** pipelines using **Debezium**. - Build and manage Data Lake architectures on AWS. - Orchestrate workflows using **Apache Airflow** and **Prefect**. - Write high-quality, production-grade code in **Python**. - Ensure data quality, monitoring, observability, and reliability. - Deploy and manage applications using **Docker** and **Kubernetes**. - Participate in **HLD/LLD**, architecture discussions, and performance optimization. **Required Skills** - Strong expertise in **Python** - **Apache Kafka** - **Apache Spark** (Batch & Streaming) - **AWS** (S3, EC2, IAM, EMR, Glue, EKS) - **CDC using Debezium** - **Apache Airflow** - **Prefect** - Strong **SQL** - **Data Structures & Algorithms (DSA)** - **Data Lake** architectures - **Docker & Kubernetes** - **HLD/LLD and System Design** - Experience building **streaming data pipelines** **Good to Have** - Apache Superset or similar BI tools - Data observability, monitoring, and alerting - ML data pipelines and feature engineering - Iceberg, Delta Lake, or Hudi **Education** **B.Tech / B.E.** in Computer Science, Information Technology, or a related engineering discipline.