AWS Data Engineer (Pune,Hyderabad,Chennai,Kochi,Kolkata)

Job description

**Key Responsibilities** - Design and develop **scalable data pipelines using Python and PySpark** - Build and maintain **ETL/ELT workflows on AWS platforms** - Work with **AWS services such as EMR, S3, Lambda, SNS, and SQS** - Optimize and debug **Spark jobs using Spark UI and performance tuning techniques** - Develop data ingestion and transformation pipelines for **large-scale datasets** - Write efficient **SQL queries (joins, subqueries, CTEs)** - Ensure **data quality, performance, and scalability** - Collaborate with business and technology teams for requirements and delivery - Participate in **Agile development and sprint cycles** - Contribute to estimation, planning, and project execution **MustHave Skills** **Programming & Big Data** - Strong expertise in: - **Python** - **PySpark / Apache Spark** - Experience handling **large-scale distributed data processing** **AWS Data Platform** - Hands-on experience with: - **AWS EMR** - **S3** - **Lambda, SNS, SQS** - **IAM (basic security concepts)** **Data Engineering** - Strong experience in: - **ETL/ELT pipelines** - Data ingestion frameworks - Good understanding of: - Data processing and transformation **Database & SQL** - Strong knowledge of: - **SQL (joins, subqueries, CTEs)** - Understanding of: - Database technologies **Performance & Debugging** - Experience with: - **Spark optimization and debugging** - Performance tuning techniques **GoodtoHave Skills** - Knowledge of: - **Data Vault / Data Mesh / Data Fabric architectures** - Exposure to: - **CI/CD pipelines for data workflows** - Experience working in: - **Agile environments** - Domain exposure: - **BFSI / Insurance**