AWS Spark Scala Data Engineer

Tata Consultancy Services · Bengaluru, Karnataka, India - Chennai, Tamil Nadu, India - Pune, Maharashtra, India

3–10 yrs experiencefull_timePosted 1w ago

Job description

**Desired Competencies (Technical/Behavioral Competency)** - Strong Expertise in AWS Data Engineering - Hands-on expertise in Spark and Scala - Extensive experience with AWS Cloud Services (S3, EMR, Glue, Redshift, Athena etc) - Solid understanding of Distributed computing - Proficiency in SQL **Experience in Data Pipeline Orchestration tools (Airflow, Step Functions etc)** **Responsibility of / Expectations from the Role** - Design Develop and maintain Scalable Data Pipelines using Apache Spark (Scala) - Build and Optimize ETL/ELT workflows on AWS - Work with large scale structured and unstructured Datasets - Develop Data Solutions using AWS services such as S3, EMR, Glue, Redshift, Athena,Lambda - Ensure Data Security, Governance and Compliance Best Practices