Pyspark Data Engineer

Job description

Walk-in Drive | PySpark Data Engineer | Bangalore | 8th August **Are you passionate about Big Data, PySpark, and AWS technologies?** Join us for an exciting **Walk-in Drive** and be a part of a dynamic Data Engineering team working on cutting-edge cloud and big data solutions. Position: PySpark Data Engineer **Location:** Bangalore **Walk-in Date:** 8th August **Experience Required:** 5 - 10 Years Required Technical Skills - Python Programming - PySpark - Big Data Technologies - Hadoop Ecosystem - Hive, Impala - SQL - AWS Services: - EMR - S3 - IAM - Lambda - SNS - SQS - Redshift - Unix/Linux - HDFS Commands Key Responsibilities - Design, develop, and optimize scalable data pipelines using PySpark and Big Data technologies. - Work on enterprise-level Data Engineering and Data Analytics projects. - Develop and maintain ETL processes for large-scale datasets. - Analyze and troubleshoot Spark jobs using Spark UI and performance tuning techniques. - Write efficient SQL queries involving joins, subqueries, CTEs, and complex data transformations. - Collaborate with cross-functional teams to understand business requirements and deliver data solutions. - Work with AWS cloud services including EMR, S3, Lambda, SNS, SQS, IAM, and Redshift. - Implement best practices for data quality, governance, and performance optimization. Must-Have Skills Strong experience in: - Hadoop - Spark / PySpark - Python - Hive - Impala - SQL - Data Engineering Projects - Unix/Linux Environment - HDFS Commands Good-to-Have Skills Experience with: - Spark UI Monitoring and Optimization - Performance Tuning & Debugging - Python Scripting - Advanced SQL Concepts - Joins - Subqueries - CTEs - Database Technologies - AWS Cloud Services (EMR, S3, IAM, Lambda, SNS, SQS, Redshift)