Data Engineer (AWS+Pyspark)

Job description

**Data Engineer (AWS + pySpark)** - Having 5+ years of relevant experience, which includes hands on experience in Big Data technologies. - Mandatory - Hands on experience in Python and PySpark. - Build pySpark applications using Spark Dataframes in Python. - Worked on optimizing spark jobs that processes huge volumes of data. - Hands on experience in version control tools like Git. - Worked on Amazon’s Analytics services like Amazon EMR, Amazon Athena, AWS Glue. - Worked on Amazon’s Compute services like Amazon Lambda, Amazon EC2 and Amazon’s Storage service like S3 and few other services like SNS. - Good to have knowledge of datawarehousing concepts – dimensions, facts, schemas- snowflake, star etc. - Have worked with columnar storage formats - Parquet etc. Well versed with compression techniques – Snappy, Gzip. - Good to have knowledge of AWS databases (atleast one) Aurora, RDS, Redshift, ElastiCache, DynamoDB. am