DATA ENGINEER - PySpark

Happiest Minds Technologies · Bengaluru, Karnataka, India

3–10 yrs experiencefull_timePosted 1w ago

Job description

Key skill set for DE are - pyspark, AWS, glue , Athena ,s3, Lambda ,step function, Kenisis , Pyhton , sql, redshift Mandatory Technical Skills Big Data & Data Engineering - Strong experience in PySpark - Spark SQL, DataFrames, RDDs ,memory optimization - Data Lake Architecture - ETL/ELT Framework Development AWS Services - AWS Glue - Amazon Athena - Amazon Redshift - AWS Lambda - Amazon Kinesis (Data Streams / Firehose) - Amazon S3 - AWS IAM - AWS CloudWatch - AWS Secrets Manager Programming - Python - Advanced SQL Data Warehousing - Star Schema & Snowflake Schema - Dimensional Modeling - SCD Type 1 & Type 2 - Data Mart Design **Preferred Skills** - Apache Iceberg / Delta Lake / Hudi - AWS Step Functions - Terraform / CloudFormation - CI/CD using GitHub Actions, Jenkins, or AWS CodePipeline - Airflow or equivalent orchestration tools - Power BI / Tableau - Healthcare domain knowledge is a plus PySpark