DATA ENGINEER - PySpark

Job description

Key skill set for DE are - pyspark, AWS, glue , Athena ,s3, Lambda ,step function, Kenisis , Pyhton , sql, redshift Mandatory Technical Skills Big Data & Data Engineering • Strong experience in PySpark • Spark SQL, DataFrames, RDDs ,memory optimization • Data Lake Architecture • ETL/ELT Framework Development AWS Services • AWS Glue • Amazon Athena • Amazon Redshift • AWS Lambda • Amazon Kinesis (Data Streams / Firehose) • Amazon S3 • AWS IAM • AWS CloudWatch • AWS Secrets Manager Programming • Python • Advanced SQL Data Warehousing • Star Schema & Snowflake Schema • Dimensional Modeling • SCD Type 1 & Type 2 • Data Mart Design Preferred Skills • Apache Iceberg / Delta Lake / Hudi • AWS Step Functions • Terraform / CloudFormation • CI/CD using GitHub Actions, Jenkins, or AWS CodePipeline • Airflow or equivalent orchestration tools • Power BI / Tableau • Healthcare domain knowledge is a plus PySpark