AWS Data Engineer

Job description

Key Skills: IAM, AWS, Python, AWS Lambda, Data Engineer, AWS S3, Spark Roles and Responsibilities: • Design, build, and maintain scalable data pipelines on AWS. • Develop ETL/ELT workflows using AWS Glue, Lambda, EMR, and Step Functions. • Develop data ingestion frameworks using Kinesis, Kafka, and SQS. • Build and optimize data lakes using S3, Glue Data Catalog, and Lake Formation. • Develop and maintain data warehousing solutions using Redshift and Snowflake. • Implement data quality checks, monitoring, logging, and CI/CD deployments. • Collaborate with analytics and business teams to deliver high-quality datasets. • Design, develop, and deploy solutions using appropriate tools, design principles, and coding standards. • Configure robotics processes and objects using core workflow principles, ensuring solutions are maintainable and easy to understand. • Understand existing processes and manage changing requirements through structured change control processes. • Troubleshoot day-to-day issues in data and automation processes and provide timely resolutions. • Maintain proper documentation for solutions, test procedures, and scenarios throughout UAT and Production phases. • Coordinate with process owners and business stakeholders to understand as-is processes and design appropriate automation workflows. Skills Required: • 6+ years of strong hands-on experience with AWS Glue, S3, Lambda, IAM, and Step Functions. • Strong proficiency in Python, PySpark, SQL, and data transformation logic. • Experience with Redshift, Athena, and Glue Data Catalog. • Strong knowledge of Apache Spark and distributed data processing using EMR/Glue. • Experience with data ingestion technologies such as Kinesis, Kafka, or SQS. • Good understanding of data modeling, data lake architecture, and data engineering best practices. • Experience implementing data quality, monitoring, logging, and CI/CD processes. • Familiarity with Terraform or CloudFormation for Infrastructure as Code (IaC) is an added advantage. • Strong problem-solving and analytical skills. • Good communication and stakeholder management skills. Education: Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.