ETL & DevOps Engineer (AWS)

Job description

**Job Title:** ETL & DevOps Engineer (AWS) **Location:** Delhi / Pune **Experience:** 710 Years **Employment Type:** Full-Time **Role Overview** We are looking for a highly skilled and versatile **ETL & DevOps Engineer** who can design, develop, and maintain robust data pipelines while ensuring efficient deployment and operational processes. This role bridges **Data Engineering** and **DevOps**, requiring expertise in **ETL development, cloud infrastructure, automation, and CI/CD practices**. The ideal candidate should be able to handle end-to-end data workflows along with infrastructure and deployment automation in an AWS environment. **Key Responsibilities** **ETL & Data Engineering Responsibilities** - Design, develop, and maintain scalable **ETL workflows using AWS Glue** for data integration and transformation - Build data pipelines to ingest, process, and transform large volumes of structured and unstructured data - Optimize ETL jobs for **performance, scalability, and cost efficiency** - Ensure **data quality, integrity, and validation** across pipelines - Develop and maintain complex SQL queries and stored procedures - Work with data models and implement **data warehousing best practices** - Troubleshoot and resolve data-related issues and pipeline failures **DevOps & Cloud Responsibilities** - Design, implement, and maintain **CI/CD pipelines** for ETL workflows and application deployments - Automate infrastructure provisioning using **AWS CloudFormation** (or similar IaC tools) - Manage and monitor AWS infrastructure to ensure high availability and performance - Implement **logging, monitoring, and alerting** mechanisms using AWS tools (e.g., CloudWatch) - Automate routine operational tasks to improve system reliability and efficiency - Collaborate with development and operations teams to streamline deployment processes **Required Skills & Qualifications** **Education** - Bachelor’s degree in Computer Science, Engineering, or a related field **Technical Skills** - Strong hands-on experience with ETL tools such as: - **AWS Glue (preferred)** - Informatica, Talend, or similar tools - Expertise in **SQL** for data extraction, transformation, and analysis - Proficiency in: - **Python** - **PySpark** - Strong understanding of **data modeling techniques** (OLTP, OLAP, star/snowflake schemas) **Cloud & DevOps** - Hands-on experience with **AWS services**, including: - AWS Glue - S3 - CloudFormation - IAM - CloudWatch - Experience with **CI/CD tools and practices** (e.g., CodePipeline, Jenkins, GitHub Actions) - Knowledge of infrastructure automation and version control systems (Git) **Data & Analytics** - Good understanding of **data warehousing concepts** - Experience working with **BI tools** (e.g., Power BI, Tableau, etc.) **Nice to Have** - Experience with **Spark/Databricks ecosystem** - Knowledge of **Airflow or other orchestration tools** - Exposure to **containerization (Docker/Kubernetes)** - Understanding of **data governance and security best practices** **Soft Skills** - Strong analytical and problem-solving abilities - Effective communication and collaboration skills - Ability to work in cross-functional teams - Proactive approach to troubleshooting and optimization **Key Deliverables** - High-performing, scalable ETL pipelines - Reliable CI/CD pipelines and automated deployment processes - Secure and optimized AWS infrastructure - Clean, trusted, and business-ready data