AWS DevOps Engineer

Job description

Data Engineer | ZS Associates (Payroll: RSquareSoft Technologies) Worked as a Data Engineer supporting the Integrated Data & Analytics Platform (iDNA), an enterprise-scale commercial data lake integrating data from 90+ sources. Responsible for managing AWS S3 data landing zones, orchestrating ETL pipelines using Apache Airflow, and developing scalable data transformation workflows using PySpark, SQL, and Databricks. Key responsibilities included data ingestion, transformation, validation, reconciliation, and data-quality checks across multiple business domains such as Customer, Product, Alignment, Sales, Activity, Contract, and Claims. Managed Databricks environments and Delta tables and supported the availability of curated datasets for analytics and reporting teams. Also handled production support, pipeline monitoring, troubleshooting, root-cause analysis, and performance optimization. Supported deployments across Development, QA, UAT, and Production environments and maintained GitLab-based deployment workflows. Used Python and Shell scripting to automate operational activities and collaborated with Data Engineering, Analytics, and Business teams for data onboarding and production enhancements. Technology Stack: AWS S3, Databricks, PySpark, SQL, Python, Apache Airflow, Delta Lake, GitLab, CI/CD, Terraform, Linux, Shell Scripting, Docker, Kubernetes, Grafana, Prometheus, and AWS CloudWatch. Pay: ₹1,899,137.43 - ₹2,812,653.30 per year Work Location: In person