Senior Data Engineer

Job description

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth. If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi! **JOB ROLE - Senior Data Engineer** **As a** **Senior Data Engineer**, you will be responsible for designing, developing, and optimizing scalable cloud-native data platforms on AWS. You will build high-performance batch and analytical data pipelines using **Spark/PySpark**, **Apache Iceberg**, and AWS services, while ensuring reliability, performance, and operational excellence. The role requires strong hands-on expertise in data engineering, SQL optimization, distributed data processing, and cloud-native architectures. **Must have skills:** - **3+ years** of hands-on experience building large-scale AWS data engineering solutions. - **Strong hands-on expertise in** **SQL** (analytical queries, window functions, stored procedures), **Spark/PySpark, and Python.** - **Hands-on experience designing and implementing Lakehouse solutions using** **Apache Iceberg**. - **Strong knowledge of AWS services including** **EMR, S3, Athena, Glue Catalog, Aurora PostgreSQL, Lambda, CloudWatch, SQS, SNS, EventBridge, and IAM**. - **Experience building scalable** **batch and streaming** data pipelines on AWS. - **Strong experience in Spark/PySpark** **performance tuning and optimization.** - **Hands-on experience optimizing** **Apache Iceberg and Aurora PostgreSQL** for performance and scalability. - **Strong understanding of data modeling, partitioning, file formats, and** **Lakehouse/Data Lake** architectures. - **Experience implementing monitoring, logging, alerting, and observability for production data pipelines.** - **Strong knowledge of data quality, troubleshooting, debugging, and production support.** - **Experience with** **CI/CD, Git,** and data engineering **best practices.** - **Strong communication, problem-solving, and stakeholder management skills.** - **AWS Data Engineer Associate or equivalent** **AWS Certification** is preferred. **Good to have skills :** - **Experience with** **Apache Airflow or AWS Step Functions.** - **Experience with** **Terraform or Infrastructure as Code.** - **Exposure to** **Kafka, Hive, HDFS or other Big Data** technologies. - **Hands-on experience using** **GenAI-assisted development tools** such as Kiro, Copilot,Cursor, or similar AI coding assistants to improve engineering productivity. - **Experience with** **ClickHouse** Database - **Telecom/Mobile Network domain knowledge is preferred.** *If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us**!*