Senior Data Engineer

Job description

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth. If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi! JOB ROLE - Senior Data Engineer As a Senior Data Engineer, you will be responsible for designing, developing, and optimizing scalable cloud-native data platforms on AWS. You will build high-performance batch and analytical data pipelines using Spark/PySpark, Apache Iceberg, and AWS services, while ensuring reliability, performance, and operational excellence. The role requires strong hands-on expertise in data engineering, SQL optimization, distributed data processing, and cloud-native architectures. Must have skills: • 3+ years of hands-on experience building large-scale AWS data engineering solutions. • Strong hands-on expertise in SQL (analytical queries, window functions, stored procedures), Spark/PySpark, and Python. • Hands-on experience designing and implementing Lakehouse solutions using Apache Iceberg. • Strong knowledge of AWS services including EMR, S3, Athena, Glue Catalog, Aurora PostgreSQL, Lambda, CloudWatch, SQS, SNS, EventBridge, and IAM. • Experience building scalable batch and streaming data pipelines on AWS. • Strong experience in Spark/PySpark performance tuning and optimization. • Hands-on experience optimizing Apache Iceberg and Aurora PostgreSQL for performance and scalability. • Strong understanding of data modeling, partitioning, file formats, and Lakehouse/Data Lake architectures. • Experience implementing monitoring, logging, alerting, and observability for production data pipelines. • Strong knowledge of data quality, troubleshooting, debugging, and production support. • Experience with CI/CD, Git, and data engineering best practices. • Strong communication, problem-solving, and stakeholder management skills. • AWS Data Engineer Associate or equivalent AWS Certification is preferred. Good to have skills : • Experience with Apache Airflow or AWS Step Functions. • Experience with Terraform or Infrastructure as Code. • Exposure to Kafka, Hive, HDFS or other Big Data technologies. • Hands-on experience using GenAI-assisted development tools such as Kiro, Copilot,Cursor, or similar AI coding assistants to improve engineering productivity. • Experience with ClickHouse Database • Telecom/Mobile Network domain knowledge is preferred. If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us*!*