Senior Test Engineer - AI/ML

Job description

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth. If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi! **Role:** Senior Test Engineer - AI/ML **Experience:** 3-6 years **Location:** Mumbai/Bangalore (Hybrid) **Key Responsibilities:** - Design, develop, and maintain UI and API automation test suites using modern automation frameworks. - Independently own end-to-end testing activities including test planning, automation development, execution, defect management, and reporting. - Validate AI/ML applications, Generative AI solutions, LLM-powered applications, and intelligent workflows. - Design and execute functional, integration, regression, and end-to-end test automation for traditional and AI-enabled applications. - Develop and maintain reusable automation frameworks, libraries, and utilities. - Automate API testing using REST Assured (Java) or Python Requests. - Develop UI automation using Playwright. - Validate backend data using SQL and perform database verification. - Develop automated validations for AI model outputs, prompt responses, RAG pipelines, and chatbot conversations. - Validate AI responses for correctness, consistency, relevance, grounding, safety, hallucinations, and bias. - Collaborate with ML Engineers, Software Engineers, and Product Owners to define AI quality validation strategies. - Integrate automated test suites into CI/CD pipelines and implement quality gates. - Leverage GenAI tools to accelerate test case generation, automation development, debugging, documentation, and productivity. - Contribute to AI testing frameworks, accelerators, and Quality Engineering best practices. - Mentor junior engineers and promote automation-first and AI-first Quality Engineering practices. **Must have skills:** - Strong understanding of Software Testing Life Cycle (STLC), SDLC, Agile, and Scrum methodologies. - Hands-on experience in designing test strategies, test scenarios, test cases, and test execution. - Strong expertise in UI automation using Playwright and API automation using REST Assured (Java) or Python Requests. - Hands-on experience with Postman for API testing and validation. - Strong SQL skills with hands-on experience in database testing and backend data validation. - Experience integrating automation suites with CI/CD pipelines using Jenkins, GitHub Actions, or similar. - Proficiency with Git and version control best practices. - Hands-on experience testing AI/ML or Generative AI applications. - Understanding of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), chatbots, copilots, or AI agents. - Experience validating AI responses for accuracy, relevance, consistency, hallucinations, toxicity, bias, and safety. - Experience designing test scenarios for prompt engineering and AI workflow validation. - Understanding of AI evaluation techniques and automated output validation. - Familiarity with AI evaluation frameworks or libraries (e.g., DeepEval, Ragas, LangSmith, Promptfoo, MLflow, or similar). - Experience using GenAI tools such as ChatGPT, GitHub Copilot, Kiro, Claude, or similar to accelerate test design, automation development, debugging, test data generation, and documentation. - Strong debugging, root cause analysis, and analytical problem-solving skills. - Ability to independently own end-to-end testing activities with minimal supervision. - Excellent communication, collaboration, and stakeholder management skills. - Ability to mentor junior engineers and promote automation and Quality Engineering best practices. - Strong attention to detail with a quality-first and automation-first mindset. **Nice to have skills** - **Knowledge of AWS services such as Amazon Bedrock, SageMaker, EC2, Lambda, API Gateway, S3, RDS, DynamoDB, CloudWatch, ECS/EKS, and IAM.** - **Experience testing cloud-native applications deployed on AWS.** - **Experience in Data Warehouse, ETL, or Data Pipeline testing.** - **Experience with performance testing tools such as JMeter or k6.** - **Exposure to Docker, Kubernetes, and containerized application testing.** - **Relevant certifications in Playwright, AWS, Quality Engineering, or Agile are an added advantage.** *If you like wild growth and working with happy, enthusiastic over-achievers, you'll enjoy your career with us**!*