GenAI Testing Consultant/Architect

Job description

- Experience architecting Cloud AI solutions, including scalable inference, data security patterns, and cost/performance optimization. - Hands-on experience with LLM application patterns such as RAG, tool/function calling, prompt engineering, and automated evaluation frameworks. - Strong MLOps/LLMOps expertise: experiment tracking, model registry, CI/CD, observability, drift detection, and incident response for AI services. - Experience with data engineering concepts (feature stores, batch/stream processing) to support ML and GenAI workloads. - Proven ability to lead architecture governance, create reference architectures, and drive adoption across teams and portfolios. - Lead discovery workshops to identify high-value GenAI and ML use cases, define success metrics (standard and custom), and create solution roadmaps. - Architect end-to-end Generative AI systems, including data pipelines, model selection, orchestration, evaluation, and deployment patterns using Infosys and industry-standard tools and platforms. - Design and implement ML/AI solutions using Python (or other relevant Data Science/AI languages), applying strong software engineering practices for maintainability, performance, and reliability. - Build MCP and agentic-based solutions using appropriate frameworks, enabling tool/function calling, workflow orchestration, and multi-step reasoning patterns aligned to business needs. - Test and validate GenAI applications using relevant tools and frameworks; capture, analyze, and report standard and custom quality metrics (e.g., relevance, groundedness, toxicity, latency, cost) and drive continuous improvement. - Collaborate with cloud and platform teams to design scalable Cloud AI architectures and cost-efficient deployments across AWS, Azure, or GCP. - Explore and evaluate new technologies, tools, and testing methodologies to improve development processes and solution quality; stay up-to-date with advancements in Generative AI and evaluation practices. Good to have skills: Agile methodologies, CI/CD pipelines, API testing, SQL/Database basics Location: BANGALORE, PUNE, HYDERABAD, CHENNAI, TVM, INDORE, NAGPUR, MLR, MYS, NOIDA, BBSR, KOLKATA, COIMBATORE, HUBLI