AI Analyst

Job description

**Role & responsibilities** - Design, build, and optimize enterprise Generative AI applications using Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG). - Validate AI-generated responses against business data sources (e.g., Looker Explore/semantic models) to ensure factual accuracy and semantic alignment. - Develop and optimize prompt engineering strategies to improve response quality, consistency, and reduce hallucinations. - Build and enhance RAG pipelines using vector databases, embeddings, semantic search, and retrieval optimization techniques. - Implement AI evaluation frameworks to measure response accuracy, relevance, faithfulness, latency, and overall LLM performance. - Collaborate with data engineering, analytics, and business teams to integrate AI capabilities into enterprise applications. - Develop scalable backend services and APIs using Python and FastAPI for AI solutions. - Monitor production AI systems using observability tools, perform root cause analysis, and continuously improve model performance. - Ensure AI solutions follow Responsible AI, security, governance, and enterprise best practices. **Preferred candidate profile** - 610+ years of overall software development experience with **2–4+ years in Generative AI/LLM engineering**. - Strong hands-on expertise in **Python** and backend API development (FastAPI preferred). - Experience building production-grade **LLM**, **RAG**, and **Agentic AI** applications. - Hands-on experience with **LangChain/LangGraph** or similar orchestration frameworks. - Experience working with **Vector Databases** such as Azure AI Search, Pinecone, ChromaDB, FAISS, or OpenSearch. - Strong understanding of **Prompt Engineering**, embeddings, semantic search, hybrid retrieval, and reranking techniques. - Experience implementing **AI Evaluation/Validation** frameworks (RAGAS, DeepEval, LLM-as-a-Judge, hallucination detection, response quality evaluation). - Familiarity with Azure OpenAI, AWS Bedrock, OpenAI APIs, or similar enterprise AI platforms. - Strong knowledge of SQL and enterprise data models. - Experience with Docker, Git, CI/CD, and cloud deployment is preferred. - Excellent analytical, debugging, stakeholder communication, and problem-solving skills. - Experience working in enterprise, banking, financial services, or large-scale production environments is an added advantage.