Artificial Intelligence Engineer

Job description

Role & responsibilities • Design, develop, and enhance AI/LLM applications using Python, LangChain, RAG, and Agentic AI frameworks. • Build and optimize RAG pipelines covering document ingestion, chunking, embeddings, vector search, retrieval, reranking, and LLM response generation. • Develop and integrate LLM/agent observability to monitor application health, performance, reliability, and AI quality. • Implement tracing and monitoring across LLM calls, agent workflows, tools, retrieval pipelines, and APIs. • Track key AI metrics such as latency, token usage, response quality, hallucination/error rates, retrieval accuracy, failures, and cost. • Build dashboards, alerts, and monitoring mechanisms to identify LLM failures, degraded retrieval, latency issues, and production anomalies. • Work with tools such as Azure Application Insights, Azure Monitor, OpenTelemetry, LangSmith, Azure AI Foundry/evaluation capabilities, or equivalent observability platforms. • Implement LLM evaluation and RAG evaluation using appropriate metrics and feedback mechanisms. • Monitor and troubleshoot multi-agent workflows, tool calls, prompt execution, and model responses. • Integrate observability into FastAPI/microservices and cloud-native AI applications. • Deploy and support AI workloads on Azure, with exposure to Docker/Kubernetes being an advantage. • Establish logging, error handling, audit trails, and production monitoring for enterprise AI systems. • Collaborate with AI engineers, data engineers, architects, and business stakeholders to improve AI reliability, performance, and user experience. • Continuously identify opportunities to improve model performance, retrieval quality, scalability, latency, and operational cost. Preferred candidate profile Must-Have Skills • Python strong hands-on development • RAG – strong practical experience • LangChain – strong hands-on experience • LLMs / Generative AI • Agentic AI / LangGraph • Azure OpenAI • Azure AI Search / Vector Search • Embeddings, semantic search and document retrieval • FastAPI / REST APIs • Prompt engineering and LLM evaluation • Production deployment of AI applications