AI Observability Engineer

Job description

**Role & responsibilities** **AI Model Monitoring & Observability** - Design and implement observability frameworks for AI/ML and GenAI applications. - Monitor model performance, prediction quality, latency, throughput, and availability. - Track model drift, data drift, concept drift, and performance degradation. - Establish AI-centric KPIs, SLAs, and SLOs. **Generative AI Observability** - Monitor LLM applications for: - Hallucinations - Response quality - Prompt effectiveness - Toxicity detection - Cost and token consumption - Retrieval quality in RAG systems **MLOps & Governance** - Collaborate with data scientists and ML engineers to deploy monitoring solutions. - Ensure compliance with Responsible AI and governance policies. **Incident Management** - Investigate AI service outages and model failures. - Perform root cause analysis (RCA). **Preferred candidate profile** - Bachelor's or Master's degree in Computer Science, Data Science, Engineering, or related field. - 5+ years in Data Engineering, AI/ML, MLOps, or Platform Engineering. - 2+ years working with GenAI/LLM solutions. - Experience supporting production AI applications.