Senior AI Data Engineer

Job description

Key Responsibilities • Design and develop LLM-powered applications using agentic patterns (single/multi-agent) for business use cases • Build and optimise end-to-end RAG pipelines (ingestion, embeddings, retrieval, orchestration, response synthesis) • Implement prompt engineering and orchestration techniques (prompt chaining, tool/function calling, structured outputs) • Develop production-grade APIs and services (FastAPI/Flask/Streamlit) for GenAI applications • Integrate LLM solutions with enterprise systems, data platforms, and workflows • Apply guardrails and evaluation frameworks to improve response quality, reduce hallucinations, and ensure responsible AI usage • Collaborate with Data Engineering and MLOps teams for data pipelines, deployment, monitoring, and scaling • Contribute to reusable components, documentation, and engineering best practices Experience & Core Requirements (Must-Have) Overall Experience • 6–9 years total experience • 1–3+ years in hands-on GenAI / LLM application development (production use cases) LLM / GenAI & Agentic Engineering • Strong hands-on experience with: - LLMs (Claude, OpenAI, etc.) • RAG pipelines and retrieval optimisation • GPT + Agentic AI implementation experience • Experience with: - LangChain, LangGraph, or similar frameworks • Agent orchestration and tool-calling architectures • Deep understanding of: - LLM limitations, evaluation, and optimisation strategies Core Engineering • Strong Python/Pyspark engineering expertise (production-grade development) with proven API integration experience • Deep data analysis experience and handling large volume of data • Fabric/Azure Databricks/Snowflake data engineering integration skills • Good exposure to: - Cloud platforms (Azure/AWS/GCP) • SQL • Containers, CI/CD, monitoring Data / AI Foundations (Mandatory) Prior Experience In One Or More • Data Engineering (ETL/ELT, pipelines, orchestration) • Data Science / ML lifecycle (especially NLP) • Analytics engineering / data products Good-to-Have / Preferred • Experience with fine-tuning techniques (LoRA, PEFT) or prompt tuning strategies • Experience with enterprise GenAI security & privacy practices (data masking, access control, compliance) • Familiarity with Azure AI ecosystem (Azure OpenAI, Azure AI Search, Fabric, etc.) Exposure to agentic coding tools (e.g., Claude Code or similar environments)