Senior AI Data Engineer

Job description

**Job Description: Key Responsibilities** - Design and develop **LLM-powered applications** using **agentic patterns (single/multi-agent)** for business use cases - Build and optimise **end-to-end RAG pipelines** (ingestion, embeddings, retrieval, orchestration, response synthesis) - Implement **prompt engineering and orchestration techniques** (prompt chaining, tool/function calling, structured outputs) - Develop **production-grade APIs and services** (FastAPI/Flask/Streamlit) for GenAI applications - Integrate LLM solutions with **enterprise systems, data platforms, and workflows** - Apply **guardrails and evaluation frameworks** to improve response quality, reduce hallucinations, and ensure responsible AI usage - Collaborate with **Data Engineering and MLOps teams** for data pipelines, deployment, monitoring, and scaling - Contribute to **reusable components, documentation, and engineering best practices** **Experience & Core Requirements (Must-Have)** **Overall Experience** - **6–9 years total experience** - **1–3+ years in hands-on GenAI / LLM application development (production use cases)** **LLM / GenAI & Agentic Engineering** - Strong hands-on experience with: - LLMs (Claude, OpenAI, etc.) - RAG pipelines and retrieval optimisation - GPT + Agentic AI implementation experience - Experience with: - LangChain, LangGraph, or similar frameworks - Agent orchestration and tool-calling architectures Deep understanding of: LLM limitations, evaluation, and optimisation strategies **Core Engineering** - Strong Python/Pyspark engineering expertise (production-grade development) with proven API integration experience - Deep data analysis experience and handling large volume of data - Fabric/Azure Databricks/Snowflake data engineering integration skills - Good exposure to: - Cloud platforms (Azure/AWS/GCP) - SQL Containers, CI/CD, monitoring **Data / AI Foundations (Mandatory)** Prior experience in one or more: - Data Engineering (ETL/ELT, pipelines, orchestration) - Data Science / ML lifecycle (especially NLP) Analytics engineering / data products **Good-to-Have / Preferred** - Experience with **fine-tuning techniques (LoRA, PEFT) or prompt tuning strategies** - Experience with **enterprise GenAI security & privacy practices** (data masking, access control, compliance) - Familiarity with **Azure AI ecosystem** (Azure OpenAI, Azure AI Search, Fabric, etc.) - Exposure to **agentic coding tools (e.g., Claude Code or similar environments)** **Responsibilities: Key Responsibilities** - Design and develop **LLM-powered applications** using **agentic patterns (single/multi-agent)** for business use cases - Build and optimise **end-to-end RAG pipelines** (ingestion, embeddings, retrieval, orchestration, response synthesis) - Implement **prompt engineering and orchestration techniques** (prompt chaining, tool/function calling, structured outputs) - Develop **production-grade APIs and services** (FastAPI/Flask/Streamlit) for GenAI applications - Integrate LLM solutions with **enterprise systems, data platforms, and workflows** - Apply **guardrails and evaluation frameworks** to improve response quality, reduce hallucinations, and ensure responsible AI usage - Collaborate with **Data Engineering and MLOps teams** for data pipelines, deployment, monitoring, and scaling - Contribute to **reusable components, documentation, and engineering best practices** **Experience & Core Requirements (Must-Have)** **Overall Experience** - **6–9 years total experience** - **1–3+ years in hands-on GenAI / LLM application development (production use cases)** **LLM / GenAI & Agentic Engineering** - Strong hands-on experience with: - LLMs (Claude, OpenAI, etc.) - RAG pipelines and retrieval optimisation - GPT + Agentic AI implementation experience - Experience with: - LangChain, LangGraph, or similar frameworks - Agent orchestration and tool-calling architectures Deep understanding of: LLM limitations, evaluation, and optimisation strategies **Core Engineering** - Strong Python/Pyspark engineering expertise (production-grade development) with proven API integration experience - Deep data analysis experience and handling large volume of data - Fabric/Azure Databricks/Snowflake data engineering integration skills - Good exposure to: - Cloud platforms (Azure/AWS/GCP) - SQL Containers, CI/CD, monitoring **Data / AI Foundations (Mandatory)** Prior experience in one or more: - Data Engineering (ETL/ELT, pipelines, orchestration) - Data Science / ML lifecycle (especially NLP) Analytics engineering / data products **Good-to-Have / Preferred** - Experience with **fine-tuning techniques (LoRA, PEFT) or prompt tuning strategies** - Experience with **enterprise GenAI security & privacy practices** (data masking, access control, compliance) - Familiarity with **Azure AI ecosystem** (Azure OpenAI, Azure AI Search, Fabric, etc.) - Exposure to **agentic coding tools (e.g., Claude Code or similar environments)** **Qualifications: Key Responsibilities** - Design and develop **LLM-powered applications** using **agentic patterns (single/multi-agent)** for business use cases - Build and optimise **end-to-end RAG pipelines** (ingestion, embeddings, retrieval, orchestration, response synthesis) - Implement **prompt engineering and orchestration techniques** (prompt chaining, tool/function calling, structured outputs) - Develop **production-grade APIs and services** (FastAPI/Flask/Streamlit) for GenAI applications - Integrate LLM solutions with **enterprise systems, data platforms, and workflows** - Apply **guardrails and evaluation frameworks** to improve response quality, reduce hallucinations, and ensure responsible AI usage - Collaborate with **Data Engineering and MLOps teams** for data pipelines, deployment, monitoring, and scaling - Contribute to **reusable components, documentation, and engineering best practices** **Experience & Core Requirements (Must-Have)**