Principal Architect AI Data Engineer

Job description

**Job Description: Key Responsibilities** **Architecture & Solution Leadership** - Lead the design of **enterprise-grade GenAI and agentic architectures** (single-agent, multi-agent, tool-driven systems). - Define **reference architectures, reusable frameworks, and best practices** for LLM applications across the organisation. - Architect and oversee implementation of **end-to-end RAG pipelines** : Data ingestion chunking embeddings vector search orchestration - response synthesis. - Drive **scalability, reliability, cost optimisation, and performance** across GenAI platforms. **Agentic & LLM Engineering (Hands-on + Oversight)** - Provide technical leadership in **prompt engineering, prompt orchestration, and agent workflows** (LangChain, LangGraph, etc.). - Guide teams on **tool-calling, function-calling, memory handling, and multi-agent system design** . - Lead efforts in **hallucination reduction, guardrails, safety mechanisms, and output evaluation frameworks** . **Platform & Engineering Excellence** - Architect **production-grade APIs and services** (FastAPI/Flask/enterprise microservices) for LLM solutions. - Define **MLOps / LLMOps pipelines** including CI/CD, monitoring, observability, and evaluation. - Partner with Data Engineering teams to ensure: - Data quality, lineage, governance, and compliance Seamless integration with enterprise data platforms **Organisation-Level Responsibilities (Critical)** **Capability Building & CoE Development** - Build and scale **GenAI / Agentic AI Centre of Excellence (CoE)** . - Define **standardised frameworks, accelerators, and reusable components** to improve delivery velocity. - Drive organisation-wide adoption of **GenAI best practices and tooling standards** . **Strategic & Stakeholder Leadership** - Engage with **CXOs, business stakeholders, and clients** to translate business problems into AI-led solutions. - Lead **solutioning, pre-sales, RFP responses, and client workshops** for GenAI opportunities. - Influence **AI strategy, roadmap, and investment decisions** at organisational level. **Governance, Risk & Compliance** - Establish **enterprise governance frameworks** for GenAI: - Responsible AI, security, privacy, ethical usage, and compliance - Define policies for: - Data access, redaction, model usage, auditability, and explainability **Mentorship & Team Leadership** - Mentor and guide **architects, engineers, and data scientists** . - Drive **technical upskilling, hiring strategy, and capability maturity** . - Review solution designs and enforce **architecture quality standards** . **Experience & Must-Have Skills** **Experience** - **15+ years of total experience** in Data Engineering / Data Science / AI - **3+ years of hands-on experience in LLM / GenAI solutions at scale** - Proven experience in **architecture, solution design, and enterprise delivery** **LLM / GenAI & Agentic Engineering** - Strong hands-on experience with: - LLMs (Claude, OpenAI, etc.) - RAG pipelines and retrieval optimisation - GPT + Agentic AI implementation experience - Experience with: - LangChain, LangGraph, or similar frameworks - Agent orchestration and tool-calling architectures Deep understanding of: LLM limitations, evaluation, and optimisation strategies **Core Engineering** - Strong Python/Pyspark engineering expertise (production-grade development) with proven API integration experience - Deep data analysis experience and handling large volume of data - Fabric/Azure Databricks/Snowflake data engineering integration skills - Good exposure to: - Cloud platforms (Azure/AWS/GCP) - SQL Containers, CI/CD, monitoring **Data / AI Foundations (Mandatory)** Prior experience in one or more: - Data Engineering (ETL/ELT, pipelines, orchestration) - Data Science / ML lifecycle (especially NLP) Analytics engineering / data products **Good-to-Have / Preferred** - Fine-tuning techniques ( **LoRA, PEFT, prompt tuning, few-shot learning** ) - Experience with **enterprise GenAI deployments** (security, privacy, governance) - Experience with **Azure ecosystem** (Azure OpenAI, AI Search, Fabric, etc.) - Exposure to **industry use cases** (Insurance, BFSI, Healthcare, Retail, etc.) **Responsibilities: Key Responsibilities** **Architecture & Solution Leadership** - Lead the design of **enterprise-grade GenAI and agentic architectures** (single-agent, multi-agent, tool-driven systems). - Define **reference architectures, reusable frameworks, and best practices** for LLM applications across the organisation. - Architect and oversee implementation of **end-to-end RAG pipelines** : Data ingestion chunking embeddings vector search orchestration - response synthesis. - Drive **scalability, reliability, cost optimisation, and performance** across GenAI platforms. **Agentic & LLM Engineering (Hands-on + Oversight)** - Provide technical leadership in **prompt engineering, prompt orchestration, and agent workflows** (LangChain, LangGraph, etc.). - Guide teams on **tool-calling, function-calling, memory handling, and multi-agent system design** . - Lead efforts in **hallucination reduction, guardrails, safety mechanisms, and output evaluation frameworks** . **Platform & Engineering Excellence** - Architect **production-grade APIs and services** (FastAPI/Flask/enterprise microservices) for LLM solutions. - Define **MLOps / LLMOps pipelines** including CI/CD, monitoring, observability, and evaluation. - Partner with Data Engineering teams to ensure: - Data quality, lineage, governance, and compliance Seamless integration with enterprise data platforms **Organisation-Level Responsibilities (Critical)** **Capability Building & CoE Development** - Build and scale **GenAI / Agentic AI Centre of Excellence (CoE)** . - Define **standardised frameworks, accelerators, and reusable components** to improve delivery velocity. - Drive organisation-wide adoption of **GenAI best practices and tooling standards** . **St