Principal Architect AI Data Engineer

Job description

Key Responsibilities Architecture & Solution Leadership • Lead the design of enterprise-grade GenAI and agentic architectures (single-agent, multi-agent, tool-driven systems). • Define reference architectures, reusable frameworks, and best practices for LLM applications across the organisation. • Architect and oversee implementation of end-to-end RAG pipelines: - Data ingestion → chunking → embeddings → vector search → orchestration → response synthesis. • Drive scalability, reliability, cost optimisation, and performance across GenAI platforms. Agentic & LLM Engineering (Hands-on + Oversight) • Provide technical leadership in prompt engineering, prompt orchestration, and agent workflows (LangChain, LangGraph, etc.). • Guide teams on tool-calling, function-calling, memory handling, and multi-agent system design. • Lead efforts in hallucination reduction, guardrails, safety mechanisms, and output evaluation frameworks. Platform & Engineering Excellence • Architect production-grade APIs and services (FastAPI/Flask/enterprise microservices) for LLM solutions. • Define MLOps / LLMOps pipelines including CI/CD, monitoring, observability, and evaluation. • Partner with Data Engineering teams to ensure: - Data quality, lineage, governance, and compliance • Seamless integration with enterprise data platforms Organisation-Level Responsibilities (Critical) Capability Building & CoE Development • Build and scale GenAI / Agentic AI Centre of Excellence (CoE). • Define standardised frameworks, accelerators, and reusable components to improve delivery velocity. • Drive organisation-wide adoption of GenAI best practices and tooling standards. Strategic & Stakeholder Leadership • Engage with CXOs, business stakeholders, and clients to translate business problems into AI-led solutions. • Lead solutioning, pre-sales, RFP responses, and client workshops for GenAI opportunities. • Influence AI strategy, roadmap, and investment decisions at organisational level. Governance, Risk & Compliance • Establish enterprise governance frameworks for GenAI: - Responsible AI, security, privacy, ethical usage, and compliance • Define policies for: - Data access, redaction, model usage, auditability, and explainability Mentorship & Team Leadership • Mentor and guide architects, engineers, and data scientists. • Drive technical upskilling, hiring strategy, and capability maturity. • Review solution designs and enforce architecture quality standards. Experience Experience & Must-Have Skills • 15+ years of total experience in Data Engineering / Data Science / AI • 3+ years of hands-on experience in LLM / GenAI solutions at scale • Proven experience in architecture, solution design, and enterprise delivery LLM / GenAI & Agentic Engineering • Strong hands-on experience with: - LLMs (Claude, OpenAI, etc.) • RAG pipelines and retrieval optimisation • GPT + Agentic AI implementation experience • Experience with: - LangChain, LangGraph, or similar frameworks • Agent orchestration and tool-calling architectures • Deep understanding of: - LLM limitations, evaluation, and optimisation strategies Core Engineering • Strong Python/Pyspark engineering expertise (production-grade development) with proven API integration experience • Deep data analysis experience and handling large volume of data • Fabric/Azure Databricks/Snowflake data engineering integration skills • Good exposure to: - Cloud platforms (Azure/AWS/GCP) • SQL • Containers, CI/CD, monitoring Data / AI Foundations (Mandatory) Prior Experience In One Or More • Data Engineering (ETL/ELT, pipelines, orchestration) • Data Science / ML lifecycle (especially NLP) Analytics engineering / data products Good-to-Have / Preferred • Fine-tuning techniques (LoRA, PEFT, prompt tuning, few-shot learning) • Experience with enterprise GenAI deployments (security, privacy, governance) • Experience with Azure ecosystem (Azure OpenAI, AI Search, Fabric, etc.) • Exposure to industry use cases (Insurance, BFSI, Healthcare, Retail, etc.)