Architect AI Data Engineer

Job description

**Job Description: Key Responsibilities** **1. Solution Architecture & Strategy** - Define and lead **end-to-end architecture** for enterprise GenAI platforms and use cases - Design **scalable agentic systems** (single-agent, multi-agent, orchestration frameworks) - Establish **reference architectures, design patterns, and reusable frameworks** - Lead architecture decisions on **RAG vs fine-tuning vs hybrid approaches** - Conduct **technology evaluations (LLMs, vector DBs, orchestration frameworks)** and recommend best-fit solutions **2. Agentic AI & LLM Engineering Leadership** - Design and implement **complex agentic workflows** with tool calling, function orchestration, and memory strategies - Build **enterprise-grade RAG pipelines** with strong focus on **retrieval accuracy and evaluation** - Drive **prompt architecture standards** (prompt libraries, chaining, orchestration governance) - Optimise solutions for **latency, cost, scalability, and reliability** **3. Platform & Engineering Excellence** - Lead development of **GenAI platforms, APIs, and microservices** (FastAPI, Flask, etc.) - Define **engineering best practices** : coding standards, testing, packaging, observability - Ensure seamless integration with **enterprise data platforms, APIs, and business applications** - Collaborate with MLOps teams for **CI/CD, deployment pipelines, versioning, and monitoring** **4. Governance, Risk & Responsible AI** - Define and enforce **LLM guardrails** (hallucination control, safety filters, policy enforcement) - Implement **evaluation frameworks** (RAG evaluation, prompt testing, benchmarking) - Ensure compliance with **data security, privacy, and enterprise governance standards** - Drive adoption of **Responsible AI practices** (bias mitigation, explainability, auditability) **5. Data & Ecosystem Collaboration** - Partner with Data Engineering teams on: - Data ingestion, pipelines, and quality controls - Metadata management and knowledge graph strategies - Work with business stakeholders to: - Identify high-value GenAI use cases Translate business problems into AI-driven solutions **6. Leadership & Stakeholder Management** - Provide **technical leadership and mentorship** to engineering teams - Act as a **solution advisor to clients/stakeholders** (including pre-sales, PoCs, solutioning) - Present architecture and design decisions to **senior leadership and CXOs** - Drive **COE initiatives, knowledge sharing, and internal capability building** **Must-Have Skills & Experience** **Experience** - **12–15 years total experience** , with **3+ years in GenAI / LLM-based systems** - Proven experience in **leading architecture and delivery of enterprise solutions** **LLM / GenAI & Agentic Engineering** - Strong hands-on experience with: - LLMs (Claude, OpenAI, etc.) - RAG pipelines and retrieval optimisation - GPT + Agentic AI implementation experience - Experience with: - LangChain, LangGraph, or similar frameworks - Agent orchestration and tool-calling architectures Deep understanding of: LLM limitations, evaluation, and optimisation strategies **Core Engineering** - Strong Python/Pyspark engineering expertise (production-grade development) with proven API integration experience - Deep data analysis experience and handling large volume of data - Fabric/Azure Databricks/Snowflake data engineering integration skills - Good exposure to: - Cloud platforms (Azure/AWS/GCP) - SQL Containers, CI/CD, monitoring **Cloud & Platform** - Hands-on experience with **Azure / AWS / GCP** - Familiarity with: - Containers (Docker/Kubernetes) - CI/CD pipelines Monitoring & observability **Data / AI Foundations (Mandatory)** Prior experience in one or more: - Data Engineering (ETL/ELT, pipelines, orchestration) - Data Science / ML lifecycle (especially NLP) Analytics engineering / data products **Good-to-Have / Preferred** - Fine-tuning techniques ( **LoRA, PEFT, prompt tuning** ) - Experience with **Azure AI stack (Azure OpenAI, Cognitive Search)** - Knowledge of **knowledge graphs, semantic layers, or enterprise search** - Experience in **domain-specific GenAI solutions** (Insurance, BFSI, Healthcare) **Responsibilities: Key Responsibilities** **1. Solution Architecture & Strategy** - Define and lead **end-to-end architecture** for enterprise GenAI platforms and use cases - Design **scalable agentic systems** (single-agent, multi-agent, orchestration frameworks) - Establish **reference architectures, design patterns, and reusable frameworks** - Lead architecture decisions on **RAG vs fine-tuning vs hybrid approaches** - Conduct **technology evaluations (LLMs, vector DBs, orchestration frameworks)** and recommend best-fit solutions **2. Agentic AI & LLM Engineering Leadership** - Design and implement **complex agentic workflows** with tool calling, function orchestration, and memory strategies - Build **enterprise-grade RAG pipelines** with strong focus on **retrieval accuracy and evaluation** - Drive **prompt architecture standards** (prompt libraries, chaining, orchestration governance) - Optimise solutions for **latency, cost, scalability, and reliability** **3. Platform & Engineering Excellence** - Lead development of **GenAI platforms, APIs, and microservices** (FastAPI, Flask, etc.) - Define **engineering best practices** : coding standards, testing, packaging, observability - Ensure seamless integration with **enterprise data platforms, APIs, and business applications** - Collaborate with MLOps teams for **CI/CD, deployment pipelines, versioning, and monitoring** **4. Governance, Risk & Responsible AI** - Define and enforce **LLM guardrails** (hallucination control, safety filters, policy enforcement) - Implement **evaluation frameworks** (RAG evaluation, prompt testing, benchmarking) - Ensure compliance with **data security, privacy, and enterprise governance standards** - Drive adoption of **Responsible AI practices** (bias mitigati