AI Trust & Governance Architect

Job description

**Role & responsibilities:** AI Assurance Architecture - Architect platforms and frameworks for AI assurance, evaluation, and benchmarking - Design systems for LLM, agent, and RAG evaluation across functional, nonfunctional, and risk dimensions - Define architectural patterns for Responsible AI, bias detection, explainability, and safety validation - Build reusable assurance components supporting Business Assurance, Risk Assurance, and Reliability **Security, Reliability & Governance** - Architect AI testing and validation for security, privacy, prompt injection, and adversarial robustness - Integrate redteaming, threat simulation, and chaosstyle validation for AI systems - Define governance mechanisms for model usage, auditability, traceability, and compliance - Ensure AI systems meet enterprise standards for resilience, fault tolerance, and observability **Platform & Engineering Enablement** - Design AI assurance platforms supporting automated test execution, reporting, and insights - Enable integration with CI/CD pipelines to enforce AI quality gates - Collaborate with QE engineering teams to embed AI assurance into the SDLC - Mentor teams on AI risk identification and mitigation from an engineering perspective **Core Platforms, Frameworks & Tooling** - LLM and AI evaluation frameworks (PromptFoo, DeepEval, custom LLM evaluation harnesses) - Prompt, RAG, and agent validation tooling (prompt testing frameworks, retrieval accuracy validators, agent workflow evaluators) - Responsible AI and model risk tooling (Fairlearn, SHAP, Explainable AI libraries, toxicity and bias scanners) - Security and adversarial testing tools for AI systems (PyRIT, Garak) - AI redteaming and threat simulation frameworks (automated redteam scripts, adversarial test suites for LLMs and agents) - AI assurance automation and QE frameworks (Galileo) - Observability for AI behavior and drift (Langfuse, Arize, Evidently, custom telemetry dashboards) Client Orientation & Leadership - Partner with product and engineering teams to identify AI Assurance opportunities and shape roadmaps - Support client workshops, RFPs, and solution presentations - Mentor engineers on AI/ML/Gen AI best practices and emerging technologies - Translate complex AI concepts into business-friendly narratives **Preferred candidate profile:** **MustHave Qualifications** - 13+ years of experience in software engineering with 3+ years in AI with strong architecture ownership - Handson expertise in AI/ML systems, LLM evaluation, and assurance frameworks - Experience with AI red teaming, model risk management, or AI audit tooling - Strong understanding of Responsible AI, AI risks, and governance principles - Experience with security testing, adversarial testing, and reliability engineering - Proficiency in Python, automation frameworks, and cloud platforms **GoodtoHave Skills** - Knowledge of regulatory or compliance considerations for AI systems - Exposure to performance engineering, chaos engineering, or resilience testing for AI - Contributions to internal platforms, frameworks, or standards