Copilot

Job description

#### **Key Responsibilities** - **Opportunity Identification:** Audit IT support workflows to identify high-ROI automation candidates. - **Agent Development:** - Design and build intelligent, conversational agents using **Copilot Studio Agent Builder** . - Develop complex, orchestrated agentic workflows using the **Microsoft Agent Framework (MAF)** . - **Validation & Quality Assurance:** - Implement rigorous validation protocols (e.g., adversarial testing, prompt rubrics, and RAG retrieval evaluation) to ensure AI outputs align with business rules and compliance requirements. - Maintain "human-in-the-loop" oversight mechanisms and monitor for drift (data, prompt, and output) to ensure ongoing accuracy. - **Cost Management & FinOps:** - Analyze and optimize **Total Cost of Ownership (TCO)** , including API consumption, token usage, infrastructure overhead, and maintenance costs. - Build "value-versus-spend" models to justify AI investments and identify opportunities for rightsizing/deprovisioning inefficient agents. - **Framework & Risk Alignment:** Ensure all developments satisfy **Shell framework** standards and **IRA guidelines** regarding data privacy, security, and ethical AI. - **Productionization:** Monitor performance via telemetry and establish continuous feedback loops for iterative improvements. #### **Technical Requirements** - **Core Platforms:** Proven experience with **Copilot Studio** and **Microsoft Agent Framework (MAF)** . - **Validation Tooling:** Experience with AI evaluation frameworks (e.g., Promptfoo, Ragas) or automated testing suites for LLM outputs. - **Microsoft Stack:** Strong proficiency in Power Platform, Azure AI Services, and associated M365 governance. - **Financial/Analytical Skills:** Ability to perform cost-benefit analysis, track consumption metrics, and translate technical usage into business ROI. - **Governance & Security:** Familiarity with implementing enterprise-grade security within Microsoft environments. #### **Soft Skills** - **Analytical Mindset:** Ability to translate complex business problems into modular technical requirements. - **Governance-First Approach:** Eagerness to balance rapid innovation with the rigor required by internal risk and assurance policies. - **Financial Fluency:** Ability to articulate the cost drivers of AI to stakeholders and manage project budgets effectively.