Sunset Learning Turns 30! Discover our story, exclusives, and celebrations →
→
- COURSE
Auditing AI: Hands-On for Internal Auditors
Price:
Duration:
Certification:
Exam:
Continuing Education Credits:
Learning Credits:
Understand and evaluate the foundational concepts, mechanisms, risks, and governance implications of Artificial Intelligence (AI) and Generative AI systems, with a specific focus on how these technologies impact audit scope, risk assessment, and control requirements. Internal and external auditors will assess whether AI initiatives are governed effectively from planning through deployment using recognized frameworks and documented controls. Design and apply AI governance policies and procedures for GenAI applications by aligning organizational controls with trust principles, regulatory requirements, and AI lifecycle oversight. Will be able to apply the full AI audit lifecycle to a real-world case, assess governance maturity, and produce closure documentation appropriate for either internal or external auditor roles in accordance with ISO/IEC 42001 and the NIST AI Risk Management Framework.
Upcoming Class Dates and Times
All Sunset Learning courses are guaranteed to run
Course Outline and Details
Prerequisites
The following is recommended before attending:
- A fundamental understanding of AI
- Familiarity with the IIA AI Framework
- Review the NIST AI RMF
- The IIA beginner course: Essentials for AI Auditing
Target Audience
Who should Attend:
- IT Auditors and Compliance Professionals: This is the primary audience. If you're already doing IT audits (SOX, SOC 2, HIPAA, etc.) and your organization is deploying AI, you need to evolve your skillset or risk becoming obsolete. AI-based systems are inherently more complex and fundamentally different from traditional IT systems, particularly with deep learning models that lack transparent logic, introducing new risks that are not widely understood by IT auditors.
- Infrastructure and Platform Engineers: A person that understands how AI systems are deployed, hosted, and operated, but adding audit methodology gives you the ability to assess whether those systems are governed properly. Most auditors struggle with the technical side; a technical infrastructure person coming at it from the opposite direction is incredibly valuable.
- Risk and Governance Leaders: RCISOs, GRC managers, and anyone responsible for enterprise risk management. As AI gets embedded into business processes, they need to understand what controls should exist and how to verify them.
Course Objectives
- Define key AI concepts including Artificial Intelligence, Machine Learning, Generative AI, LLMs, and Small Language Models (SLMs).
- Explain how LLMs generate responses and identify the risks of hallucination, inconsistency, and lack of explainability.
- Differentiate between traditional automation, ML models, GenAI, and SLMs in terms of audit risk and control requirements.
- Evaluate core risks that apply across all AI systems, including data bias, model drift, and overfitting.
- Describe the auditor’s responsibilities in reviewing early-stage GenAI adoption, with attention to role separation between internal and external audit functions.
- Compare major AI auditing and governance frameworks used by internal and external auditors (NIST AI RMF, ISO/IEC 42001, EU AI Act, GAIA, etc.).
- Apply risk and governance concepts in a hands-on LLM prompt lab and generate auditor-aligned reflections.
- Identify key points in the AI lifecycle where internal and external auditors should engage
- Differentiate responsibilities between audit, IT, and data science teams regarding AI risk
- Recognize the audit implications of model degradation, drift, and bias ownership
- Apply prompt engineering as a method to surface audit-relevant model behaviors
- Interpret how prompt-based audit observations support assurance, documentation, and escalation
Course Outline
Understanding and Auditing AI Applications
Learning Path 1: Understanding AI Systems and Establishing Audit Scope
- Module 1: Exploring AI and Generative AI Services
- Intro and Objectives
- AI Fundamentals
- GenAI and Language Models
- Risk Awareness
- Frameworks and Governance
- Internal versus External Role Comparison
- Module 1 – Lab: Exploring LLM Behavior and Drafting AI Compliance Assessments
- Compliance Assessments
- Summary and Takeaways
- Module 2: How to Audit the Intricate Components of AI Applications
- What to Audit in AI Systems
- How Audit Checklist Items Map to Frameworks
- Metrics for Evaluating GenAI Outputs
- Mapping Metrics to NIST AI RMF Functions
- GenAI Output Review: Internal vs External Auditor Roles
- Introduction to Auditing Tools
- Auditing Tools by Role
- Module 2 – Lab Part A: Evaluating and Tracking Compliance Measures with Microsoft Purview Compliance Manager
- Module 2 – Lab Part B: Fairness and Bias Review Using Aequitas
- Classifying Data and AI Models
- Real-World Risk Case: $1 Car Chatbot
- Module Summary and Takeaways
- Module 3: Investigating Internal AI Usage – Governance
- Introduction to Internal AI Governance
- AI Activity Logging and Monitoring Practices
- Using Microsoft Purview to Audit AI Usage
- Policy Adherence and Risk Signal Evaluation
- Internal vs. External Auditor Responsibilities
- Module 3 – Lab1: Governance Audit of AI-driven Traffic Sign Recognition
- Module 3 – Lab 2: Reviewing AI Prompt Trails with Microsoft Purview
- Summary and Key Takeaways
- Knowledge Check
Learning Path 2: Structuring Risk-Base AI Engagements
- Module 4: Redefining Audit Engagement Across the AI Lifecycle
- Framing the Auditor’s Role in AI Governance
- Auditor Engagement Across the AI Lifecycle
- Who Owns AI Risk? Role Differentiation Matrix
- Understanding Model Degradation, Drift, and Accountability
- Prompt Engineering as an Audit Tool
- Prompt Audit Patterns: Red Flag Prompts for Risk Discovery
- Module 4 – Lab: Conducting a Prompt-Based AI Audit
- Reflection and Role Exercise: Who Should Respond to This Risk?
Learning Path 3: Executing Fieldwork Across the AI Lifecycle
- Module 5: Execute AI Project Management Efficiently
- AI Project Governance: Scope and Oversight
- Auditing the AI Vision, Strategy, and Roadmap
- Evaluating Project Roles and Cross-Functional Accountability
- Auditing Risk Registers, Use Case Alignment, and Business Impact
- Module 5 – Lab: Reviewing AI Project Governance Templates
- Summary and Key Takeaways
- Knowledge Check
- Module 6: Monitoring AI Systems and Governance in Action
- Monitoring What Matters
- What Should Be Audited Post-Deployment
- Types of Audit Evidence: Logs, Outputs, Labeling, Risk
- Frameworks in Action
- Manual Audit Techniques: No-Tools? No Problem
- Module 6 Lab: Documenting AI Risk Using the Risk Register tool
- Summary and Maturity Takeaways
- Knowledge Check
- AI Governance, Monitoring, and Capstone Execution
Learning Path 4: Assessing Maturity, Governance, and Strategic Closure
- Module 7: Designing Governance in New AI and GenAI Applications
- AI Governance Principles and Standards Overview
- Translating Governance Principles into Policy
- Domain-Based Governance Structures
- Auditing Governance Implementation
- Governance Gaps and Red Flags
- Knowledge Check and Reflection Questions
- Module 7 Lab: Assessing Your Organizations AI Governance Maturity
- Summary and Key Takeaways
- Module 8: Auditing AI Improvement Cycles and Profile-Driven Risk Tailoring
- Understanding AI Improvement Obligations.
- NIST AI RMF Profiles and Audit Customization.
- Auditing the AI Feedback Loop: Are Controls Evolving?
- Internal versus External Auditor Roles in the Improvement Lifecycle.
- Evaluating Evidence of Corrective and Preventive Actions.
- Knowledge Checks.
- Module 8 – Lab: Auditing Evidence of AI Governance Improvement
- Summary and Key Takeaways
- Module 9: Administering Trust and Accountability in Emerging AI Platforms
- Introduction to Trust and AI Platform Governance
- Auditing Platform-Level Trustworthiness Characteristics
- Provisioning and Onboarding AI Services Securely
- Controls for Post-Deployment Behavior and Drift
- Auditing Multimodal AI and GenAI Capabilities
- Module 9 – Lab: Privacy Trust Assessment
- Summary and Key Takeaways
- Module 10: Finalizing the AI Audit – Synthesis, Reporting, and Strategic Readiness
- Reviewing Multi-Domain AI Audit Findings
- Mapping Risks to ISO Clauses and NIST Functions
- Evaluating Governance Maturity and Improvement Signals
- Final Audit Judgment: Certification, Readiness, or Escalation
- Internal versus External Auditor Roles in Final Reporting
- Knowledge Check: Risk Readiness versus Risk Documentation
- Capstone Simulation
- Summary and Key Takeaways
Capstone Final Event: Business Audit Simulation
- Capstone: Auditing a National AI Program – The Australian Taxation Office Case
Capstone Deliverables:
- Learners will submit one of the following, based on their assigned role:
- Internal Auditor Role:
- A completed AI Audit Closure Memo, including:
- Summary of findings
- Residual risk analysis
- Clause 10.2 alignment
- Closure determination or monitoring plan
- A completed AI Audit Closure Memo, including:
- External Auditor Role:
- A completed Readiness Opinion Letter, including:
- Scope of review
- Key observations
- Maturity and risk assessment
- Certification readiness opinion
- Recommendations for improvement
- A completed Readiness Opinion Letter, including:
- Internal Auditor Role:
Lab Outline
Understanding and Auditing AI Applications
- Learning Path 1: Understanding AI Systems and Establishing Audit Scope
- Optional Hands-On Lab
- Module 2: How to Audit the Intricate Components of AI Applications
- Optional Lab: Microsoft Purview
- Optional Lab: Aequitas
- Module 3: Investigating Internal AI Usage – Governance
- Hands-On Lab: Reviewing Copilot Activity and Prompt Trails
Learning Path 2: Structuring Risk-Base AI Engagements
- Module 4: Redefining Audit Engagement Across the AI Lifecycle
- Hands-On Lab (Optional): Conducting a Prompt-Based AI Audit
Learning Path 3: Executing Fieldwork Across the AI Lifecycle
- Module 5: Execute AI Project Management Efficiently
- Hands-On Lab: Reviewing AI Project Governance Templates
- Module 6: Monitoring AI Systems and Governance in Action
- Optional Lab: Investigating Copilot and Purview Logs (Demo)
- AI Governance, Monitoring, and Capstone Execution
Learning Path 4: Assessing Maturity, Governance, and Strategic Closure
- Module 7: Designing Governance in New AI and GenAI Applications
- Optional Lab: Assigning AI and Data Governance Roles
- Module 8: Auditing AI Improvement Cycles and Profile-Driven Risk Tailoring
- Lab: Auditing Evidence of AI Governance Improvement.
- Module 9: Administering Trust and Accountability in Emerging AI Platforms
- Lab: Privacy Trust Assessment
Course Delivery Options
Train face-to-face with the live instructor. (Please note, not all classes will have this option)
Attend the live class from the comfort of your home or office.
Join us in person at our Denver or Reston training facilities! Learn alongside a live, remote instructor in our HD-equipped classrooms. We love having students on-site! An SLI sales rep can confirm availability and reserve your seat.
Access to on-demand training content anytime, anywhere. (Please note, not all classes will have this option)