An internal audit department notices that AI-generated audit reports are producing false conclusions. Which of the following is the BEST way to correct this issue?
Correct Answer: B
If AI-generated audit reports containfalse conclusions, the integrity of audit evidence and overall assurance is compromised. According to AAIA, when AI systems compromiseaccuracy of audit outputs, the immediate step should be tosuspend use(B) to prevent further incorrect or misleading reporting. This allows auditors to: * Investigate the root cause of incorrect conclusions * Validate the model's training data, rules, and prompt structures * Reassess controls, safeguards, and human review processes * Prevent reliance on unreliable AI-generated audit material Increasing context (A) or reducing creativity (C) may help generative models but do not guarantee correction of fundamental errors. Updating SLAs (D) is administrative and does not solve the immediate integrity issue. References: AAIA Domain 3: Audit Evidence, Professional Skepticism, and AI output validation. AAIA Domain 2: Monitoring AI Outputs for Accuracy.
AAIA Exam Question 77
In the context of an AI implementation, which of the following actions is MOST critical for an organization's change management program?
Correct Answer: C
The AAIA™ Study Guide emphasizes that AI implementations introduce dynamic and non-deterministic elements into systems, increasing the risk associated with changes. A comprehensive, AI-specific risk assessment is therefore the most critical component of a change management program to ensure that updates, retraining, or parameter adjustments do not introduce vulnerabilities or unintended consequences. "Risk assessments tailored to AI are crucial because changes to models, training data, or infrastructure can affect performance, ethical compliance, or expose the system to new threats. A standard IT change review is often insufficient." While having a governance committee (A) and reviewing documentation (B) are important supporting practices, only option C directly mitigates the core risks of AI system change. Ethics training (D) supports awareness but is not directly tied to change control. Reference: ISACA Advanced in AI Audit™ (AAIA™) Study Guide, Section: "AI Governance and Risk Management," Subsection: "Change Management and AI Risk Control"
AAIA Exam Question 78
An organization deploys an AI recruitment platform to screen job applicants. The IS auditor identifies that the platform's decisions may be influenced by model bias. Which of the following risk mitigation strategies is BEST for the auditor to recommend?
Correct Answer: A
Periodic testing and monitoring for bias is a sustainable, proactive strategy aligned with best practices outlined in the AAIA™ Study Guide. This approach ensures that the AI system remains compliant over time, even as data and hiring conditions change. "Ongoing fairness assessments help detect emerging biases and ensure that the AI model maintains equitable decision-making standards. Periodic testing also allows organizations to take corrective action before regulatory or reputational damage occurs." Suspending the system (B) or relying solely on external datasets (C) are temporary or limited in scope. Manual reviews (D) are effective but do not solve the root issue. Therefore, A provides a comprehensive, audit-aligned solution. Reference: ISACA Advanced in AI Audit™ (AAIA™) Study Guide, Section: "Ethical and Legal Considerations in AI," Subsection: "Bias Mitigation and Monitoring"
AAIA Exam Question 79
An organization uses an AI-powered tool to detect and respond to cybersecurity threats in real time. An IS auditor finds that the tool produces excessive false positives, increasing the workload of the security team. Which of the following techniques should the auditor recommend to BEST evaluate the tool's effectiveness in managing this issue?
Correct Answer: D
The AAIA™ Study Guide recommends using validation tools to fine-tune and evaluate ML models, particularly when high false positives undermine operational efficiency. ML validation can identify threshold adjustments, retraining needs, or feature misweighting contributing to excessive alerting. "Model validation enables organizations to quantify performance, reduce false alarms, and recalibrate AI behavior to align with operational needs and threat landscapes." While logs (A) and benchmarks (B) help with diagnosis, they don't improve the model. Penetration testing (C) evaluates detection, not alert noise. D is the most effective solution. Reference: ISACA Advanced in AI Audit™ (AAIA™) Study Guide, Section: "AI Operations and Performance," Subsection: "Model Tuning and False Positive Mitigation"
AAIA Exam Question 80
When using off-the-shelf AI models, which of the following is the MOST appropriate way for organizations to approach vendor management?
Correct Answer: B
When organizations leverage off-the-shelf AI models, effective vendor management is critical to ensure operational reliability, compliance, and long-term support. The ISACA Advanced in AI Audit™ (AAIA™) Study Guide highlights that the "establishment of clear contractual terms regarding responsibilities for ongoing model updates, maintenance, support, and incident response is essential for managing third-party AI risks." By clearly defining the roles and expectations for updates and support (option B), organizations reduce the risk of unaddressed vulnerabilities, outdated models, or unclear recourse in the event of an incident or system failure. This approach supports ongoing risk management and ensures that both parties understand their obligations throughout the model's lifecycle. While market research, vendor accreditation, and contract review by information security are important due diligence steps, they do not directly address the need for clarity in ongoing vendor responsibilities, which is critical for effective governance and sustained operation of AI solutions. Reference:ISACA Advanced in AI Audit™ (AAIA™) Study Guide, Section: "Vendor Management for AI Systems," Subsection: "Third-Party AI Risk and Contractual Obligations"
