The Challenge of Governing AI
The NIST Second Cyber AI Profile Workshop gathered over 1,400 comments from professionals grappling with AI adoption within existing risk frameworks. These questions aren't hypothetical; they're what risk managers, CISOs, and compliance leads are asking as they try to build governance structures for rapidly evolving AI systems. The workshop highlighted practical concerns about accountability, testing, and policy development that won't become obsolete in mere months.
Q1: Assigning Accountability for AI Decisions
Start with the NIST CSF's GOVERN function, which establishes roles and responsibilities. However, you can't delegate accountability to the AI itself. Organizations are forming multidisciplinary AI governance bodies and creating roles like chief AI officers. This is a step forward, but you must map specific decision points to specific people. If an AI-powered security tool blocks legitimate traffic, causing a service outage, who is responsible? The security team that deployed it? The vendor who trained it? The executive who approved the budget?
Document this before deployment. Create a RACI matrix (Responsible, Accountable, Consulted, Informed) that covers AI system decisions. The "Accountable" box can't be empty, and it can't say "the algorithm."
Human-in-the-loop processes are essential for AI accountability. Your governance framework should require human review at decision points that carry material risk, even if it slows things down.
Q2: Verifying AI Vendor Claims of Security
When an AI vendor claims their model is "secure," it often lacks specificity. The workshop highlighted the absence of a consistent AI taxonomy as a barrier to clear communication about risk.
Request an AI Bill of Materials (AIBOM) detailing the model's training data sources, dependencies, and supply chain components. Cryptographic signing and certification systems can help verify AI system integrity across the supply chain.
Test the system yourself. Don't rely on vendor assertions. Conduct adversarial testing, try prompt injection attacks if it's a language model, and check whether the system degrades predictably under load or fails catastrophically.
If the vendor won't provide technical documentation or allow independent testing, you're dealing with a black box, and you can't govern what you can't inspect.
Q3: Integrating AI into Existing Cybersecurity Frameworks
Integrate AI governance into your existing cybersecurity framework. Creating a separate AI policy can create silos and gaps.
NIST is developing the Cyber AI Profile to show how organizations can adapt the CSF 2.0 to manage AI-related risks. This approach treats AI as a technology layer within your existing risk framework, not as a separate entity requiring distinct governance.
Update your existing policies to address AI-specific considerations. Your incident response plan should cover AI system failures. Your third-party risk management process should include AI vendor assessments. Your access control policy should specify who can deploy AI tools.
This approach manages AI-specific risks within proven frameworks instead of inventing new ones.
Q4: Managing "Shadow AI" in Your Organization
Treat "shadow AI" like shadow IT, using visibility tools and clear acceptable use policies.
Deploy network monitoring to detect API calls to common AI services. Review browser extension installations and check cloud spend for unexpected charges to AI platforms. You're looking for usage patterns, not trying to catch individual employees.
Create a path of least resistance for approved AI use. If your team needs AI tools to do their jobs and you've banned everything, they'll work around you. Provide approved alternatives with appropriate controls: data loss prevention, usage logging, and clear boundaries about what data can be processed.
Your acceptable use policy should explicitly address AI tools. Specify what's allowed, what requires approval, and what's prohibited.
Q5: Implementing Human-in-the-Loop for AI Security Decisions
Human-in-the-loop (HITL) means a qualified human reviews and approves decisions before they execute or can intervene when the system's confidence is low.
Design decision workflows that keep humans engaged. For example, if your AI system flags suspicious network traffic, HITL means the system alerts a security analyst who reviews the context and decides whether to block. Partial automation might allow the system to block automatically for high-confidence threats but escalate borderline cases.
Your implementation depends on risk tolerance and response time requirements. For decisions with high business impact or unclear context, require human approval. For repetitive, low-risk decisions where the AI has proven reliable, allow automation with human oversight through audit logs and periodic review.
Train your team. HITL fails if the person in the loop doesn't understand what the AI is doing or reflexively approves every recommendation. Your training should cover how the AI makes decisions, its limitations, and when to override it.
Q6: Writing AI Governance Policies That Last
Focus on principles and decision frameworks, not specific technologies or controls.
Avoid overly specific guidelines that become obsolete as technology evolves. For example, instead of specifying a particular AI model architecture, establish a principle like "AI systems processing sensitive data must use deterministic outputs where possible."
Structure your policies around the NIST CSF Core Functions: Govern, Identify, Protect, Detect, Respond, Recover. These functions are technology-agnostic. Your AI governance policy should specify how you'll identify AI-related risks, protect against them, detect incidents, and respond when things go wrong. The specific tools and techniques can change; the functions don't.
Review and update policies on a defined schedule, but design them to flex with technological change.
Next Steps
NIST is developing the Initial Public Draft of the Cyber AI Profile based on workshop feedback. Join their Community of Interest at [email protected] to receive updates on working sessions and draft releases.
Review the NIST AI Risk Management Framework (AI RMF) for foundational risk concepts. Map your existing CSF 2.0 implementation to AI-specific considerations using the three Focus Areas NIST has outlined: AI systems you develop, AI systems you procure, and AI systems used against you.
Engage with your peers. The 1,400+ comments NIST received show that everyone's figuring this out in real time. You're not behind; you're building governance frameworks for technology that didn't exist when you learned risk management. Share what works and what doesn't.
