BXUP Track progress & test yourself — free
Security & AI Security · Module · ~14 min read
SECURITY & AI SECURITY · MODULE

AI Governance, Risk & Compliance

This module installs the skill that makes technical AI security provable: writing the register row that cites a framework, a control, a threat category and a legal obligation at once, and attaching the test evidence that keeps it true between audits.

Governance is where every finding from the red-team, every guardrail from the attack-surface modules, and every AI feature the org ships becomes something you can defend to an auditor, a board, and a regulator. This module maps your AI risk to the frameworks enterprises actually adopt — NIST AI RMF, ISO/IEC 42001, and the OWASP LLM Top 10 — secures the AI supply chain, turns policy into an enforceable acceptable-use document, and grounds all of it in India's real obligations: the DPDP Act, the 2025 Rules, and CERT-In's six-hour clock.

The scenario — the audit question with no evidence behind it

  1. THE GRC LEAD

    Meera inherited "AI governance" when her company — a mid-size firm processing customer data for BFSI clients — started shipping LLM features.

  2. AUG 4

    The ISO/IEC 42001 pre-assessment lands on her desk, driven not by ambition but by a Fortune-500 client's vendor questionnaire that now asks, in plain text, "Is your AI management system certified to ISO 42001?"

  3. 10:00

    The auditor's opening request is disarmingly simple: "Show me your AI risk register, and for your customer-support LLM, show me how you identified, measured, and are managing the risk of it disclosing personal data."

  4. THE GAP

    Meera has a spreadsheet of AI tools. She does not have a register mapped to a framework, she cannot point to a Measure activity, and when the auditor asks who signed off on the acceptable use of these tools, the answer is nobody.

  5. AUG 5

    She stops treating governance as documentation and starts treating it as a control system, mapping the support LLM to the NIST AI RMF Generative AI Profile (NIST AI 600-1) across its four functions — Govern, Map, Measure, Manage. The Measure gap immediately has an answer: Rohan's red-team suite from M7, with its ATLAS-mapped findings and a measured 5% leakage threshold, is the Measure evidence.

  6. AUG 7

    The risk register exists: each row carries a NIST function, an ISO 42001 Annex A control, the OWASP LLM Top 10 category, and — for the rows touching personal data — the DPDP obligation it implicates.

  7. AUG 8

    She drafts a one-page AI acceptable-use policy: which tools are approved, what data may never be pasted into them, the redaction rule for client data, and the human-review requirement for AI output that reaches a customer. She adds the two clocks every Indian security team must reconcile: CERT-In's 6-hour incident-reporting rule and the Data Protection Board's 72-hour personal-data-breach rule.

  8. AUG 11

    The auditor returns. The same question — "show me how you manage disclosure risk" — has a traceable answer: a register row citing NIST Measure 2.7, ISO 42001 control A.6.2, OWASP LLM02, and DPDP's security-safeguard obligation, backed by a red-team report with a threshold and a trend.

Governance stopped being a binder and became the layer that makes everything else auditable.

Key Takeaway:

Governance is not paperwork about the work — it is the layer that makes the technical work provable. A guardrail you cannot cite to a framework and evidence with a test is, to an auditor, a guardrail that does not exist.

The four frameworks — what each one is for

The common mistake is picking one framework and treating the others as competitors. They do different jobs and stack cleanly:

  • NIST AI RMF + GenAI Profile (NIST AI 600-1)a voluntary US framework organized into four functions (Govern, Map, Measure, Manage). The GenAI Profile adds twelve generative-AI-specific risks. It answers "what is our process for managing AI risk?" Free to adopt, no certification.
  • ISO/IEC 42001the certifiable AI Management System (AIMS) standard, structured like ISO 27001 with Annex A controls. It answers "can a third party audit and certify that our process runs?" This is the one clients ask for by name.
  • OWASP LLM Top 10 (2025)the technical risk taxonomy (LLM01 prompt injection through LLM10 unbounded consumption). It answers "what specifically can go wrong in an LLM app?" It is where your register rows get their concrete threat.
  • MITRE ATLASthe adversary-technique taxonomy (from M7). It answers "how would an attacker actually do it?" and links findings to mitigations.

NIST gives you the process, ISO 42001 makes it certifiable, OWASP names the technical risks you populate the process with, and ATLAS ties each to adversary behavior. You run all four; they reference one another.

The reason enterprises reach for ISO 42001 first is that it is the one a client can demand and verify. Its structure mirrors ISO 27001 — a management system with a plan-do-check-act cycle, a Statement of Applicability, internal audits, management review, and a set of Annex A controls (AI policy, roles, impact assessment, data management, third-party and lifecycle controls) that an external certification body checks. You are not certified against "AI safety" in the abstract; you are certified that a documented, running management system exists and that you can produce evidence for each control you claimed applies. That distinction — a running system with evidence, not a binder of intentions — is the whole game, and it is why the M7 red-team program (which produces continuous evidence) is so load-bearing here.

Exam Tip

ISO 42001 is the only one of the four you can be certified against — it defines an auditable management system. NIST AI RMF, OWASP LLM Top 10, and MITRE ATLAS are voluntary references you map to, not certify against.

The framework-mapping table

A register row is only defensible when it speaks all four languages at once. This is the mapping you build once and reuse for every AI feature — the single most useful artifact in the module.

Risk in your appNIST AI RMF (function · sub-category)ISO/IEC 42001 (Annex A)OWASP LLM Top 10India DPDP obligation
LLM discloses personal data in outputMeasure 2.7 · Manage 2.2A.6.2 (AI system impact assessment), A.8.3 (data for AI systems)LLM02 Sensitive Info DisclosureSec 8 security safeguards; purpose limitation
Prompt injection subverts guardrailsMap 1.1 · Measure 2.6A.6.1 (AI lifecycle), A.9.2 (responsible use)LLM01 Prompt InjectionSec 8 safeguards; breach reporting if data exposed
Poisoned/untrusted training or RAG dataMap 3.1 · Measure 2.8A.7.4 (data quality for AI)LLM04 Data & Model PoisoningSec 8; accuracy of processing
Compromised third-party model/dependencyGovern 6.1 · Map 4.1A.10.2 (third-party & supplier), A.7.2 (data provenance)LLM03 Supply ChainData-processor/sub-processor obligations
Unbounded consumption / cost-DoSManage 2.3A.9.3 (operational controls)LLM10 Unbounded ConsumptionAvailability limb of security safeguards
Excessive agent autonomy / agencyGovern 1.3 · Manage 4.1A.9.2 (responsible use)LLM06 Excessive AgencyAccountability of Data Fiduciary

The table is the deliverable auditors and clients actually want, because it proves you can locate any given risk in the process framework (NIST), the certification standard (ISO), the technical taxonomy (OWASP), and the law that applies to it (DPDP) simultaneously. Build it once per AI feature; each red-team finding from M7 drops into the OWASP column and inherits the rest of the row.

Mapping to NIST AI RMF — the four functions in practice

NIST's power is that it forces you to show activity, not intent. Walk the support LLM through the four functions:

  • Governwho owns AI risk, what the acceptable-use policy says, how autonomy tiers get approved. Evidence: named owner, signed AUP, tier-change log.
  • Mapwhat the system is, its context, and its risks. Evidence: the threat model (M6's STRIDE + lethal-trifecta review), the data-flow diagram, the OWASP-mapped risk list.
  • Measurehow you quantify the risk. Evidence: the M7 red-team program — Garak baseline, Promptfoo regression rate, the 5% leakage threshold and its trend. This is the function orgs most often cannot evidence, and it is exactly what the previous module produced.
  • Managehow you respond and prioritize. Evidence: the risk register with treatment decisions, the CI gate that blocks regressions, the incident runbook.

The GenAI Profile's twelve risks are a checklist you run Map against — for each, either a register row or a documented reason it does not apply. They include confabulation (the model stating false things confidently), dangerous or violent content, data privacy leakage, harmful bias and homogenization, human-AI configuration failures (over-trust or automation bias), information integrity (at-scale disinformation), information security (the LLM as an attack surface — where your M5–M7 work lives), intellectual-property exposure, obscene or abusive content, environmental impact, value-chain and component-integration risk (the supply-chain concern), and CBRN or dangerous-capability information.

The point of the checklist is not to have a control for every one but to have considered every one — an auditor's favorite question is "which of the twelve did you decide are out of scope, and why?", and "we never looked at that" is the answer that fails the review.

The four functions are also not a waterfall; they cycle. A new red-team finding (Measure) can reopen the threat model (Map), which changes a treatment decision (Manage), which may need a new owner or policy line (Govern). Wiring the M7 gate into CI is what keeps Measure alive between audits instead of being a snapshot you scramble to refresh the week before the assessor arrives.

Remember:

NIST Measure is where most AI governance programs are hollow — they have policies (Govern) and threat models (Map) but no numbers. Your red-team suite from M7 is the Measure evidence; wire it in explicitly.

Securing the AI supply chain

The register row for LLM03 needs real controls behind it, and the supply chain for an AI system is wider than for ordinary software: it includes the model weights, the training data provenance, the fine-tuning datasets, and every dependency in the serving stack.

# ai-bom.yaml — an SBOM-for-AI entry per deployed model
model:
  name: support-agent-llm
  source: "vendor-x/model-7b"           # provenance: where weights came from
  version: "2026.06"
  license: "apache-2.0"
  model_card: "docs/cards/support-agent.md"   # intended use, evals, limits
  integrity: "sha256:0f3c...e91a"        # verify weights on pull
  data_provenance: "vendor-disclosed; no PII in training set (attested)"
  approved_by: "elena@company; 2026-08-06"
dependencies:
  scanned_with: ["trivy", "pip-audit"]   # dep CVE scanning in CI
  last_scan: "2026-08-07"

Three controls make the supply chain defensible:

  • Model provenance and integrityyou know where the weights came from, you pin a version, and you verify a hash on pull so a swapped model is caught.
  • A model cardthe vendor's or your own, documenting intended use, eval results, and known limitations, which is both an ISO 42001 A.10.2 expectation and the thing your risk assessment reads.
  • Dependency scanning in CITrivy, pip-audit, or equivalents (the same tools from AppSec) run against the serving stack, because an LLM app is still a Python app with a CVE surface.

For services companies, add the sub-processor question: if a client's personal data flows through a hosted model, that model provider is a DPDP data processor and must be in your contracts and your AI-BOM.

The AI supply chain has failure modes ordinary software does not, and the AI-BOM is where you record that you considered them. A model pulled from a public hub can be a look-alike with a poisoned fine-tune (a typosquat on the model name), so provenance means the specific namespace and hash, not just "the 7B model." Training-data provenance you often cannot fully verify — the honest AI-BOM records what the vendor attested rather than what you proved, and that attestation is itself the artifact an auditor inspects. And the OWASP LLM03 row on your mapping table is not closed by the AI-BOM alone: it is closed by the AI-BOM plus a documented cadence for re-scanning dependencies and re-checking the model card when you bump a version, because supply-chain risk is continuous, not a one-time checkbox at adoption.

Pro Tip

Treat model weights like any other untrusted dependency — pin a version, verify a hash, keep a model card, and record who approved the adoption. "We use vendor-x's model" is not a control; the AI-BOM row that pins and attests it is.

Writing the acceptable-use policy

A policy nobody can act on is not a control. The AUP must be specific enough that an engineer knows, without asking, whether pasting a given payload into a given tool is allowed.

## AI Acceptable-Use Policy (extract)

1. APPROVED TOOLS. Only tools on the AI Tool Register (Annex A) are
   approved. Using any other AI tool for company or client work requires
   written GRC approval.

2. DATA CLASSIFICATION. NEVER input into any public/consumer AI tool:
   - Client data of any kind, personal data, or PII
   - Credentials, secrets, source code under a client MSA
   Public-tier tools are permitted ONLY for public, non-personal data.

3. CLIENT & PERSONAL DATA. Client or personal data may be processed only
   via approved enterprise/on-prem endpoints (e.g. local models), after
   redaction/data-minimization, and only for the consented purpose.

4. HUMAN REVIEW. AI output reaching a customer or a production system
   must be reviewed by a named human. AI is assistive, not authoritative.

5. INCIDENT REPORTING. Suspected AI-related security incident or personal-
   data breach → report to security within 1 hour (see CERT-In / DPB clocks).

The policy earns its keep through four moves:

  • A closed list of approved toolsdefault-deny, not default-allow.
  • A data-classification rulenaming what may never be entered; the single most important line for Indian services firms bound by client MSAs.
  • A compliance-first pathsensitive data goes through enterprise or local endpoints, after redaction.
  • A human-review requirementfor any consequential output.

Tie clause 1 to your AI-BOM and clause 5 to the reporting clocks below, and the AUP stops being aspirational.

India: DPDP and CERT-In obligations

Two frameworks bind Indian operations concretely, and the trap is the two different breach clocks.

The Digital Personal Data Protection Act 2023, with the DPDP Rules 2025 (notified by MeitY on 14 November 2025), phases in: the Data Protection Board is live, Consent Manager registration opens 13 November 2026, and substantive Data Fiduciary obligations become enforceable from 13 May 2027, with penalties up to INR 250 crore. For AI usage this means three things: processing personal data through an AI tool must respect purpose limitation (you cannot repurpose data into a model beyond the consented use), must meet security safeguards (Sec 8 — your guardrails and red-team program are evidence of this), and if the AI feature is high-risk, Significant Data Fiduciary obligations (a DPIA-like Data Protection Impact Assessment, plus periodic audit) apply.

Three DPDP intersections bite hardest on AI systems, and each needs a concrete control:

  • Consent and purpose limitationif personal data was collected for support, feeding it into a model to train or fine-tune a different feature is a new purpose that needs its own consent; the AUP's data-classification rule and your AI-BOM's data_provenance field are how you evidence you did not.
  • Data-principal rightsa person can request erasure, but a model that memorized their data cannot simply "forget"; the defensible posture is to keep personal data in the retrieval layer (deletable) rather than the weights, so a deletion request is a database operation, not a retraining project.
  • Cross-border and processor chainsa hosted model that sees personal data is a data processor, and for BFSI-sector clients, data-residency terms in the MSA may forbid that data leaving India at all, which is the practical reason the compliance-first path routes sensitive data through on-prem or local models.

The two clocks you must reconcile:

ObligationClockWhoTrigger
CERT-In incident reporting6 hoursCERT-In (IT Act)Notice of a listed cyber incident (incl. AI systems)
DPDP personal-data breach72 hours (to the Board) + intimation to affected principalsData Protection BoardAny personal-data breach

An AI incident that exposes personal data trips both: the 6-hour CERT-In clock for the cyber incident and the DPDP breach obligation to the Board and the affected individuals. Your incident runbook must fire both, in the right order, or you are compliant with neither. Because India has no standalone AI law yet — governance is policy-driven and sector-specific — ISO 42001 and NIST AI RMF are the practical control frameworks Indian enterprises adopt voluntarily, very often because a Fortune-500 client's vendor requirements demand it.

Warning

The 6-hour CERT-In clock and the 72-hour DPDP Board clock are different obligations with different triggers — reconcile them in one runbook. A team that reports the data breach in 72 hours but misses the 6-hour CERT-In window has still failed compliance.

Where this lands in your week

This module is the layer that makes the whole track defensible to someone outside your team. You mapped AI risk to the four frameworks that matter — NIST AI RMF for process, ISO/IEC 42001 for certification, OWASP LLM Top 10 for the technical taxonomy, MITRE ATLAS for adversary behavior — built the one mapping table that speaks all four languages plus DPDP at once, secured the AI supply chain with an AI-BOM (provenance, model cards, dependency scanning), turned policy into an enforceable acceptable-use document, and reconciled India's two breach clocks. The recurring theme: every guardrail and red-team finding from earlier modules only counts once it is cited to a framework and backed by evidence.

It anchors Lab 15 — AI risk register + acceptable-use policy (UC-58, UC-60): take one real AI feature, map it to the NIST AI RMF GenAI Profile and the OWASP LLM Top 10, cite the ISO 42001 control and the DPDP obligation on each row, and draft a one-page ISO-42001-aligned, DPDP-aware AI acceptable-use policy — artifact: the completed risk register plus the AUP, with the M7 red-team report attached as the NIST Measure evidence. The register you build here is the direct answer to the vendor questionnaire and audit question that opened this module, and it is the single strongest exhibit you carry into a GRC-into-AI-governance interview: not "we care about AI risk," but a traceable row proving you can locate, measure, and manage it.

Your playbooks & retrieval check

Pro Tip

Your first 15 minutes: take one AI guardrail your team already relies on and try to write its single register row — NIST function, ISO 42001 control, OWASP LLM category, evidence link — and if any cell is blank, that blank is the gap an auditor finds first.

Check yourself: An engineer shows you a working output filter that blocks PII leakage in your customer-support LLM. To an auditor, why might that guardrail still "not exist," and what turns it into evidence?

▸Show answeroptional deep dive — click to expand

A control an auditor cannot trace to a framework and cannot see tested is, to them, indistinguishable from a control that does not exist — governance is the layer that makes technical work provable, not the paperwork around it. The filter becomes evidence when it is a risk-register row that cites the NIST AI RMF Measure activity, the ISO/IEC 42001 Annex A control, the OWASP LLM Top 10 category, and — because it touches personal data — the DPDP security-safeguard obligation, all backed by a red-team report with a measured threshold and a trend. A guardrail you can cite to a framework and evidence with a test is real; one you cannot is, to an auditor, absent.

Companion playbooks — take these into your week alongside Lab 15: pb-47 (Map an AI System to the NIST AI RMF GenAI Profile), pb-49 (Build an OWASP LLM Top 10 Risk Register for Your AI App), and pb-51 (Author an Enforceable AI Acceptable-Use Policy).

This is the free reading view. Open BXUP to track your progress, run the hands-on labs, and prove it with original, exam-pattern assessments.
© Anvi Innovate · BXUP — Be Exponentially Upskilled · BXUP is an independent learning platform, not affiliated with or endorsed by AWS, Microsoft, Google, NVIDIA, IBM, Salesforce, Databricks, Oracle or Anthropic. All trademarks belong to their respective owners. · bxup.in