Building a Gen AI Security Framework, Part 4: A Complete Worked Example

Part 4 of 4 in a series on building a practical, regulator-ready risk framework for generative AI.

Executive Summary

• The use case is an internal, retrieval-based chatbot deployed to financial advisors and relationship managers, connected to a knowledge base containing customer account data, transaction histories, and internal policy documents.
• Applying the three-question triage from Part 2 classifies this system as an internal, retrieval-augmented generation use case handling high-sensitivity data, which pulls in a known baseline risk profile instead of starting from a blank page.
• The threat model identifies three concrete scenarios drawn from NIST AI 600-1, OWASP LLM Top 10, and MITRE ATLAS techniques, rather than treating those lists as abstract references.
• The top risk, unauthorized disclosure of another customer’s financial data through prompt injection, is rated three ways: qualitative (High), semi-quantitative (a score of 20 out of 25), and quantitative, with a worked annualized loss expectancy of 360,000 dollars per year, anchored to a published industry cost figure.
• Each rated risk is mapped to specific SP 800-53 control families, with a completed evidence record showing what would actually sit in an audit file, not just a policy statement.
• The series closes with a practical starting point: apply this same sequence to one real use case in your own inventory before trying to build the full pattern library at once.

1. The Use Case

The use case is an internal chatbot, referred to here as Advisor Insight, built by a mid-sized financial institution to help its advisors and relationship managers answer client questions faster. The system uses retrieval-augmented generation: when an advisor asks a question, the system searches a knowledge base and feeds relevant material to a language model, which generates a response grounded in that retrieved content.

The knowledge base includes three types of material: customer account summaries and transaction histories pulled nightly from the core banking system, product terms and rate sheets, and internal credit and lending policy documents. Access is limited to employees, roughly 2,000 advisors, relationship managers, and contact center staff, authenticated through the institution’s existing single sign-on system. The chatbot is not exposed to customers and has no connection to the public internet.

Advisor Insight does not take any action on its own. It answers questions and generates draft text. An advisor still has to act on what it says, whether that means quoting a rate to a client or updating an account record through a separate system. This distinction matters for what comes next.

2. Pattern Classification and Triage

Part 2 introduced a three-question triage so a new use case could be routed to a known pattern instead of requiring a full manual assessment before anyone even knows how much scrutiny it needs.

Triage question	Answer for Advisor Insight
Does the system access sensitive, regulated, or confidential data?	Yes. Customer account numbers, balances, transaction history, and credit information.
Does the system take autonomous action, or does it only generate content for human review?	No. It only generates responses. Advisors act on the information separately.
Is the system exposed to external users, or only to internal staff?	Internal only. No customer or public access.

These three answers route Advisor Insight into the internal knowledge assistant pattern from Part 2. That pattern carries a default risk tier of Low to Medium. The first triage answer overrides that default: the knowledge base contains regulated financial data and personal information, which elevates the baseline to High before any scenario-specific analysis begins. Instead of starting the Map exercise from nothing, the analyst starts from that elevated baseline and confirms or adjusts it against the specific details of this system.

3. Threat Model

A pattern-level baseline tells you what to look for. A threat model turns that into specific, testable scenarios for this exact system.

Scenario 1: Unauthorized disclosure across account boundaries. An advisor, whether out of curiosity or malicious intent, phrases a question in a way that causes the retrieval system to surface account details for a customer outside that advisor’s assigned book of business. This maps to Data Privacy in AI 600-1 and to Sensitive Information Disclosure (LLM02) in the OWASP Top 10. MITRE ATLAS documents the underlying technique as LLM Prompt Injection (AML.T0051), with a direct sub-technique covering a crafted prompt that causes the model to disregard its intended boundaries. The objective of this scenario, unauthorized data leaving the system’s intended access boundary, corresponds to the Exfiltration tactic (AML.TA0010) in ATLAS.

Scenario 2: Poisoned retrieval content. The knowledge base is refreshed nightly from internal document repositories. If a document with embedded instructions is placed into one of those repositories, whether through a compromised upload process or an insider, the retrieval step can pull that content into the model’s context and alter its behavior for any advisor who later asks a related question. This is the indirect sub-technique of AML.T0051 and corresponds to Data and Model Poisoning (LLM04) in the OWASP Top 10. Unlike Scenario 1, this failure mode does not require the attacker to be a system user at all.

Scenario 3: Confabulation on a lending or product question. The system generates a confident but incorrect summary of a policy exception or a product’s terms, and the advisor repeats it to a client without independently verifying it. This maps to Confabulation and Human-AI Configuration in AI 600-1, and to Misinformation (LLM09) in the OWASP Top 10. Unlike the first two scenarios, this one requires no attacker at all. It happens through ordinary use.

4. SP 800-30 Risk Rating

Each scenario gets rated for likelihood and impact, following the SP 800-30 method introduced in Part 1. The table below shows the qualitative and semi-quantitative ratings for all three.

Scenario	Likelihood (qual.)	Impact (qual.)	Overall (qual.)	Likelihood score	Impact score	Combined score
1: Cross-boundary disclosure	High	High	High	4	5	20
2: Poisoned retrieval content	Low	High	Moderate	2	5	10
3: Confabulation on policy terms	Moderate	Moderate	Moderate	3	3	9

Scenario 1 rates High on likelihood because the system serves 2,000 internal users, retrieval is not yet scoped to an advisor’s specific book of business, prompt injection is a well-documented technique, and there is no output filtering in place at this stage of the build. Impact rates High because the exposed data is regulated financial and personal information, and a confirmed cross-boundary disclosure would trigger internal incident response and likely a compliance review, independent of whether external notification requirements apply in a given case.

Scenario 1 is the highest-priority risk, so it gets the full quantitative treatment.

Quantitative rating for Scenario 1. A single loss expectancy figure needs an anchor. IBM’s 2026 Cost of a Data Breach Report, published July 29, 2026, puts the average cost of a data breach in the financial services industry at 6.3 million dollars, and separately reports that AI-enabled breaches averaged 6 million dollars globally, about 1 million dollars above the all-industry average. Those figures describe full-scale breaches, typically involving a large number of records and often external disclosure. Scenario 1 as described is narrower: one advisor viewing one other customer’s data, caught by monitoring before it spreads further. Using the industry figure directly would overstate this specific scenario.

A more grounded single loss expectancy for a contained incident of this kind is built from its actual components: internal investigation and containment, typically in the range of 40,000 to 60,000 dollars for a review of this scope; remediation, meaning the engineering work to add proper access scoping to the retrieval system, roughly 50,000 to 70,000 dollars; and compliance review and potential regulatory inquiry response, roughly 60,000 to 80,000 dollars. Summing the midpoints gives a single loss expectancy of approximately 180,000 dollars per incident. The IBM figures are the reason this number is taken seriously rather than dismissed as a rare edge case: they establish what happens when a similar control gap is not caught early and scales into a full breach.

For the annualized rate of occurrence, an estimate of 2 incidents per year is reasonable given the current state of the system: broad internal access, no per-advisor retrieval scoping, and a known, publicly documented attack technique. That is an estimate the institution should refine using its own red team testing results once available, not treat as a fixed number.

Annualized loss expectancy: 180,000 dollars multiplied by 2 equals 360,000 dollars per year in expected loss if Scenario 1 is left unaddressed.

That figure can now be compared directly to the cost of a mitigating control. Adding permission-aware retrieval, so that the system can only retrieve records within an advisor’s assigned book of business, along with expanded query logging, is estimated at roughly 75,000 dollars a year to build and maintain. An expected loss of 360,000 dollars against a control cost of 75,000 dollars is a straightforward case to bring to a budget conversation, in a way that a qualitative High rating alone is not.

5. Control Mapping and Evidence Record

Using the mapping approach from Part 3, each scenario is assigned to specific SP 800-53 control families, a CSF 2.0 function, and a completed evidence record.

Scenario	SP 800-53 families	CSF 2.0	Evidence	Review trigger
1: Cross-boundary disclosure	Access Control, System and Information Integrity	Protect	Permission-aware retrieval configuration, quarterly access recertification records, red team test results against AML.T0051 direct injection	Any change to advisor territory assignments, or a new class of user added to the system
2: Poisoned retrieval content	System and Information Integrity, Supply Chain Risk Management	Detect	Content integrity checks on documents entering the knowledge base, version history for the document repository, anomaly monitoring on retrieval outputs	Any change to the document ingestion pipeline or its upstream sources
3: Confabulation on policy terms	System and Information Integrity, Planning	Protect	Accuracy testing results against a benchmark set of policy questions, a citation requirement so every answer links to its source document, a documented human review step for lending and pricing questions	Any update to lending or pricing policy documents, or a model version change

This table is what an examiner or an internal auditor would actually ask to see, not the risk register entry alone. The register from Part 2 says Scenario 1 is rated High. This table says what was done about it, who is responsible for the evidence, and what event should trigger another look. Put together, the register entry and this table are the difference between a documented concern and a managed risk.

Conclusion

This series set out to answer a specific question: how does an organization go from being aware that generative AI carries risk to being able to demonstrate, with evidence, that the risk is managed. Part 1 supplied the vocabulary and the rating method. Part 2 supplied the means to apply that method at scale without repeating full analysis for every new system. Part 3 supplied the bridge from a rated risk to an actual control and the proof that the control works. This part carried one real use case through every step of that chain, with real numbers attached at each stage rather than leaving the arithmetic as an exercise.

The practical next step is not to build a complete pattern library and a full risk register in one pass. It is to take one real gen AI use case already running inside the organization and carry it through these same four steps: classify it, model its specific threats, rate them three ways, and map the top risks to controls with an evidence record. That single exercise, done properly on one system, teaches more about where an organization’s actual gaps are than a broad survey of every use case done shallowly. The framework scales once the first pass is done well.

References

1. NIST. Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, January 2023. https://www.nist.gov/itl/ai-risk-management-framework
2. NIST. Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, July 2024. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf
3. NIST. Guide for Conducting Risk Assessments, SP 800-30 Revision 1. https://csrc.nist.gov/pubs/sp/800/30/r1/final
4. NIST. Security and Privacy Controls for Information Systems and Organizations, SP 800-53 Revision 5. https://csrc.nist.gov/pubs/sp/800/53/r5/upd1/final
5. NIST. The NIST Cybersecurity Framework (CSF) 2.0, NIST CSWP 29, February 2024. https://www.nist.gov/cyberframework
6. ISO. ISO/IEC 27001:2022, Information security management systems. https://www.iso.org/standard/27001.html
7. ISO. ISO/IEC 42001:2023, Artificial intelligence management system. https://www.iso.org/standard/42001.html
8. OWASP Gen AI Security Project. OWASP Top 10 for LLM Applications, 2025. https://genai.owasp.org/llm-top-10/
9. MITRE. ATLAS, Adversarial Threat Landscape for Artificial-Intelligence Systems, including techniques and tactics. https://atlas.mitre.org/
10. IBM. IBM Study: One in Four Malicious Breaches are AI-Enabled, Costing Companies 6 Million Dollars on Average, July 29, 2026. https://newsroom.ibm.com/2026-07-29-ibm-study-one-in-four-malicious-breaches-are-ai-enabled,-costing-companies-6-million-on-average
11. IBM. Cost of a Data Breach Report 2026. https://www.ibm.com/reports/data-breach